linear conic programming theory andlinear conic programming theory and applications1 shu-cherng fang...
TRANSCRIPT
![Page 1: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/1.jpg)
Linear Conic Programming Theory andApplications1
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
1Fang, S. C., and W. Xing. “Linear Conic Programming” (2013).
Conic Programming 1 / 25
![Page 2: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/2.jpg)
Content• Part I : Introduction• Part II: Preliminary and Convex Cone Structure• Part III: Duality Theory of Linear Conic Programming• Part IV: Interior Point Methods and Solution Software• Part V: Modelling and Applications• Part VI: Recent Research• Part VII: Practical LCoP
Conic Programming 2 / 25
![Page 3: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/3.jpg)
LCoP Part I – Introduction
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
Conic Programming 3 / 25
![Page 4: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/4.jpg)
Introduction
Content• Linear Conic Programs• Applications• Duality Theory and Algorithms• References
Conic Programming 4 / 25
![Page 5: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/5.jpg)
Linear Conic Program
Min C •Xs.t. A •X = B
X ∈ K(LCoP)
where K is a closed, convex cone; A, B and C are in the space ofinterests with • being an appropriate linear operator.
Conic Programming 5 / 25
![Page 6: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/6.jpg)
K = Rn+ (First Orthant)
When K = Rn+ = {x ∈ Rn|xi ≥ 0, i = 1, ..., n}, LCoP becomes LP.
Min cTxs.t. Ax = b
x ∈ Rn+
(LP)
where A ∈ Rm×n, b ∈ Rm and c ∈ Rn.
Equivalently,
Min cTxs.t. aTi x = bi, i = 1, . . . ,m
x ≥Rn+
0(LP)
where ai ∈ Rn, bi ∈ R and c ∈ Rn.
Conic Programming 6 / 25
![Page 7: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/7.jpg)
K = Rn+
Figure:R1+ Figure:R2
+ Figure:R3+
Conic Programming 7 / 25
![Page 8: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/8.jpg)
K = Ln (Lorentz Cone/Second OrderCone)
When K = Ln = {x ∈ Rn|√x21 + · · ·+ x2n−1 ≤ xn}, LCoP becomes
SOCP.Min cTxs.t. Ax = b
x ∈ Ln(SOCP)
where A ∈ Rm×n, b ∈ Rm and c ∈ Rn.
Equivalently,
Min cTxs.t. aTi x = bi, i = 1, . . . ,m
x ≥Ln 0(SOCP)
where ai ∈ Rn, bi ∈ R and c ∈ Rn.
Conic Programming 8 / 25
![Page 9: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/9.jpg)
K = Ln
Figure: L2 Figure: L3
Conic Programming 9 / 25
![Page 10: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/10.jpg)
K = Sn+ (Positive Semidefinite Cone)
When K = Sn+ = {X ∈ Rn×n|X = XT � 0}, LCoP becomes SDP.
Min C •Xs.t. Ai •X = bi, i = 1, ...,m
X ∈ Sn+(SDP)
where C,A1, ..., Am are given n× n symmetric matrices and b1, ..., bm aregiven scalars, and
M •X =∑i,j
MijXij = tr(MTX).
Equivalently,
Min C •Xs.t. Ai •X = bi, i = 1, ...,m
X � 0(SDP)
Conic Programming 10 / 25
![Page 11: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/11.jpg)
K = Sn+
S2+ =
{(x, y, z) ∈ R3|
[x yy z
]� 0.
}⇐⇒ x ≥ 0, z ≥ 0, xz ≥ y2.
Conic Programming 11 / 25
![Page 12: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/12.jpg)
Application of SOCP – I
Torricelli Point ProblemThe problem is proposed by Pierre de Fermat in 17th century. Giventhree points a, b and c on the R2 plane, find the point in the plane thatminimizes the total distance to the three given points. The solutionmethod was found by Torricelli, hence know as Torricelli point.
SOCP Formulation
Min t1 + t2 + t3
s.t.
[x− at1
]∈ L3,
[x− bt2
]∈ L3,
[x− ct3
]∈ L3
Question:C =? A =? X =? · · ·
Conic Programming 12 / 25
![Page 13: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/13.jpg)
Application of SOCP – II
Robust Portfolio DesignAssume returns r are known within an ellipsoid
E = {r = r + κΣ1/2u : ‖u‖2 ≤ 1}.
where r is the expected return, Σ is the empirical covariance matrix,0 < κ < 1 is a given constant.
robust counterpart: (optimize the worst case)
maxω
minr∈E{rTω : eTω = 1, ω ≥ 0}.
Conic Programming 13 / 25
![Page 14: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/14.jpg)
SOCP Formulation
Notice thatminr∈E
rTω
= min‖u‖2≤1
{rTω + κuT Σ1/2ω}
= rTω − κ‖Σ1/2ω‖2
Robust portfolio problem is an SOCP
Max rTω − κ‖Σ1/2ω‖2s.t. eTω = 1, ω ≥ 0
⇐⇒
Max ts.t. eTω = 1, ω ≥ 0[
κΣ1/2ωrTω − t
]∈ Ln+1
Question:C =? A =? X =? · · ·
Conic Programming 14 / 25
![Page 15: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/15.jpg)
Application of SDP – I
Correlation Matrix Verification
Consider three random variables A, B and C. By definition, theircorrelation coefficients ρAB , ρAC and ρBC are valid if and only if 1 ρAB ρAC
ρAB 1 ρBC
ρAC ρBC 1
� 0
Suppose we know from some prior knowledge (e.g. empirical results ofexperiments) that −0.2 ≤ ρAB ≤ −0.1 and 0.4 ≤ ρBC ≤ 0.5. What are thesmallest and largest values that ρAC can take?
Conic Programming 15 / 25
![Page 16: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/16.jpg)
SDP Formulation
The problem can be formulated as the following problem:
Min/Max ρAC
s.t. −0.2 ≤ ρAB ≤ −0.1
0.4 ≤ ρBC ≤ 0.5
ρAA = ρBB = ρCC = 1 ρAA ρAB ρAC
ρAB ρBB ρBC
ρAC ρBC ρCC
� 0
Conic Programming 16 / 25
![Page 17: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/17.jpg)
SDP Formulation
In order to formulate the problem as in standard form, we handle theinequality constraints by augmenting the variable matrix and introducingslack variables, for example
0 12 0 0 0 0 0
12 0 0 0 0 0 00 0 0 0 0 0 00 0 0 1 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0
•
ρAA ρAB ρAC 0 0 0 0ρAB ρBB ρBC 0 0 0 0ρAC ρBC ρCC 0 0 0 0
0 0 0 s1 0 0 00 0 0 0 s2 0 00 0 0 0 0 s3 00 0 0 0 0 0 s4
= ρAB + s1 = −0.1
Conic Programming 17 / 25
![Page 18: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/18.jpg)
SDP Formulation
X =
1 ρAB ρAC 0 0 0 0ρAB 1 ρBC 0 0 0 0ρAC ρBC 1 0 0 0 00 0 0 s1 0 0 00 0 0 0 s2 0 00 0 0 0 0 s3 00 0 0 0 0 0 s4
, C =
0 0 12
0 0 0 00 0 0 0 0 0 012
0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0
A1 =
0 12
0 0 0 0 012
0 0 0 0 0 00 0 0 0 0 0 00 0 0 1 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0
A2 =?, A3 =? and A4 =?
Conic Programming 18 / 25
![Page 19: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/19.jpg)
Other Applications - QCQP =⇒ SOCP
The popularity of SOCP is also due to the fact that it is a generalized formof convex QCQP (Quadratically Constrained Quadratic Programming).Specifically, consider the following QCQP:
Min xTA0x+ 2bT0 x+ c0s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m
where A0 � 0, Ai � 0 for i = 1, . . . ,m.Note that
t ≥n∑
i=1
x2i ⇐⇒
∣∣∣∣∣∣∣∣∣
∣∣∣∣∣∣∣∣∣
x1...xn
(t− 1)/2
∣∣∣∣∣∣∣∣∣
∣∣∣∣∣∣∣∣∣2
≤ t+ 1
2⇐⇒
x1...xn
(t− 1)/2(t+ 1)/2
∈ Ln+2
Conic Programming 19 / 25
![Page 20: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/20.jpg)
Other Applications - QCQP =⇒ SOCP
Therefore, for each i = 1, . . . ,m
xTAix+ 2bTi x+ ci ≤ 0⇐⇒
A1/2i x
−1/2− bTi x− ci/21/2− bTi x− ci/2
∈ Ln+2
QCQP can be equivalently written as
Min u
s.t.
A1/20 x
−1/2− bT0 x+ u/2− c0/21/2− bT0 x+ u/2− c0/2
∈ Ln+2
A1/2i x
−1/2− bTi x− ci/21/2− bTi x− ci/2
∈ Ln+2, i = 1, . . . ,m.
Conic Programming 20 / 25
![Page 21: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/21.jpg)
Other Applications - SOCP =⇒ SDP
Ln+1 can be easily embedded into Sn+1+ by observing the fact that[
xt
]∈ Ln+1 ⇐⇒
[t xT
x tIn
]∈ Sn+1
+
Based on this, we will focus on the theorems and algorithms for SDP. Butthis does not mean that SOCP is useless or we should transform SOCPto SDP in any case.
Conic Programming 21 / 25
![Page 22: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/22.jpg)
Duality Theory of LP
Theorem (Weak Duality Theorem of LP)If x is primal feasible and y is dual feasible, then cTx ≥ bT y.
Theorem (Strong Duality Theorem of LP)
• If either LP or LD has a finite optimal solution, then so does the otherand they achieve the same optimal objective value.
• If either LP or LD has an unbounded objective value, then the otherhas no feasible solution.
How about the duality theorems of LCoP?
Conic Programming 22 / 25
![Page 23: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/23.jpg)
Algorithms for LP
Simplex Method for LP
• Starting from one vertex• Check whether current vertex is optimal or not. If yes, stop.
Otherwise, go to the next step.• Move to a neighboring vertex, go to the previous step.
The complexity of simplex method is not polynomial.
Polynomial-time Algorithms
• Ellipsoid Method• Karmarkar’s Projective Scaling Algorithm• Affine Scaling Algorithm: Primal, Dual and Primal-Dual
How about the algorithms for LCoP?
Conic Programming 23 / 25
![Page 24: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/24.jpg)
References
Books• Bertsekas D.P., Nedic A. and Ozdaglar A.E., Convex Analysis and
Optimization, Athena Scientific: Belmont, MA USA 2003• Boyd S. and Vandenberghe L., Convex Optimization, Cambridge University
Press: Cambridge, UK 2004• Fang S. and Puthenpura S., Linear Optimization and Extensions: Theory
and Algorithms, Prentice-Hall Inc.: Englewood Cliffs, NJ USA 1993• Nemirovski A., Lectures on Modern Convex Optimization: Analysis,
Algorithms, and Engineering Applications, Society for Industrial and AppliedMathematics: Philadelphia, PA USA 2001
• Renegar J., A Mathematical View of Interior-point Methods in ConvexOptimization, Society for Industrial and Applied Mathematics: Philadelphia,PA USA 2001
• Handbook of Semidefinite Programming: Theory, Algorithms, andApplications, edited by Wolkowicz H., Saigal R. and Vandenberghe L.,Kluwer Academic Publisher: Norwell, MA USA 2000
Conic Programming 24 / 25
![Page 25: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/25.jpg)
References
• Rockafellar R.T., Convex Analysis, Princeton University Press:Princeton, NJ USA 1970
• Wright. S., Primal-Dual Interior-Point Methods, Society for Industrialand Applied Mathematics: Philadelphia, PA USA 1997
Others
• Ye Y., Linear Conic Programming, lecture notes online: http://www.stanford.edu/class/msande314/sdpmain.pdf
• Todd M.J., Semidefinite Programming, lecture notes online:http://people.orie.cornell.edu/~miketodd/cornellonly/or637/or637.html
A very popular general purpose SDP solver, SeDuMi, of Jos F. Sturm canbe found in: http://sedumi.ie.lehigh.edu/
Conic Programming 25 / 25
![Page 26: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/26.jpg)
LCoP Part II – Preliminaries and Convex ConeStructures
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
Conic Programming 1 / 39
![Page 27: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/27.jpg)
Preliminaries and Convex Cone Structures
Content• Vectors, Matrices, and Spaces• Inner Products and Norms• Open, Closed, Interior, and Boundary Sets• Functions• Linear Systems• Convex Sets and Functions
Conic Programming 2 / 39
![Page 28: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/28.jpg)
Vectors, Matrices and Spaces
• Real numbers: R, R+, R++
• Euclidean space: Rn
• First orthant: Rn+• n-dimensional (column) vector:
x = (x1, x2, . . . , xn)T
• Matrices space: Rm×n
• Matrix: M ∈ Rm×n, ith row Mi·, jth column M·j , ijth entry Mij(Mi,j)
• Symmetric square matrices space (n(n+ 1)/2-dimensional space):
Sn = {M ∈ Rn×n |M = MT }.
Conic Programming 3 / 39
![Page 29: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/29.jpg)
Vectors, Matrices and Spaces
Given M ∈ Rm×n, N ∈ Rn×m, S ∈ Rn×n
• Determinant: det(S)
• Trace: tr(S)tr(MN) = tr(NM)
• Null space: N (M)= {x ∈ Rn|Mx = 0}.• Range space: R(M)= {y ∈ Rm|y = Mx for some x ∈ Rn}.• Positive semidefinite matrix:
S � 0 ⇐⇒ zTSz ≥ 0, ∀ z ∈ Rn
• Positive definite matrix:
S � 0 ⇐⇒ zTSz > 0, ∀ z ∈ Rn and z 6= 0
Conic Programming 4 / 39
![Page 30: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/30.jpg)
Vectors, Matrices and Spaces
Theorem: (Schur complementary theorem)Given
X =
[A BBT C
]and S = C −BTA−1B,
if A � 0 thenX � (�)0⇔ S � (�)0
Conic Programming 5 / 39
![Page 31: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/31.jpg)
Inner Products and Norms
• Inner products:x • y = xT y =
∑i xiyi
X • Y = tr(XTY ) =∑i,j XijYij
• Norms:• Euclidean norm: ‖x‖2 =
√x • x
• p-norm: ‖x‖p = (∑n
i=1 |xi|p)1/p for p ≥ 1
• Infinity-norm: ‖x‖∞ = max{|x1|, . . . , |xn|}• Frobenius norm:
‖X‖F =√X •X =
√tr(XTX)
• Note that: xTAx = A • xxT
Conic Programming 6 / 39
![Page 32: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/32.jpg)
Open, Closed, Interior and Boundary Sets
• Neighborhood: N(x0; ε) = {x ∈ Rn| ‖x− x0‖ < ε}.
• Open: X ⊂ Rn is open if for any x ∈ X , there exists ε > 0 such thatN(x; ε) ⊂ X .
• Closed: X ⊂ Rn is closed, if Rn\X = {x ∈ Rn|x /∈ X} is open.
• Closure of a set X ⊂ Rn is the smallest closed set containing X andis denoted as cl(X ).
Conic Programming 7 / 39
![Page 33: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/33.jpg)
Open, Closed, Interior and Boundary Sets
• Interior: the interior of a given set X ⊂ Rn is
int(X ) = {x ∈ X |∃ εx > 0 such that N(x; εx) ⊂ X}
• Boundary of a set X ⊂ Rn:
bdry(X ) = cl(X )\int(X ) = {x ∈ cl(X )|x /∈ int(X )}
• Bounded: a set X ⊂ Rn is bounded if there exist an r > 0 such that
‖x‖ < r,∀x ∈ X
Conic Programming 8 / 39
![Page 34: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/34.jpg)
Functions
• Continuous: f : X ⊂ Rn is continuous at x0 if( i ) x0 ∈ X(i i) limx→x0 f(x) = f(x0)
• Continuous function: f ∈ C0(X ) means f is continuous at all pointsin X ⊂ Rn.
• Gradient: For f : X ⊂ Rn → R
∇f(x) = [∂f(x)
∂x1, · · · , ∂f(x)
∂xn]1×n
• Hessian: For f : X ⊂ Rn → R
F (x) = [∂2f(x)
∂xi∂xj]n×n
• Continuously differentiable function: f ∈ Cp(X ) (p = 1, 2, · · · ) meansf is p-th continuously differentiable over X ⊂ Rn.
Conic Programming 9 / 39
![Page 35: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/35.jpg)
Functions
Theorem (Taylor theorem)Let X be open, f ∈ Cp(X ), x1, x2 ∈ X , x1 6= x2 and
x(θ) = θx1 + (1− θ)x2 ∈ X , ∀ 0 ≤ θ ≤ 1.
Then ∃ x = θx1 + (1− θ)x2 ∈ X , 0 < θ < 1, s.t.
f(x2) = f(x1) +
p−1∑k=1
1
k!dkf(x1;x2 − x1) +
1
p!dpf(x;x2 − x1)
where dkf(x;h) is the k-th order differential of function f along h.
Conic Programming 10 / 39
![Page 36: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/36.jpg)
Functions: Big O and Small o
Let g(·) be a real-valued function on R.
• g(x) = O(m(x))
∃ c ≥ 0 such that ∣∣∣∣ g(x)
m(x)
∣∣∣∣ ≤ c as x→ 0 (or +∞)
• g(x) = o(m(x)) ∣∣∣∣ g(x)
m(x)
∣∣∣∣ = 0 as x→ 0 (or +∞)
Conic Programming 11 / 39
![Page 37: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/37.jpg)
Functions
Taylor theorem in small o formulation:• p = 1
f(x+ h) = f(x) +∇f(x)h+ o(‖h‖)• p = 2
f(x+ h) = f(x) +∇f(x)h+1
2hTF (x)h+ o(‖h‖2)
Conic Programming 12 / 39
![Page 38: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/38.jpg)
Linear Systems
Given x1, · · · , xm ∈ Rn• Linear combination:
m∑i=1
λixi,
where λi ∈ R, i = 1, . . . ,m.• Linearly independent
m∑i=1
λixi = 0⇒ λ1 = · · · = λm = 0
• Affine combination: a linear combination withm∑i=1
λi = 1
• Affinely independent: if x2 − x1, · · · , xm − x1 are linearlyindependent.
Conic Programming 13 / 39
![Page 39: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/39.jpg)
Linear Systems
• Convex combination: a linear combination withm∑i=1
λi = 1 and λi ≥ 0, i = 1, . . . ,m
• Hyperplane:
X = {x ∈ Rn|aTx =
n∑i=1
aixi = b}
• Affine space: affine combination of any two points in the space is stillin the space. (An intersection of finitely many hyperplanes.)
• Linear subspace: an affine space containing the origin.
We can always transform an affine space Y ⊂ Rn into a linearsubspace X ⊂ Rn by choosing x0 ∈ Y such that
X = {x− x0|x ∈ Y}
Conic Programming 14 / 39
![Page 40: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/40.jpg)
Linear Systems
• Half space:
X = {x ∈ Rn|aTx =
n∑i=1
aixi ≤ b}
• Polyhedron: an intersection of finitely many half spaces.
• Polytope: a bounded polyhedron
• Dimension of a linear subspace: the maximum number of linearlyindependent vectors in the subspace.
• Dimension of an affine space: the dimension of the transformedlinear subspace.
• Dimension of a polyhedron: the dimension of the smallest affinespace containing it.
Conic Programming 15 / 39
![Page 41: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/41.jpg)
Linear Systems
• Linear equations
a1 • x = b1a2 • x = b2· · · · · · · · ·
am • x = bm
⇒ Ax = b,
where a1, · · · , am and x are all in Rn.
A1 •X = b1A2 •X = b2· · · · · · · · ·
Am •X = bm
⇒ AX = b,
where A1, · · · , Am and X are all in Sn.• For convenience, A∗y =
∑mi=1 yiAi.
Conic Programming 16 / 39
![Page 42: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/42.jpg)
Convex Sets and Properties
• A set X ⊂ Rn is convex if for any x1 ∈ X and x2 ∈ X , we haveλx1 + (1− λ)x2 ∈ X , for all 0 ≤ λ ≤ 1.
• Convex hull: the smallest convex set containing a given set
conv(X ) = {x ∈ Rn|x =∑mi=1 λiy
i for some m ∈ N+,λi ≥ 0,
∑mi=1 λi = 1, and yi ∈ X , i = 1, . . . ,m}
• Dimension of a convex set: the dimension of the smallest affinespace containing it.
• Relative interior of a convex set X ⊂ Rn: suppose H is the smallestaffine space containing X ,
ri(X ) = {x ∈ Rn|∃ open set Y ⊆ Rn such that x ∈ Y ∩H ⊂ X}
• Supporting hyperplane H = {x ∈ Rn|aTx = b} of a convex set X :
aT y ≥ b,∀ y ∈ X and X ∩H 6= ∅.
Conic Programming 17 / 39
![Page 43: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/43.jpg)
Convex Functions and Properties
• Epigraph of a function f : X ⊂ Rn → R
epif = {(x, λ) ∈ Rn+1|λ ≥ f(x), x ∈ X}
• Closed function: if epif is a closed set.
• Convex function: if epif is a convex set.
• Concave function: if −f is a convex function.
• Convex hull function conv(f) of a function f : X ⊂ Rn → R is afunction on X such that epi(conv(f)) = conv(epi(f)).
Lemmaf : X ⊂ Rn → R is a convex function if and only if for any x1, x2 ∈ X and0 ≤ λ ≤ 1, we have
f(λx1 + (1− λ)x2) ≤ λf(x1) + (1− λ)f(x2).
Conic Programming 18 / 39
![Page 44: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/44.jpg)
Convex Functions and Properties
• Subgradient d ∈ Rn of a convex function f : X ⊂ Rn at x ∈ X :
if for any y ∈ X ,f(y) ≥ f(x) + dT (y − x)
• The set {(y, λ) ∈ Rn+1|λ− dT y = f(x)− dTx} is a supportinghyperplane of epif at x.
• Subdifferential of a convex function f : X ⊂ Rn at x ∈ X :
∂f(x) = {d ∈ Rn|d is a subgradient of f at x}
Conic Programming 19 / 39
![Page 45: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/45.jpg)
Convex Functions and Properties
Figure: (x, f(x))↔ (m, b) or (y, h(y))
Conic Programming 20 / 39
![Page 46: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/46.jpg)
Convex Functions and Properties
• Conjugate (transform) of f : X ⊂ Rn → R:
h(y) = supx∈X{y • x− f(x)}
with h being defined on Y = {y ∈ Rn|h(y) < +∞}.
Lemmah : Y is a closed, convex function.
Lemma (Fenchel’s inequality)Given f : X and its conjugate h : Y, then
x • y ≤ f(x) + h(y), ∀ x ∈ X and y ∈ Y.
Moreover,x • y = f(x) + h(y) ⇐⇒ y ∈ ∂f(x)
Conic Programming 21 / 39
![Page 47: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/47.jpg)
Conjugate Functions and Properties
Let f : X ⊂ Rn → R be a function with its conjugate transform h : Y.• For α ∈ R, the conjugate of f + α is h− α.• For a ∈ Rn, the conjugate of f(x) = f(x) + x • a on X ish(y) = h(y − a), ∀ y ∈ Y.
• For a ∈ Rn, the conjugate of f(x) = f(x− a) on X ish(y) = h(y) + y • a, ∀ y ∈ Y.
• For λ > 0, the conjugate of f1(x) = λf(x) on X is h1(y) = λh( yλ ),∀ y ∈ λY.
• For λ > 0, the conjugate of f2(x) = f(xλ ) on λX is h2(y) = h(λy),∀ y ∈ Y/λ.
TheoremAssume that f1 : X and f2 : X have the same convex hull function. Thenthey have the same conjugate transform h : Y when it exists.
Conic Programming 22 / 39
![Page 48: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/48.jpg)
Conjugate Functions and Properties
We know the dual problem of LD is LP again. When will the conjugatetransform of h : Y become f : X?
Proper functionA convex function f is proper if its epigraph is non-empty and contains novertical lines, i.e. if f(x) < +∞ for at least one x and f(x) > −∞ forevery x.
TheoremLet f : X ⊂ Rn → R be a proper closed convex function with conjugatetransform h : Y. Then the conjugate transform of h : Y is f : X . Moreover,y ∈ ∂f(x) if and only if x ∈ ∂h(y). In this case,
x • y = f(x) + h(y) ⇐⇒ y ∈ ∂f(x) or x ∈ ∂h(y)
Conic Programming 23 / 39
![Page 49: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/49.jpg)
Convex Cone Structure
Content• Convex Cones and Properties• Partial Order and Ordered Vector Space• Some Examples
Conic Programming 24 / 39
![Page 50: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/50.jpg)
Convex Cones and Properties
• A set K ⊂ Rn is a cone if
∀x ∈ K and λ > 0⇒ λx ∈ K;
• A cone K ⊂ Rn is pointed if
K ∩ −K = {0};
• A cone K ⊂ Rn is solid ifintK 6= ∅;
• A cone K ⊂ Rn is proper if it is pointed, solid, closed and convex.
Conic Programming 25 / 39
![Page 51: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/51.jpg)
Convex Cones and Properties
• Conic combination: a linear combination∑mi=1 λix
i with λi ≥ 0,xi ∈ Rn for all i = 1, . . . ,m.
• The conic hull of a set X ⊂ Rn is
cone(X ) = {x ∈ Rn|x =∑mi=1 λix
i, for some m ∈ N+
and xi ∈ X , λi ≥ 0, i = 1, . . . ,m.}
• The dual cone K∗ ⊂ Rn of a cone K ⊂ Rn is
K∗ = {y ∈ Rn|y • x ≥ 0,∀ x ∈ K}
K∗ is a closed, convex cone.• If K∗ = K, then K is a self-dual cone.
Conic Programming 26 / 39
![Page 52: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/52.jpg)
Convex Cones and Properties
K, K1, K2 are convex cones in Rn.• (K∗)∗ = cl(K).• K1 ⊆ K2 ⇒ K∗2 ⊆ K∗1 .• K1 ∩K2, K1 ∪K2, K1 +K2, K1 ×K2 are all cones.• (K1 +K2)∗ = K∗1 ∩K∗2 .• K1, K2 closed⇒ K1 +K2 and K1 ×K2 are closed.• ri(K1 +K2) = ri(K1) + ri(K2).• ri(K1 ×K2) = ri(K1)× ri(K2).• The supporting hyperplane of K always contains the origin• If K is solid, then K∗ is pointed.• If K is pointed, then K∗ is solid.• If K is proper, then K∗ is proper.
Conic Programming 27 / 39
![Page 53: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/53.jpg)
Convex Cones and Properties
• Rn+ ⊆ Rn, Ln ⊆ Rn, Sn+ ⊆ Rn×n are pointed, closed, convex andsolid cones, i.e., proper cones.
• (Rn+)∗ = Rn+, (Ln)∗ = Ln, (Sn+)∗ = Sn+.
• Rn1+ ×R
n2+ × · · · ×R
nk+ is a proper cone in R
k∑i=1
ni
.
• Ln1 × Ln2 × · · · × Lnk is a proper cone in Rk∑
i=1ni
.
• Sn1+ × S
n2+ × · · · × S
nk+ is a proper cone in R
k∑i=1
ni×ni
.• Rn1
+ × Ln2 × Sn3+ is a proper cone in Rn1+n2+n3×n3 .
Conic Programming 28 / 39
![Page 54: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/54.jpg)
Partial Order and Ordered Vector Space
• A relation “≥” is a partial order on a set X if it has:
1. reflexivity: a ≥ a for all a ∈ X ;
2. antisymmetry: a ≥ b and b ≥ a imply a = b;
3. transitivity: a ≥ b and b ≥ c imply a ≥ c.
• An ordered vector space X is equipped with a partial order “≥” whichalso satisfies:
• homogeneity: a ≥ b and λ ∈ R+ imply λa ≥ λb;
• additivity: a ≥ b and c ≥ d imply a+ c ≥ b+ d.
Conic Programming 29 / 39
![Page 55: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/55.jpg)
Partial Order and Ordered Vector Space
• A proper cone K in a vector space can induce a partial order “≥K”
a ≥K b⇔ a− b ∈ K
which leads to an ordered vector space.• Similarly, we can define “≤K”
a ≤K b⇔ b ≥K a,
• Closeness of K allows passing limits in ≥K :
ai ≥K bi, ai → a, bi → b as i→∞ ⇒ a ≥K b.
• Solidness of K allows us to define a strict inequality:
a >K b⇔ a− b ∈ intK,
anda <K b⇔ b >K a.
Conic Programming 30 / 39
![Page 56: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/56.jpg)
Examples: Rn+
• Rn+ is a proper cone.
• Inner product: x • y = xT y
• (Rn+)∗ = Rn+ (self-dual)
• Partial order: “≥Rn+
”
Conic Programming 31 / 39
![Page 57: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/57.jpg)
Examples: Ln
• Ln / SOC(n− 1)Lorentz cone (secondorder cone)
Ln = {x ∈ Rn|xn ≥√x21 + · · ·+ x2n−1}
• Ln is a proper cone.
• Inner product: x • y = xT y
• (Ln)∗ = Ln (self-dual)
• Partial order: “≥Ln ”
Conic Programming 32 / 39
![Page 58: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/58.jpg)
Examples: Sn+
• Sn+ ⊂ Sn: the set of symmetric positive semidefinite matrices• Sn+ is a proper cone.• Inner product:
X • Y = tr(XTY )
• Another view:
vec(X) = [X11,√
2X12, X22,√
2X13,√
2X23, X33, · · · , Xnn]T ∈ Rn(n+1)
2
ThenX • Y = vec(X) • vec(Y ) =
∑i,j
XijYij
• Partial order: “≥Sn+
” or “�”
Conic Programming 33 / 39
![Page 59: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/59.jpg)
Examples: Sn+
Lemma(Sn+)∗ = Sn+ (self-dual)
Proof.“⊆”: If X ∈ (Sn+)∗, then zTXz = X • zzT ≥ 0, for all z ∈ Rn.Therefore, X ∈ Sn+.
“⊇”: For any Y ∈ Sn+,
Y =∑ni=1 λiz
i(zi)T ,
with λi ≥ 0.If X ∈ Sn+, then
X • Y =
n∑i=1
λiX • zi(zi)T =
n∑i=1
λi(zi)TXzi ≥ 0.
Therefore, X ∈ (Sn+)∗.Conic Programming 34 / 39
![Page 60: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/60.jpg)
Examples: Cn and C∗n
• Copositive cone:
Cn = {X ∈ Sn|zTXz ≥ 0,∀ z ≥Rn+
0}
• Completely positive(nonnegative) cone:
C∗n =
{X ∈ Sn
∣∣∣∣∣ X =∑mi=1 z
i(zi)T , for some m ∈ N+
and zi ≥Rn+
0, i = 1, . . . ,m
}
• (Cn)∗ = C∗n and Cn = (C∗n)∗
• C∗n ⊂ Sn+ ⊂ Cn
Conic Programming 35 / 39
![Page 61: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/61.jpg)
Examples: Cones of NonnegativeQuadratic Functions — Homogeneous
• F ⊂ Rn• Nonnegative homogeneous quadratic functions over F
f(x) = xTAx ≥ 0,∀x ∈ F
f ⇔ A
• HDF = {A ∈ Sn|xTAx ≥ 0,∀x ∈ F} is a closed, convex cone.(i) Closeness:
xTAix ≥ 0 and Ai → A⇒ xTAx ≥ 0
(ii) Convexity:
xTAiX ≥ 0, i = 1, 2⇒ xT (λA1 + (1− λ)A2)x ≥ 0,∀ 0 ≤ λ ≤ 1
Conic Programming 36 / 39
![Page 62: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/62.jpg)
Examples: Cones of NonnegativeQuadratic Functions — Homogeneous
• HD∗F = cl(cone{xxT |x ∈ F})• (HDF )∗ = HD∗F and (HD∗F )∗ = HDF• Examples:
• F = Rn
HDF = HD∗F = Sn+
• F = Rn+
HDF = Cn and HD∗F = C∗n• F = {x ∈ Rn
+|eTx = 1}HDF = Cn and HD∗F = C∗n
Conic Programming 37 / 39
![Page 63: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/63.jpg)
Examples: Cones of NonnegativeQuadratic Functions — Nonhomogeneous
• Nonnegative quadratic functions over F ⊂ Rn
f(x) = xTAx+ 2bTx+ c ≥ 0,∀x ∈ F
f ⇔[c bT
b A
]• DF = {
[c bT
b A
]∈ Sn+1|
[1x
]T [c bT
b A
] [1x
]≥ 0,∀x ∈ F} is a closed,
convex cone.
• D∗F = cl(cone{[
1 xT
x xxT
]|x ∈ F})
• (D∗F )∗ = DF and (DF )∗ = D∗F
Conic Programming 38 / 39
![Page 64: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/64.jpg)
Examples: Cones of NonnegativeQuadratic Functions — Nonhomogeneous
• Examples:• F = Rn
DF = D∗F = Sn+1+
• F = Rn+
DF = Cn+1 and D∗F = C∗n+1
Conic Programming 39 / 39
![Page 65: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/65.jpg)
LCoP Part III – Duality Theory of Linear ConicProgramming
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
Conic Programming 1 / 28
![Page 66: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/66.jpg)
Duality Theory of Linear ConicProgramming
Content• Definition of LCoP and LCoD• Conjugate Duality Theory• Deriving LCoD from LCoP• Conic Duality Theorems for LCoP• Duality Theorems of LP, SOCP and SDP
Conic Programming 2 / 28
![Page 67: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/67.jpg)
Linear Conic Programs
Recall thatMin C •Xs.t. A •X = B
X ∈ K(LCoP)
where K is a closed, convex cone, A, C and B are in the space ofinterests with • being an appropriate linear operator.
Note that when K = Rn+ or Ln, X is a vector; when K = Sn+, X is an
n× n matrix.
Conic Programming 3 / 28
![Page 68: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/68.jpg)
Linear Conic Programs
Min c • xs.t. ai • x = bi, i = 1, . . . ,m
x ∈ K(LCoP)
where K is a closed, convex cone, such as Rn+, Ln, Sn+.
Max bT ys.t.
∑mi=1 yia
i + s = cs ∈ K∗, y ∈ Rm
(LCoD)
where K∗ is the dual cone of K.
Conic Programming 4 / 28
![Page 69: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/69.jpg)
Conjugate Duality Theory
Conjugate Program
inf f(x)s.t. x ∈ X ∩K
(CP)
where f : X ⊂ Rn → R and K is a cone in Rn.
Conjugate Dual
inf h(y)s.t. y ∈ Y ∩K∗
(CD)
where h : Y is the conjugate transform of f : X and K∗ is the dual coneof K.• feas(*) denotes the feasible domain of problem (*)• opt(*) denotes the optimal solution set of problem (*)• v(*) denotes the optimal value of problem (*)
Conic Programming 5 / 28
![Page 70: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/70.jpg)
Conjugate Duality Theory
Theorem (Conjugate duality theorem/KKT duality theorem)If x ∈ feas(CP) and y ∈ feas(CD), then
0 ≤ x • y ≤ f(x) + h(y)
with the equality holding if and only if
x • y = 0 and y ∈ ∂f(x),
in which casex ∈ opt(CP) and y ∈ opt(CD).
ProofThe inequality follows from Fenchel’s inequality and the definition of dualcone. The rest follows easily.
Conic Programming 6 / 28
![Page 71: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/71.jpg)
Conjugate Duality Theory
Theorem (Weak duality theorem)If both CP and CD are feasible, then
(i) v(CP) is finite and
v(CP) + h(y) ≥ 0,∀ y ∈ feas(CD);
(ii) v(CD) is finite and
v(CP) + v(CD) ≥ 0.
ProofThis theorem follows from the previous KKT duality theorem.
Conic Programming 7 / 28
![Page 72: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/72.jpg)
Conjugate Duality Theory
Theorem (Fenchel’s theorem/Strong duality theorem)Suppose that f : X and K are closed and convex. If v(CD) is finite andone of the following conditions holds:
(i) ri(K∗) ∩ ri(Y) 6= ∅,(ii) both K∗ and Y are polyhedrons,
thenv(CP) + v(CD) = 0 and opt(CP) 6= ∅.
Similarly, if v(CP) is finite and one of the following conditions holds:(i) ri(K) ∩ ri(X ) 6= ∅,(ii) both K and X are polyhedrons,
thenv(CP) + v(CD) = 0 and opt(CD) 6= ∅.
Proof: See Rockafellar’s book “Convex Analysis” Section 31.
Conic Programming 8 / 28
![Page 73: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/73.jpg)
Deriving LCoD from LCoP
LCoP
Min c • xs.t. ai • x = bi, i = 1, . . . ,m
x ∈ K(LCoP)
Deriving LCoD in the framework of conjugate program.
Conic Programming 9 / 28
![Page 74: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/74.jpg)
Deriving LCoD from LCoP
LCoP as CP
Variables: uT = (u0, u1, . . . , um) ∈ Rm+1;
f(u) = u0;
X = {u ∈ Rm+1|ui = bi, i = 1, . . . ,m};
K0 = {u ∈ Rm+1|u0 = c • x, ui = ai • x, x ∈ K, i = 1, . . . ,m}.
inf f(u)s.t. u ∈ X ∩K0
Conic Programming 10 / 28
![Page 75: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/75.jpg)
Deriving LCoD from LCoP
Corresponding CD
Variables: vT = (v0, v1, . . . , vm) ∈ Rm+1;
h(v) = supu∈X{u • v − f(u)} < +∞
= supu0∈R{(v0 − 1)u0 +
m∑i=1
bivi}
Hence
h(v) =∑m
i=1 bivi;
Y = {v ∈ Rm+1|v0 = 1};
Conic Programming 11 / 28
![Page 76: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/76.jpg)
Deriving LCoD from LCoP
Corresponding CD
Moreover,
K∗0 = {v ∈ Rm+1|v • u ≥ 0,∀u ∈ K0}
= {v ∈ Rm+1|(v0c +∑m
i=1 viai) • x ≥ 0,∀x ∈ K}
= {v ∈ Rm+1|v0c +∑m
i=1 viai ∈ K∗}.
Hence
Y ∩K∗0 = {v ∈ Rm+1|c +
m∑i=1
viai = s, s ∈ K∗}.
inf∑m
i=1 bivis.t. c +
∑mi=1 via
i = ss ∈ K∗
Conic Programming 12 / 28
![Page 77: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/77.jpg)
Deriving LCoD from LCoP
CD to LCoDDefine variables: y = −(v1, . . . , vm)T , we have
Max bT ys.t.
∑mi=1 yia
i + s = cs ∈ K∗, y ∈ Rm
(LCoD)
Therefore, the duality theorems of conjugate programs may apply toLCoP.
Conic Programming 13 / 28
![Page 78: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/78.jpg)
Conic Duality Theorems for LCoP
Theorem (Weak duality theorem)If both LCoP and LCoD are feasible, then
c • x ≥ bT y,∀x ∈ feas(LCoP) and (y, s) ∈ feas(LCoD).
Theorem (Strong duality theorem)
(i) If feas(LCoP) ∩ int(K) 6= ∅ and v(LCoP) is finite, then there exists(y∗, s∗) ∈ feas(LCoD) such that bT y∗ = v(LCoP).
(ii) If feas(LCoD) ∩ int(K∗) 6= ∅ and v(LCoD) is finite, then there existsx∗ ∈ feas(LCoP) such that c • x = v(LCoD).
Proof: See Aharon Ben-Tal and Arkadi Nemirovski’s book “Lectures onmodern convex optimization” Chapter 2.
Conic Programming 14 / 28
![Page 79: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/79.jpg)
Conic Duality Theorems for LCoP
Theorem (KKT duality theorem)If feas(LCoP) and feas(LCoD) are both nonempty andfeas(LCoP) ∩ int(K) 6= ∅, then x∗ is optimal for LCoP if and only if thefollowing conditions hold:
(i) x∗ ∈ feas(LCoP);
(ii) There exists (y∗, s∗) ∈ feas(LCoD);
(iii) c • x∗ = bT y∗ (or equivalently x∗ • s∗ = c • x∗ − bT y∗ = 0).Proof: =⇒ follows from strong duality theorem.
⇐= is obvious.
Conic Programming 15 / 28
![Page 80: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/80.jpg)
Linear Program (LP)
Min cTxs.t. Ax = b
x ≥Rn+
0(LP)
Max bT ys.t. AT y + s = c
s ≥Rn+
0(LD)
Conic Programming 16 / 28
![Page 81: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/81.jpg)
Linear Program (LP)
Theorem (LP duality theorem)
(i) If either LP or LD is unbounded, then the other one is infeasible.
(ii) If either v(LP) or v(LD) is finite, then there exist x∗ ∈ feas(LP) and(y∗, s∗) ∈ feas(LD) such that v(LP) = cTx∗ = bT y∗ = v(LD).
(iii) If LP is feasible and v(LP) is finite, then x∗ is optimal for LP if andonly if the following conditions hold:
(a) Ax∗ = b, x∗ ≥Rn+
0;
(b) there exists (y∗, s∗) satisfying AT y∗ + s∗ = c, s ≥Rn+
0;
(c) (x∗)T s∗ = cTx∗ − bT y∗ = 0.
Conic Programming 17 / 28
![Page 82: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/82.jpg)
Second Order Cone Program (SOCP)
Min cTxs.t. Ax = b
x ≥K 0(SOCP)
where K = Ln1 × · · · × Lnr = {x ∈ Rn|n1 + · · ·+ nr = n, (x1, ..., xn1)T ∈
Ln1 , ..., (xn−nr+1, ..., xn)T ∈ Lnr}.
Max bT ys.t. AT y + s = c
s ≥K 0(SOCD)
Conic Programming 18 / 28
![Page 83: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/83.jpg)
Second Order Cone Program (SOCP)
Theorem (SOCP duality theorem)
(i) If either SOCP or SOCD is unbounded, then the other one isinfeasible.
(ii) If there exists a feasible solution x such that x ∈ int(K), andv(SOCP) is finite, then there exist (y∗, s∗) ∈ feas(SOCD) such thatv(SOCP) = bT y∗ = v(SOCD).
(iii) If there exists a feasible solution (y, s) such that s ∈ int(K), andv(SOCD) is finite, then there exist x∗ ∈ feas(SOCP) such thatv(SOCP) = cTx∗ = v(SOCD).
Conic Programming 19 / 28
![Page 84: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/84.jpg)
Second Order Cone Program (SOCP)
Theorem (SOCP duality theorem)
(iv) If both SOCP and SOCD are feasible, and there exists a feasiblesolution x such that x ∈ int(K), then x∗ is optimal for SOCP if andonly if the following conditions hold:
(a) Ax∗ = b, x∗ ≥K 0;
(b) there exists (y∗, s∗) satisfying AT y∗ + s∗ = c, s∗ ≥K 0;
(c) (x∗)T s∗ = cTx∗ − bT y∗ = 0.
Conic Programming 20 / 28
![Page 85: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/85.jpg)
Difference between LP and SOCP (interior feasible solution):
Min −x2
s.t. x1 − x3 = 0x ∈ L3
Max 0 · y
s.t.
0−10
− y
10−1
=
−y−1y
∈ L3
v(SOCP ) = 0 but SOCD is infeasible.
Figure: Feasible domain is a ray x1 = x3 in hyperplane x2 = 0. No feasibleinterior point.
Conic Programming 21 / 28
![Page 86: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/86.jpg)
Second Order Cone Program (SOCP)
Finite nonzero duality gap:
Min −x2
s.t. x1 + x3 − x4 + x5 = 0x2 + x4 = 1x ∈ L3 × L2
Max y2
s.t.
y1 +s1 = 0y2 +s2 = −1
y1 +s3 = 0−y1 +y2 +s4 = 0y1 +s5 = 0
s ∈ L3 × L2
x∗ =
−101
× [11
]y∗ =
[−1−1
]s∗ =
101
× [01
]
v(SOCP) = 0 6= −1 = v(SOCD)
Conic Programming 22 / 28
![Page 87: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/87.jpg)
Second Order Cone Program (SOCP)
Zero duality gap with non-attainable value:
Min x1
s.t. −x2 −x3 = 0x2 = −1
x ∈ L3
Max −y2s.t. s1 = 1
−y1 +y2 +s2 = 0−y1 +s3 = 0
s ∈ L3
x∗ =
0−11
v(SOCD) = 0 but not attainable.
Conic Programming 23 / 28
![Page 88: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/88.jpg)
Semidefinite Program (SDP)
Min C •Xs.t. AX = b
X � 0(SDP)
Max bT ys.t. A∗y + S = C
S � 0(SDD)
Note:
A∗y =
m∑i=1
yiAi
Conic Programming 24 / 28
![Page 89: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/89.jpg)
Semidefinite Program (SDP)
Theorem (SDP duality theorem)
(i) If either SDP or SDD is unbounded, then the other one is infeasible.
(ii) If there exists a feasible solution X such that X � 0, and v(SDP) isfinite, then there exist (y∗, S∗) ∈ feas(SDD) such thatv(SDP) = bT y∗ = v(SDD).
(iii) If there exists a feasible solution (y, S) such that S � 0, and v(SDD)is finite, then there exist X∗ ∈ feas(SDP) such thatv(SDP) = C •X∗ = v(SDD).
Conic Programming 25 / 28
![Page 90: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/90.jpg)
Semidefinite Program (SDP)
Theorem (SDP duality theorem)
(iv) If both SDP and SDD are feasible, and there exists a feasiblesolution X such that X � 0, then X∗ is optimal for SDP if and only ifthe following conditions hold:
(a) AX∗ = b, X∗ � 0;
(b) there exists (y∗, S∗) satisfying A∗y∗ + S∗ = C, S∗ � 0;
(c) X∗ • S∗ = C •X∗ − bT y∗ = 0.
Conic Programming 26 / 28
![Page 91: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/91.jpg)
Semidefinite Program (SDP)
Interior feasible solutionInfinite duality gap:
C =
[0 11 0
], A =
[0 00 1
], b = 0
X∗ =
[0 00 0
]and SDD is infeasible.
Zero duality gap with non-attainable value:
C =
[1 00 0
], A =
[0 11 0
], b = 1
v(SDP ) = 0 but is not attainable. y∗ = 0 and S∗ =
[1 00 0
].
Conic Programming 27 / 28
![Page 92: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/92.jpg)
Semidefinite Program (SDP)
Finite nonzero duality gap:
C =
0 0 00 0 10 1 0
, A1 =
0 0 00 1 00 0 0
, A2 =
1 0 00 0 −10 −1 0
, b =
[01
]
X∗ =
1 0 00 0 00 0 0
, y∗ =
[0−1
], S∗ =
1 0 00 0 00 0 0
v(SDP ) = 0 6= −1 = v(SDD)
Conic Programming 28 / 28
![Page 93: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/93.jpg)
LCoP Part IV – Interior Point Methods andSolution Software
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
Conic Programming 1 / 39
![Page 94: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/94.jpg)
Interior Point Methods and SolutionSoftware
Content• Interior Points and Primal-Dual Model• Barrier Functions and Optimal Systems• Central Path and Newton Methods• Path Following Method• CVX Solution Software
Conic Programming 2 / 39
![Page 95: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/95.jpg)
Interior Point Methods
• Interior point approach• Start from an interior point solution.• If the current solution is not good enough, then move to another interior
point solution.• Stop at an interior point solution whose objective value is close to the
optimum (within an ε gap).• Advantages:
• Polynomial time complexity (comparing to the simplex method for LP)• Excellent computational performance in practice (comparing to the
ellipsoid method)• Three types: primal; dual; primal-dual
Conic Programming 3 / 39
![Page 96: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/96.jpg)
Primal-dual Model
• Primal-dual type of LP
Min sTxs.t. Ax = b
AT y + s = cx ≥Rn
+0, s ≥Rn
+0
(LPD)
• Primal-dual type of SDP
Min S •Xs.t. AX = b
A∗y + S = CX � 0, S � 0
(SDPD)
• Note:AX = [A1 •X, · · · , Am •X]T
and A∗y =∑mi=1 yiAi
Conic Programming 4 / 39
![Page 97: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/97.jpg)
Interior Points
feas+(LP) = {x ∈ Rn|Ax = b, x >Rn+
0}feas+(LD) = {(y, s) ∈ Rm ×Rn|AT y + s = c, s >Rn
+0}
feas+(LPD) = feas+(LP)× feas+(LD)
feas+(SDP) = {X ∈ Sn|AX = b,X � 0}feas+(SDD) = {(y, S) ∈ Rm × Sn|A∗y + S = C, S � 0}feas+(SDPD) = feas+(SDP)× feas+(SDD)
• Assumptions:• feas+(LP) and feas+(LD) are not empty and the rows of A are linearly
independent.
• feas+(SDP) and feas+(SDD) are not empty and the vectors formed byAi in A are linearly independent.
Conic Programming 5 / 39
![Page 98: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/98.jpg)
Barrier function
• Properties required:• Strictly convex (concave).• Goes to +∞ (−∞) when the point is close to the boundary.• Sufficient continuous differentiability.
• Barrier functions:
LP : −∑ni=1 log xi SDP : − log det(X)
LD :∑ni=1 log si SDD : log det(S)
LPD : −∑ni=1 log(xisi) SDPD : − log det(XS)
Conic Programming 6 / 39
![Page 99: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/99.jpg)
LP with Barrier
Min cTx− µ∑ni=1 log xi
s.t. Ax = bx >Rn
+0
(LPB)
Max bT y + µ∑ni=1 log si
s.t. AT y + s = cs >Rn
+0
(LDB)
Min sTx− µ∑ni=1 log(xisi)
s.t. Ax = bAT y + s = cx >Rn
+0, s >Rn
+0
(LPDB)
Conic Programming 7 / 39
![Page 100: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/100.jpg)
Common Optimal System for LP withBarrier
Ax = bAT y + s = cΛxs = µex >Rn
+0, s >Rn
+0,
where e = (1, . . . , 1)T and Λx is a diagonal matrix with (Λx)ii = xi,i = 1, . . . , n.
Notice that
µ =xT s
n=cTx− bT y
n
When µ→ 0, sTx→ 0. Optimal!
Conic Programming 8 / 39
![Page 101: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/101.jpg)
SDP with Barrier
Min C •X − µ log det(X)s.t. AX = b
X � 0(SDPB)
Min bT y + µ log det(S)s.t. A∗y + S = C
S � 0(SDDB)
Min S •X − µ log det(XS)s.t. AX = b
A∗y + S = CX � 0, S � 0
(SDPDB)
Conic Programming 9 / 39
![Page 102: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/102.jpg)
Common Optimal System for SDP withBarrier
AX = bA∗y + S = CXS = µIX � 0, S � 0
Notice that
µ =S •Xn
=C •X − bT y
n
When µ→ 0, S •X → 0. Optimal!
Conic Programming 10 / 39
![Page 103: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/103.jpg)
Central Path for LP and SDP
CLP = {(x, y, s) ∈ feas+(LPD)|Λxs = µe, 0 < µ < +∞}
CSDP = {(X, y, S) ∈ feas+(SDPD)|XS = µI, 0 < µ < +∞}
Under proper assumptions:• For any 0 < µ < +∞, there exists a unique point on central path.
LP: (x(µ), y(µ), s(µ))
SDP: (X(µ), y(µ), S(µ))
• Given µ > 0, the set {(x, y, s) ∈ feas+(LPD)|Λxs = µe, 0 < µ < µ} isbounded.Given µ > 0, the set {(X, y, S) ∈ feas+(SDPD)|XS = µI, 0 < µ < µ}is bounded.
Conic Programming 11 / 39
![Page 104: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/104.jpg)
Example: Central Path (Primal)
Min x1 + x2
s.t. x1 + x2 ≤ 3x1 − x2 ≤ 1x2 ≤ 2x1 ≥ 0, x2 ≥ 0
Figure: Projection of central path on (x1, x2)
Conic Programming 12 / 39
![Page 105: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/105.jpg)
Example: Central Path (Dual)
Max 3y1 + y2 + 2y3
s.t. y1 + y2 ≤ 1y1 − y2 + y3 ≤ 1y1 ≤ 0y2 ≤ 0y3 ≤ 0
Figure: Projection of central path on (y1, y2)
Conic Programming 13 / 39
![Page 106: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/106.jpg)
Newton Method for LP
Given (x0, y0, s0) ∈ feas+(LPD) with µ0 = (s0)T x0
n and 0 ≤ γ ≤ 1, find(dx, dy, ds) satisfying
A(x0 + dx) = bAT (y0 + dy) + (s0 + ds) = cΛx0+dx(s0 + ds) = γµ0ex0 + dx >Rn
+0, s0 + ds >Rn
+0,
After linearization A 0 00 AT I
Λs0 0 Λx0
dxdyds
=
00
γµ0e− Λx0Λs0e
x0 + dx >Rn
+0, s0 + ds >Rn
+0,
Directly solve the equation is not easy.
Conic Programming 14 / 39
![Page 107: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/107.jpg)
Newton Method for LP
Linear scaling: Given a positive diagonal matrix D ∈ Rn×n,
A = AD, x0 = D−1x0, s0 = Ds0, c = Dc
A 0 00 AT I
Λs0 0 Λx0
dxdyds
=
00
γµ0e− Λx0Λs0e
x0 + dx >Rn
+0, s0 + ds >Rn
+0,
• D = Λx0 : x0 = e ⇒ x0 + dx >Rn+
0, ∀‖dx‖2 < 1 (Primal)
• D = Λ−1s0 : s0 = e ⇒ s0 + ds >Rn
+0, ∀‖ds‖2 < 1 (Dual)
• D = Λ1/2x0 Λ
−1/2s0 : v0 = x0 = s0 = Λ
1/2x0 Λ
1/2s0 e (Primal-dual)
Conic Programming 15 / 39
![Page 108: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/108.jpg)
Primal-Dual Interior-Point Method for LP
D = Λ1/2x0 Λ
−1/2s0 : A 0 0
0 AT II 0 I
dxdyds
=
00
γµ0Λ−1v0 e− v
0
x0 + dx >Rn
+0, s0 + ds >Rn
+0,
One can solveAAT dy = −A(γµ0Λ−1
v0 e− v0)
And then solve ds and dx:
ds = −AT dydx = −ds + γµ0Λ−1
v0 e− v0
Conic Programming 16 / 39
![Page 109: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/109.jpg)
Newton Method for SDP
Given (X0, y0, S0) ∈ feas+(SDPD) with µ0 = S0•X0
n and 0 ≤ γ ≤ 1, find(4X, dy,4S) satisfying
A(X0 +4X) = bA∗(y0 + dy) + (S0 +4S) = C(X0 +4X)(S0 +4S) = γµ0IX0 +4X � 0, S0 +4S � 0
After linearization
A4X = 0A∗dy + 4S = 0
4XS0 + X04S = γµ0I −X0S0
X0 +4X � 0, S0 +4S � 0.
Directly solve the equation is not easy.
Conic Programming 17 / 39
![Page 110: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/110.jpg)
Newton Method for SDP
Linear transformation: Given an invertible matrix L ∈ Rn×n, letA = (A1, . . . , Am), Ai = LTAiL for i = 1, . . . ,m.X0 = L−1X0L−T , S0 = LTS0L, C = LTCL.
A4X = 0
A∗dy + 4S = 0
4XS0 + X04S = γµ0I − X0S0
X0 +4X � 0, S0 +4S � 0
• L = (X0)1/2: X0 = I ⇒ X0 +4X � 0, ∀‖4X‖F < 1 (Primal)• L = (S0)−1/2: S0 = I ⇒ S0 +4S � 0, ∀‖4S‖F < 1 (Dual)• LLT = (S0)−1/2[(S0)1/2X0(S0)1/2]1/2(S0)−1/2:V 0 = X0 = S0 (Primal-dual)
Conic Programming 18 / 39
![Page 111: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/111.jpg)
Primal-Dual Interior-Point Method for SDP
LLT = (S0)−12 [(S0)
12X0(S0)
12 ]
12 (S0)−
12 :A 0 0
0 A∗ II 0 I
4Xdy4S
=
00
γµ0(V 0)−1 − V 0
X0 +4X � 0, S0 +4S � 0
One can solveAA∗dy = −A(γµ0(V 0)−1 − V 0)
And then solve 4S and 4X:
4S = −A∗dy4X = −4S + γµ0(V 0)−1 − V 0
Conic Programming 19 / 39
![Page 112: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/112.jpg)
Neighborhood of Central Path for LP
Notice that x0 = s0 = v0
• Distance to central path: u >Rn+0
δ(u) = ‖e− n
uTuΛuu‖2
• Neighborhood of the central path
N2(β) = {u|u >Rn+
0, δ(u) ≤ β}
N−∞(β) = {u|u >Rn+
0,Λuu ≥Rn+
(1− β)uTu
nI}
Conic Programming 20 / 39
![Page 113: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/113.jpg)
Examples: N2(12) and N−∞(12)
Figure: Neighborhood N2( 12) Figure: Neighborhood N−∞( 1
2)
Conic Programming 21 / 39
![Page 114: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/114.jpg)
Finding Step Length for LP
x0 + αdxs0 + αds
scaling back−−−−−−−−−−−→
[x1
s1
] new scaling−−−−−−−−−−→ v1 = x1 = s1
LemmaFor any 0 ≤ α ≤ 1,
µ1 =‖v1‖22n
=(x0 + αdx)T (s0 + αds)
n= (1− α+ γα)µ0
Conic Programming 22 / 39
![Page 115: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/115.jpg)
Finding Step Length for LP
LemmaIf δ(v0) < 1 and α satisfies x0 + αdx >Rn
+0 and s0 + αds >Rn
+0, then
(1− α+ γα)δ(v1) ≤ (1− α)δ(v0) +α2
2
(γ2δ(v0)2
1− δ(v0)+ n(1− γ)2
)Proof:
µ1δ(v1) = µ1‖e− 1
µ1Λv1v
1‖2
= ‖(1− α+ γα)µ0e− Λ(v0+αdx)(v0 + αds)‖2
≤ ‖(1− α)µ0(e− 1
µ0Λv0v
0)‖2 + ‖α2Λdx ds‖2
≤ (1− α)µ0δ(v0) +α2
2‖dx + ds‖22
= (1− α)µ0δ(v0) +α2
2(γ2‖µ0Λ−1
v0 e− v0‖22 + (1− γ)2nµ0)
≤ (1− α)µ0δ(v0) +α2
2(µ0γ2δ(v0)2
1− δ(v0)+ (1− γ)2nµ0)
Conic Programming 23 / 39
![Page 116: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/116.jpg)
Finding Step Length for LP
LemmaIf v0 ∈ N2(β) with β = 1
2 , γ = 11+1/
√2n
and α = 1, then
(i) v1 ∈ N2(β)
(ii) x1 • s1 = x1 • s1 = ‖v1‖22 = γµ0
Conic Programming 24 / 39
![Page 117: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/117.jpg)
Path Following Algorithm for LP
Step 1: (Initialization)ε > 0, (x0, y0, s0) with v0 ∈ N (β), where β = 1
2 .Set k = 0, γ = 1
1+1/√
2n, and α = 1.
Step 2: Solve the Newton system introduced above and get (dx, dy, ds).Set xk+1 = xk + αdx
yk+1 = yk + αdysk+1 = sk + αds
with vk+1 = Λ1/2
xk+1Λ1/2
sk+1e.Set k = k + 1.
Step 3: If xk • sk < ε, stop. Otherwise, go to Step 2.
Conic Programming 25 / 39
![Page 118: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/118.jpg)
Complexity for LP
TheoremGiven the above settings, we have
(i) vk ∈ N2(β), k = 0, 1, 2, . . ..(ii) The algorithms stops in
O(√n log
x0 • s0
ε)
steps and output a primal-dual solution satisfying
xk • sk < ε
Conic Programming 26 / 39
![Page 119: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/119.jpg)
Neighborhood of Central Path for SDP
Notice that X0 = S0 = V 0
• Distance to central path: U ∈ Sn+ and U � 0
δ(U) = ‖I − n
I • U2U2‖F , with U2 = UU
• Neighborhood of the central path
N2(β) = {U |U � 0, δ(U) ≤ β}
N−∞(β) = {U |U � 0, U2 � (1− β)I • U2
nI}
Conic Programming 27 / 39
![Page 120: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/120.jpg)
Finding Step Length for SDP
X0 + α4XS0 + α4S
scaling back−−−−−−−−−−−→
[X1
S1
] new scaling−−−−−−−−−−→ V 1 = X1 = S1
LemmaFor any 0 ≤ α ≤ 1,
µ1 =‖V 1‖2Fn
=tr[(X0 + α4X)(S0 + α4S)]
n= (1− α+ γα)µ0.
Conic Programming 28 / 39
![Page 121: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/121.jpg)
Finding Step Length for SDP
LemmaFor any square matrix U , we have
tr(U2) = ‖U + UT
2‖2F − ‖
U − UT
2‖2F ≤ ‖
U + UT
2‖2F
LemmaSuppose δ(V 0) < 1 and α ≥ 0 satisfies X0 + α4X � 0 andS0 + α4S � 0. Let
W =(X0 + α4X)(S0 + α4S) + ((X0 + α4X)(S0 + α4S))T
2
thenW = (1− α)(V 0)2 + αγµ0I + α24X4S +4S4X
2
andδ(V 1)2 ≤ ‖I − 1
µ1W‖2F
Conic Programming 29 / 39
![Page 122: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/122.jpg)
Finding Step Length for SDP
LemmaSuppose δ(V 0) < 1 and α ≥ 0 satisfies X0 + α4X � 0 andS0 + α4S � 0. Then
(1− α+ γα)δ(V 1) ≤ (1− α)δ(V 0) +α2
2
(γ2δ(V 0)2
1− δ(V 0)+ n(1− γ)2
)Proof
µ1δ(V 1) ≤ (1− α)µ0δ(V 0) + α2‖4X4S+4S4X2 ‖F
≤ (1− α)µ0δ(V 0) + α2
2 ‖4X +4S‖2F= (1− α)µ0δ(V 0) + α2
2 (γ2‖µ0(V 0)−1 − V 0‖2F + (1− γ)2nµ0)
≤ (1− α)µ0δ(V 0) + α2µ0
2
(γ2δ(V 0)2
1−δ(V 0) + n(1− γ)2)
Conic Programming 30 / 39
![Page 123: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/123.jpg)
Finding Step Length for SDP
LemmaIf V 0 ∈ N2(β) with β = 1
2 , γ = 11+1/
√2n
and α = 1, then
(i) V 1 ∈ N2(β)
(ii) X1 • S1 = X1 • S1 = ‖V 1‖2F = γµ0
Conic Programming 31 / 39
![Page 124: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/124.jpg)
Path Following Algorithm for SDP
Step 1: (Initialization)ε > 0, (X0, y0, S0) with V 0 ∈ N (β), where β = 1
2 .Set k = 0, γ = 1
1+1/√
2n, and α = 1.
Step 2: Solve the equation system introduced above and get(4X, dy,4S).Set Xk+1 = Xk + α4X
yk+1 = yk + αdySk+1 = Xk + α4S
with V k+1 = Xk+1 = Sk+1.Set k = k + 1.
Step 3: If Xk • Sk < ε, stop. Otherwise, go to Step 2.
Conic Programming 32 / 39
![Page 125: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/125.jpg)
Complexity
TheoremGiven the above settings, we have
(i) V k ∈ N2(β), k = 0, 1, 2, . . ..(ii) The algorithms stops in
O(√n log
X0 • S0
ε)
steps and output a primal-dual solution satisfying
Xk • Sk < ε
Conic Programming 33 / 39
![Page 126: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/126.jpg)
Example: Path Following Algorithm
min x1 + x2
s.t. x1 + x2 ≤ 3x1 − x2 ≤ 1x2 ≤ 2x1 ≥ 0, x2 ≥ 0
Figure: Path following algorithm with β = 1/2
Conic Programming 34 / 39
![Page 127: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/127.jpg)
Initialization and Improve the Performance
Initialization
• Big-M Method• Two-Phase Method• Self-Dual Embedding Method
Different Path-Following Methods
• Short Step Algorithm• Long Step Algorithm• Predictor-Corrector Algorithm• Largest Step Algorithm
Reference: Handbook of Semidefinite Programming: Theory, Algorithms,and Applications, edited by Wolkowicz H., Saigal R. and VandenbergheL., Kluwer Academic Publisher: Norwell, MA USA 2000
Conic Programming 35 / 39
![Page 128: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/128.jpg)
CVX Solution Software - Example 1
Conic Programming 36 / 39
![Page 129: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/129.jpg)
CVX Solution Software - Example 2
Conic Programming 37 / 39
![Page 130: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/130.jpg)
CVX Solution Software - Example 3
Conic Programming 38 / 39
![Page 131: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/131.jpg)
CVX Solution Software - Example 4
Conic Programming 39 / 39
![Page 132: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/132.jpg)
LCoP Part V – Modeling and Applications
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
Conic Programming 1 / 34
![Page 133: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/133.jpg)
Modeling and Applications
Content• Weber Problem• Matrix Optimization• Approximating Solutions of Linear Equations• Minimum of a Univariate Polynomial of Degree 2n
• Stochastic Queue Location Problem• Conic Reformulations of QCQP and Extensions• Robust Optimization
Conic Programming 2 / 34
![Page 134: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/134.jpg)
Weber Problem
In 1909, the German economist Alfred Weber introduced the problem offinding a best location for the warehouse of a company, such that the totaltransportation cost to serve the customers is minimum. Suppose thatthere are m customers needing to be served. Let the location ofcustomer i be ai ∈ R2, i = 1, . . . ,m. Suppose that customer may havedifferent demands, to be translated as weight ωi for customer i,i = 1, . . . ,m. Denote the desired location of the warehouse to be x.Then, the optimization problem becomes
Minm∑i=1
ωiti
s.t.
[x− aiti
]∈ L3, i = 1, . . . ,m
Conic Programming 3 / 34
![Page 135: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/135.jpg)
Matrix Optimization
Given A0, A1, . . . , Am, determine if there is y ∈ Rm such that
A0 +∑mi=1 yiAi � 0
which is equivalent to minimizing
λmax(A0 +∑mi=1 yiAi)
Notice that
t ≥ λmax(A0 +∑mi=1 yiAi) ⇔ tI −A0 −
∑mi=1 yiAi � 0
Equivalent Problem (SDP)
Min ts.t. tI −A0 −
∑mi=1 yiAi � 0
t ∈ R, y ∈ Rm
Conic Programming 4 / 34
![Page 136: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/136.jpg)
Approximating Solutions of LinearEquations
Given A ∈ Rm×n, b ∈ Rm, solve
Ax = b
Approximation:• l1 norm:
minx‖Ax− b‖1 ⇔ min
∑mi=1 ti
s.t. −ti ≤ Ai·x− bi ≤ ti, i = 1, . . . ,m.
• l2 norm:
minx‖Ax− b‖2 ⇔
min t
s.t.
[Ax− b
t
]∈ Lm+1
Conic Programming 5 / 34
![Page 137: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/137.jpg)
Approximating Solutions of LinearEquations
• l∞ norm:
minx‖Ax− b‖∞ ⇔ min t
s.t. −t ≤ Ai·x− bi ≤ t, i = 1, . . . ,m.
• Logarithm approximation: (b >Rm+
0)
minx
max1≤i≤m
| log(Ai·x)−log bi| ⇔min ts.t. 1/t ≤ Ai·x/bi ≤ t, i = 1, . . . ,m,
t > 0
⇔
min t
s.t.
t−Ai·x/bi 0 00 Ai·x/bi 10 1 t
� 0
Conic Programming 6 / 34
![Page 138: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/138.jpg)
Minimum of a Univariate Polynomial
Consider the problem of finding the minimum of a univariate polynomialof degree 2n:
Min x2n + a1x2n−1 + · · ·+ a2n−1x+ a2n
s.t. x ∈ R
This problem is equivalent to
Max ts.t. x2n + a1x
2n−1 + · · ·+ a2n−1x+ a2n − t ≥ 0 for all x ∈ R
Conic Programming 7 / 34
![Page 139: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/139.jpg)
Minimum of a Univariate Polynomial
It is well known that a univariate polynomial is nonnegative over the realdomain if and only if it can be written as sum of squares (SOS), which isequivalent to saying that there must be a positive semidefinite matrixX ∈ Sn+1
+ such that
x2n+a1x2n−1+· · ·+a2n−1x+a2n−t = (1, x, x2, · · · , xn)X(1, x, x2, · · · , xn)T .
Hence, this problem can be cast as an SDP:
Max ts.t. a2n − t = X11
a2n−k =∑
i+j=k+2
Xij , k = 1, . . . , 2n− 1
X(n+1),(n+1) = 1X ∈ Sn+1
+
Conic Programming 8 / 34
![Page 140: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/140.jpg)
Stochastic Queue Location Problem
BackgroundSuppose there are m potential customers to serve in the region.Customers’ demands are random, and once a customer calls for service,then the server in the service center will need to go to the customer toprovide the required service. In case the server is occupied, then thecustomer will have to wait. The goal is to find a good location for theservice center in order to minimize the expected waiting time of service.
Conic Programming 9 / 34
![Page 141: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/141.jpg)
Stochastic Queue Location Problem
Assumptions and notationsSuppose that the service calls from the customer are identicallyindependent distributed, and the demand process follows the Poissondistribution with overall arrival rate λ, and the probability that any servicecall is from customer i is assumed to be pi for i = 1, . . . ,m. The queueingprinciple is First Come First Service, and there is only one server in theservice center. This model can be regarded as M/G/1 queue as inQueueing theory, and the expected service time, including waiting timeand traveling, can be explicitly computed. To this end, denote the velocityof the server to be v, and the location of customer i is ai, i = 1, . . . ,m,and the location of the service center to be x.
Conic Programming 10 / 34
![Page 142: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/142.jpg)
Stochastic Queue Location Problem
Problem formulationThe expected waiting time for customer i is given by
ωi(x) =
(2λ/v2)m∑j=1
pj‖x− aj‖22
1− (2λ/v)m∑j=1
pj‖x− aj‖22+
1
v‖x− ai‖2,
where the first term in the expected term is the expected waiting time forthe server to be free and the second the term is the waiting time for theserver to travel after his departure at the service center.
Conic Programming 11 / 34
![Page 143: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/143.jpg)
Stochastic Queue Location Problem
Observing the fact that
‖x‖22/s ≤ t, s > 0⇐⇒∣∣∣∣∣∣∣∣[ x
t−s2
]∣∣∣∣∣∣∣∣2
≤ t+ s
2
We can formulate this problem as an SOCP:
Min (2mλ/v2)m∑i=1
piti + (1/v)m∑i=1
t0i
s.t.
[x− ait0i
]∈ L3,
x− aiti−s2
ti+s2
∈ L4, i = 1, ...,m
s ≤ 1− (2λ/v)m∑i=1
pisi,
[x− aisi
]∈ L3, i = 1, ...,m.
Conic Programming 12 / 34
![Page 144: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/144.jpg)
Quadratically Constrained QuadraticProgramming (QCQP)
QCQP:
min xTA0x+ 2bT0 x+ c0
s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m,
Ai ∈ Sn, bi ∈ Rn, ci ∈ R, i = 0, 1, . . . ,m.
Homogeneous QCQP(HQCQP):
min xTA0x
s.t. xTAix+ ci ≤ 0, i = 1, . . . ,m
Conic Programming 13 / 34
![Page 145: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/145.jpg)
Convex QCQP
Convex QCQP: A0, . . . , Am � 0Assume rank(Ai) = di, then Ai = BTi Bi, Bi ∈ Rdi×n, i = 0, . . . ,m.
xTAix+ 2bTi x+ ci ≤ 0 ⇔
Bix−1/2− bTi x− ci/21/2− bTi x− ci/2
∈ Ldi+2
Reformulating Convex QCQP as SOCP:
min u
s.t.
B0x−1/2− bT0 x+ u/2− ci/21/2− bT0 x+ u/2− ci/2
∈ Ld0+2
Bix−1/2− bTi x− ci/21/2− bTi x− ci/2
∈ Ldi+2 i = 1, . . . ,m
Conic Programming 14 / 34
![Page 146: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/146.jpg)
SDP Relaxation for QCQP
Recall that
xTAix+ 2bTi x+ ci =
[1x
]T [ci bTibi Ai
] [1x
]=
[ci bTibi Ai
]•[
1 xT
x xxT
]SDP Relaxation
min
[c0 bT0b0 A0
]• Y
s.t.
[ci bTibi Ai
]• Y ≤ 0, i = 1, . . . ,m
Y11 = 1Y � 0
Conic Programming 15 / 34
![Page 147: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/147.jpg)
SDP Relaxation for HQCQP
Recall thatxTAix = Ai • xxT
SDP Relaxation
min A0 •Xs.t. Ai •X ≤ −ci, i = 1, . . . ,m
X � 0
Conic Programming 16 / 34
![Page 148: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/148.jpg)
Exact Solutions from SDP Relaxation
TheoremWhen m = 1, if the optimal solution of the SDP relaxation for QCQPexists, then there exists a rank one optimal solution Y ∗ of the SDP
relaxation for QCQP. Furthermore, let Y ∗ =
[1 (x∗)T
x∗ x∗(x∗)T
], then x∗ is
optimal for QCQP.
Conic Programming 17 / 34
![Page 149: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/149.jpg)
Exact Solutions from SDP Relaxation
LemmaLet G be a symmetric matrix and X be a positive semidefinite matrix withrank d. Suppose that G •X ≤ (=)0. Then there exists a rank-onedecomposition X =
∑di=1 xix
Ti such that xTi Gxi ≤ (=)0, i = 1, . . . , d.
TheoremWhen m = 2, suppose there exists x such that xTAix+ 2bTi x+ ci < 0,i = 1, 2, and there exists y1 ≥ 0, y2 ≥ 0 such that[c0 bT0b0 A0
]+ y1
[c1 bT1b1 A1
]+ y2
[c2 bT2b2 A2
]� 0. If, for the optimal solution Y ∗,
either[c1 bT1b1 A1
]• Y ∗ < 0 or
[c2 bT2b2 A2
]• Y ∗ < 0, then there is no gap
between QCQP and its SDP relaxation and at least one optimal solutioncan be obtained from the rank-one decomposition of Y ∗.
Conic Programming 18 / 34
![Page 150: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/150.jpg)
Exact Solutions from SDP Relaxation
Theorem(HQCQP)When m = 2, suppose there exists x, such that xTAix+ ci < 0, i = 1, 2,and there exists y1 ≥ 0 and y2 ≥ 0 such that A0 + y1A1 + y2A2 � 0. Thenthere is no gap between HQCQP and its SDP relaxation and at least oneoptimal solution can be obtained from the rank-one decomposition of X∗.
Conic Programming 19 / 34
![Page 151: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/151.jpg)
One Extension of QCQP
Ext-QCQP
min xTA0x+ 2bT0 x+ c0
s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m,(x21, . . . , x
2n)T ∈ X
(Ext-QCQP)
where X is a closed convex set.SDP Relaxation:
min
[c0 bT0b0 A0
]• Y
s.t.
[ci bTibi Ai
]• Y ≤ 0, i = 1, . . . ,m
(Y22, Y33, . . . , Yn+1n+1)T ∈ XY11 = 1Y � 0
Conic Programming 20 / 34
![Page 152: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/152.jpg)
Exact Solutions from SDP Relaxation
TheoremSuppose there is σ ∈ {−1, 1}n+1, such that all the off-diagonal elements
of Λσ
[ci bTibi Ai
]Λσ, 0 ≤ i ≤ m, are nonpositive. If Y ∗ is an optimal solution
for the SDP relaxation for Ext-QCQP, then
x∗ = σ1(σ2√Y ∗22, σ3
√Y ∗33, . . . , σn+1
√Yn+1n+1)T
is optimal for Ext-QCQP.
Conic Programming 21 / 34
![Page 153: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/153.jpg)
More Results on Ext-QCQP
A Special Case of Ext-QCQP
min xTAx
s.t. (x21, . . . , x2n)T ∈ X
where X is a closed convex set.SDP Relaxation:
min A •Xs.t. (X11, X22, . . . , Xnn)T ∈ X
X � 0
Conic Programming 22 / 34
![Page 154: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/154.jpg)
Approximation by Randomized Algorithm
• Suppose Z � 0, Zii = 1, i = 1, . . . , n. If ξ ∼ N(0, Z) andσ ∈ {−1, 1}n with
σi =
{1 ξi ≥ 0−1 ξi < 0
then E(σσT ) = 2π arcsinZ.
• Suppose Z � 0, Zii = 1, i = 1, . . . , n. Then arcsinZ � Z � 0.• 2
π arcsin t ≤ 1− α+ αt, ∀t ∈ [−1, 1] with α = 0.87856 . . ..
Conic Programming 23 / 34
![Page 155: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/155.jpg)
Approximation by Randomized Algorithm
Given X � 0, let Z = D+XD+ + D, where D, D+, D are diagonalmatrices defined by
Dii =√Xii, D+
ii =
{(√Xii)
−1 Xii 6= 00 Xii = 0
, Dii =
{0 Xii 6= 01 Xii = 0
• X = DZD
• E(A • (DσσTD)) = A • (DE(σσT )D) = A • ( 2πD arcsin(Z)D).
Conic Programming 24 / 34
![Page 156: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/156.jpg)
Approximation by Randomized Algorithm
TheoremSuppose A � 0, and X∗ is the minimizer of SDP relaxation. Let Z, D,D+, and D be define from X∗, and ξ ∼ N(0, Z) with corresponding σ.Then
E(A • (DσσTD)) ≤ 2
πmin{xTAx|(x21, . . . , x2n)T ∈ X}
Proof:E(A • (DσσTD))
= A • ( 2πD arcsin(Z)D)
≤ A • ( 2πDZD)
= 2πA •X
∗
≤ 2π min{xTAx|(x21, . . . , x2n)T ∈ X}
Conic Programming 25 / 34
![Page 157: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/157.jpg)
Approximation by Randomized Algorithm
TheoremAssume that A � 0, Aij ≥ 0 for all i 6= j, and X∗ is the minimizer of SDPrelaxation. Let Z, D, D+ and D be defined from X∗ and ξ ∼ N(0, Z) withcorresponding σ. Then
E(A • (DσσTD)) ≤ αmin{xTAx|(x21, . . . , x2n)T ∈ X}
Proof:E(A • (DσσTD))
= A • ( 2πD arcsin(Z)D)
=∑i 6=j AijDiiDjj(
2π arcsinZij) +
∑ni=1AiiD
2ii
≤∑i 6=j AijDiiDjj(1− α+ αZij) +
∑ni=1AiiD
2ii
= (1− α)∑i,j AijDiiDjj + αA •X∗
≤ αA •X∗ ≤ αmin{xTAx|(x21, . . . , x2n)T ∈ X}Conic Programming 26 / 34
![Page 158: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/158.jpg)
Example: MAX CUT Problem
Given a graph G = (V,E,W ), find an optimal partition of the node set intotwo subsets V1 and V2 such that the weighted cut is maximized.
max∑ni=1
∑nj=1 wij
1−xixj
2
s.t. x2i = 1, i = 1, . . . , n(MAX − CUT )
Reformulation as Ext-QCQP:
Aij =
{wij i 6= j
−∑k 6=i wik i = j
, X = {e}
TheoremIf wij ≥ 0, ∀i 6= j. Then the expected value of randomized algorithm is atleast α ≈ 0.878 times the value of the maximum cut.
Conic Programming 27 / 34
![Page 159: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/159.jpg)
Robust Optimization
min f0(x)
s.t. fi(x, ui) ≤ 0
∀ui ∈ Ui, i = 1, . . . ,m.
Motivation
• The parameters are inexact.• The parameters cannot be foreseen.• The parameters may vary with time and other environment factors.
Conic Programming 28 / 34
![Page 160: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/160.jpg)
Example: Robust Linear Program
min cTx
s.t. aTi x ≤ bi
∀ai ∈ {a0i +∑pj=1 uja
ji | ‖u‖2 ≤ 1},
i = 1, . . . ,m
For any x
maxai aTi x = maxu(a0i )
Tx+∑pj=1 uj(a
ji )Tx
= (a0i )Tx+ ‖((a1i )Tx, . . . , (a
pi )Tx)T ‖2
Conic Programming 29 / 34
![Page 161: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/161.jpg)
Example: Robust Linear Program
Therefore
aTi x ≤ bi,∀ai ⇔ ‖((a1i )Tx, . . . , (api )Tx)T ‖2 ≤ bi − (a0i )
Tx
Equivalent SOCP
min cTx
s.t.
(a1i )
Tx...
(api )Tx
bi − (a0i )Tx
∈ Lp+1, i = 1, . . . ,m
Conic Programming 30 / 34
![Page 162: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/162.jpg)
Robust Convex QCQP
min fTx
s.t. xTATAx ≤ 2bTx+ c
∀(A, b, c) ∈ {(A0, b0, c0) +∑pj=1 uj(A
j , bj , cj) | ‖u‖2 ≤ 1},
Let U(x) = (A0x,A1x, . . . , Apx) and
V (x) =
c0 + 2(b0)Tx 1
2c1 + (b1)Tx · · · 1
2cp + (bp)Tx
12c
1 + (b1)Tx 0...
. . .12cp + (bp)Tx 0
Conic Programming 31 / 34
![Page 163: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/163.jpg)
Robust Convex QCQP
xTATAx ≤ 2bTx+ c
⇔[
1u
]TUT (x)U(x)
[1u
]−[
1u
]TV (x)
[1u
]≤ 0,∀‖u‖2 ≤ 1
Lemma (S-lemma)Let P and Q be symmetric matrices and yTPy > 0 for some y. Then theimplication
zTPz ≥ 0 ⇒ zTQz ≥ 0
is valid if and only if Q− λP � 0 for some λ ≥ 0.
Conic Programming 32 / 34
![Page 164: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/164.jpg)
Robust Convex QCQP
From S-lemma,[1u
]TUT (x)U(x)
[1u
]−[
1u
]TV (x)
[1u
]≤ 0,∀‖u‖2 ≤ 1
⇔ V (x)− UT (x)U(x)− λ[1−I
]� 0,∃λ ≥ 0
⇔
V (x)− λ[1−I
]UT (x)
U(x) I
� 0,∃λ ≥ 0
(by Schur complementary theorem)
Conic Programming 33 / 34
![Page 165: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/165.jpg)
Robust Convex QCQP
Equivalent SDP
min fTx
s.t.
c0 + 2(b0)Tx− λ 1
2c1 + (b1)Tx · · · 1
2cp + (bp)Tx (A0x)T
12c
1 + (b1)Tx λ (A1x)T
.... . .
...12cp + (bp)Tx λ (Apx)T
A0x A1x · · · Apx I
� 0
Conic Programming 34 / 34
![Page 166: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/166.jpg)
LCoP Part VI – Recent Research
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
Conic Programming 1 / 46
![Page 167: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/167.jpg)
Recent Research
Content• Quadratically Constrained Quadratic Programming
(QCQP)
Conic Programming 2 / 46
![Page 168: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/168.jpg)
Quadratically Constrained QuadraticProgramming (QCQP)
QCQP:
Min xTA0x+ 2bT0 x+ c0
s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m
Ai ∈ Sn, bi ∈ Rn, ci ∈ R, i = 0, 1, . . . ,m.
Homogeneous QCQP(HQCQP):
Min xTA0x
s.t. xTAix+ ci ≤ 0, i = 1, . . . ,m
Conic Programming 3 / 46
![Page 169: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/169.jpg)
Convex QCQP
Convex QCQP: A0, . . . , Am � 0Assume rank(Ai) = di, then Ai = BTi Bi, Bi ∈ Rdi×n, i = 0, . . . ,m.
xTAix+ 2bTi x+ ci ≤ 0 ⇔
Bix−1/2− bTi x− ci/21/2− bTi x− ci/2
∈ Ldi+2
Reformulating Convex QCQP as SOCP:
Min u
s.t.
B0x−1/2− bT0 x+ u/2− c0/21/2− bT0 x+ u/2− c0/2
∈ Ld0+2
Bix−1/2− bTi x− ci/21/2− bTi x− ci/2
∈ Ldi+2 i = 1, . . . ,m
Conic Programming 4 / 46
![Page 170: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/170.jpg)
SDP Relaxation for QCQP
Recall that
xTAix+ 2bTi x+ ci =
[1x
]T [ci bTibi Ai
] [1x
]=
[ci bTibi Ai
]•[
1 xT
x xxT
]
Let Y =
[1x
] [1x
]T=
[1 xT
x xxT
]∈ Sn+1
+ , we have SDP Relaxation
Min
[c0 bT0b0 A0
]• Y
s.t.
[ci bTibi Ai
]• Y ≤ 0, i = 1, . . . ,m
Y11 = 1
Y � 0, rank(Y ) = 1
Note: No convexity is assumed.
Conic Programming 5 / 46
![Page 171: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/171.jpg)
SDP Relaxation for HQCQP
Recall thatxTAix = Ai • xxT
Let X = xxT ∈ Sn+, we have SDP Relaxation
Min A0 •Xs.t. Ai •X ≤ −ci, i = 1, . . . ,m
X � 0, rank(X) = 1
Conic Programming 6 / 46
![Page 172: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/172.jpg)
Exact Solutions from SDP Relaxation
TheoremWhen m = 1, if the optimal solution of the SDP relaxation for QCQPexists, then there exists a rank one optimal solution Y ∗ of the SDP
relaxation for QCQP, and, furthermore, let Y ∗ =
[1 (x∗)T
x∗ x∗(x∗)T
], then x∗
is optimal for QCQP.
Conic Programming 7 / 46
![Page 173: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/173.jpg)
Exact Solutions from SDP Relaxation
LemmaLet G be a symmetric matrix and X be a positive semidefinite matrix withrank d. Suppose that G •X ≤ (=)0. Then there exists a rank-onedecomposition X =
∑di=1 xix
Ti such that xTi Gxi ≤ (=)0, i = 1, . . . , d.
TheoremWhen m = 2, assume there exists x such that xTAix+ 2bTi x+ ci < 0,i = 1, 2, and there exists y1 ≥ 0, y2 ≥ 0 such that[c0 bT0b0 A0
]+ y1
[c1 bT1b1 A1
]+ y2
[c2 bT2b2 A2
]� 0. If, for the optimal solution Y ∗,
either[c1 bT1b1 A1
]• Y ∗ < 0 or
[c2 bT2b2 A2
]• Y ∗ < 0, then there is no gap
between QCQP and its SDP relaxation, and at least one optimal solutioncan be obtained from the rank-one decomposition of Y ∗.
Conic Programming 8 / 46
![Page 174: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/174.jpg)
Exact Solutions from SDP Relaxation
Theorem(HQCQP)When m = 2, suppose there exists x, such that xTAix+ ci < 0, i = 1, 2,and there exists y1 ≥ 0 and y2 ≥ 0 such that A0 + y1A1 + y2A2 � 0. Thenthere is no gap between HQCQP and its SDP relaxation, and at least oneoptimal solution can be obtained from the rank-one decomposition of X∗.
Conic Programming 9 / 46
![Page 175: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/175.jpg)
One Extension of QCQP
Ext-QCQPMin xTA0x+ 2bT0 x+ c0
s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m(x2
1, . . . , x2n)T ∈ X
(Ext-QCQP)
where X is a closed convex set.SDP Relaxation:
Min
[c0 bT0b0 A0
]• Y
s.t.
[ci bTibi Ai
]• Y ≤ 0, i = 1, . . . ,m
(Y22, Y33, . . . , Y(n+1),(n+1))T ∈ X
Y11 = 1
Y � 0, rank(Y ) = 1
Conic Programming 10 / 46
![Page 176: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/176.jpg)
Exact Solutions from SDP Relaxation
TheoremSuppose there is σ ∈ {−1, 1}n+1, such that all the off-diagonal elements
of Λσ
[ci bTibi Ai
]Λσ, 0 ≤ i ≤ m, are nonpositive. Here
(Λσ)ij =
{σi i = j0 i 6= j
If Y ∗ is an optimal solution for the SDP relaxation for Ext-QCQP, then
x∗ = σ1(σ2
√Y ∗22, σ3
√Y ∗33, . . . , σn+1
√Y(n+1),(n+1))
T
is optimal for Ext-QCQP.
Conic Programming 11 / 46
![Page 177: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/177.jpg)
More Results on Ext-QCQP
A Special Case of Ext-QCQP
Min xTAx
s.t. (x21, . . . , x
2n)T ∈ X
where X is a closed convex set.SDP Relaxation:
Min A •Xs.t. (X11, X22, . . . , Xnn)T ∈ X
X � 0
The previous theorem is satisfied automatically. Hence the optimalsolution for this special case of Ext-QCQP can be obtained by its SDPrelaxation.
Conic Programming 12 / 46
![Page 178: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/178.jpg)
Recent Results on QCQP
Content• Introduction• KKT solution and Conic Relaxation for QCQP• Adaptive Approximation to QCQP
Conic Programming 13 / 46
![Page 179: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/179.jpg)
Introduction
QCQP:
Min f(x) = xTA0x+ 2bT0 x+ c0
s.t. gi(x) = xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m,
Ai ∈ Sn, bi ∈ Rn, ci ∈ R, i = 0, 1, . . . ,m.Feasible domain:
feas(QCQP) = {x ∈ Rn|gi(x) ≤ 0, i = 1, . . . ,m}
Conic Programming 14 / 46
![Page 180: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/180.jpg)
Introduction
Previously we have learnt:• Equivalent SOCP for convex QCQP• SDP relaxation for QCQP• m = 1: optimal rank one solution from SDP relaxation• m = 2: optimal solution from non-binding solution of SDP relaxation• m = 2: optimal solution from SDP relaxation for HQCQP• Optimal solution for Ext-QCQP with all nonpositive off-diagonal
coefficients
Conic Programming 15 / 46
![Page 181: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/181.jpg)
Recall
• Nonnegative quadratic functions over F
f(x) = xTAx+ 2bTx+ c ≥ 0,∀x ∈ F
f ⇔[c bT
b A
]• DF = {
[c bT
b A
]∈ Sn+1|
[1x
]T [c bT
b A
] [1x
]≥ 0,∀x ∈ F} is a closed,
convex cone, containing Sn+1+ .
• D∗F = cl cone{[
1 xT
x xxT
]|x ∈ F
}• (D∗F )∗ = DF and (DF )∗ = D∗F• F = Rn =⇒ DF = Sn+1
+ , D∗F = Sn+1+
• F = Rn+ =⇒ DF = Cn+1, D∗F = C∗n+1
Conic Programming 16 / 46
![Page 182: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/182.jpg)
Question
How to use the cone of nonnegative quadratic functions to solve QCQP?
• Relation to KKT solution• Adaptively approximation
Conic Programming 17 / 46
![Page 183: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/183.jpg)
KKT solution and Conic Relaxation forQCQP
Ideas:
Conic Programming 18 / 46
![Page 184: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/184.jpg)
Constraint Qualification: LICQ
Active setGiven a feasible solution x, the active set I(x) is defined as
I(x) = {1 ≤ i ≤ m | gi(x) = 0}.
Linear independence constraint qualification (LICQ)Given a feasible solution x, if the vectors ∇gi(x), i ∈ I(x), are linearlyindependent, then the LICQ condition holds at x.
Conic Programming 19 / 46
![Page 185: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/185.jpg)
Lagrangian Function
Define
A(λ) = A0 +m∑i=1
λiAi, b(λ) = b0 +m∑i=1
λibi, c(λ) = c0 +m∑i=1
λici.
Lagrangian function of QCQP
L(x, λ) = f(x) +m∑i=1
λigi(x)
= xTA(λ)x+ 2bT (λ)x+ c(λ)
where λ ∈ Rm+ .
Conic Programming 20 / 46
![Page 186: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/186.jpg)
Karush-Kuhn-Tucker Conditions
Given a feasible solution x∗, if it satisfies the LICQ and is a localminimum, then there exists a Lagrangian multiplier vector λ∗ such that(x∗, λ∗) is a KKT solution satisfying
∇xL(x∗, λ∗) = 2A(λ∗)x∗ + 2b(λ∗) = 0,
λ∗i gi(x∗) = λ∗i ((x
∗)TAix∗ + 2bTi x
∗ + ci) = 0,
gi(x∗) = (x∗)TAix
∗ + 2bTi x∗ + ci ≤ 0, λ∗i ≥ 0, i = 1, . . . ,m.
Conic Programming 21 / 46
![Page 187: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/187.jpg)
“Copositive Cone” w.r.t. the FeasibleDomain (Cone of Nonnegative QuadraticFunctions)
Define
Dn+1 =
{U ∈ Sn+1
∣∣∣∣ U • [ 1 xT
x xxT
]≥ 0,∀x ∈ feas(QCQP)
}D∗n+1 = cl cone
{X =
[1x
] [1x
]T ∣∣∣∣ x ∈ feas(QCQP)
}
Conic Programming 22 / 46
![Page 188: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/188.jpg)
Sufficient Conditions
Lemma (Positive semidefiniteness condition).Let (x∗, λ∗) be a KKT solution. If A(λ∗) ∈ Sn+, then x∗ is a globally optimalsolution of QCQP.
Theorem (Copositiveness condition)Let (x∗, λ∗) be a KKT solution. If M(x∗, λ∗) ∈ Dn+1, then x∗ is an optimalsolution of the QCQP problem, where
M(x∗, λ∗) =
[c(λ∗)− f(x∗) bT (λ∗)
b(λ∗) A(λ∗)
].
Conic Programming 23 / 46
![Page 189: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/189.jpg)
Recall
Conic Programming 24 / 46
![Page 190: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/190.jpg)
An Equivalent Conic ProgrammingProblem of QCQP
Min
[c0 bT0b0 A0
]• Y
s.t. Y11 = 1,Y ∈ D∗n+1.
(CoP)
TheoremCoP and QCQP have the same optimal objective value.
ProofSee Sturm and Zhang’s paper “On Cones of Nonnegative QuadraticFunctions”.
Conic Programming 25 / 46
![Page 191: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/191.jpg)
Adding Valid Inequality Constraints
CoP1:
Min
[c0 bT0b0 A0
]• Y
s.t.
[ci bTibi Ai
]• Y ≤ 0,
i = 1, . . . ,m,
Y11 = 1,Y ∈ D∗n+1.
CoD1:
Max σ
s.t.
[c(λ)− σ bT (λ)b(λ) A(λ)
]∈ Dn+1,
λ ≥Rm+
0.
TheoremIf QCQP is bounded below, then the optimal objective values of QCQP,CoP1 and CoD1 are equal.
Conic Programming 26 / 46
![Page 192: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/192.jpg)
Finding the Lagrangian Multiplier Vector λ∗
AssumptionThere exists a KKT solution (x∗, λ∗) satisfying the LICQ condition andcopositiveness condition.
LemmaUnder the copositiveness condition assumption, (σ∗, λ∗) = (f(x∗), λ∗) isan optimal solution of CoD1. Furthermore, if the LICQ assumption holds,then for any (σ, λ) being an optimal solution of the CoD1 problem, wehave λ ≤Rn
+λ∗.
Conic Programming 27 / 46
![Page 193: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/193.jpg)
Finding the Lagrangian Multiplier Vector λ∗
TheoremLet (x∗, λ∗) be a KKT solution. If it satisfies the LICQ condition and thecopositiveness condition, then λ∗ is the unique optimal solution of themaximization problem CoD2
Max∑mi=1 λi
s.t.
[c(λ)− σ∗ bT (λ)b(λ) A(λ)
]∈ Dn+1,
λ ≥Rn+
0,
(CoD2)
where σ∗ is the optimal objective value obtained by solving CoD1.Furthermore, if A(λ∗) is invertible, then
x∗ = −A−1(λ∗)b(λ∗).
Conic Programming 28 / 46
![Page 194: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/194.jpg)
Relaxing the Conic Constraint
Choose a computable cone Kn+1 such that Sn+1+ ⊂ Kn+1 ⊂ Dn+1.
CoD1’:Max σ
s.t.
[c(λ)− σ bT (λ)b(λ) A(λ)
]∈ Kn+1,
λ ≥Rn+
0.
CoD2’:Max
∑mi=1 λi
s.t.
[c(λ)− σ∗ bT (λ)b(λ) A(λ)
]∈ Kn+1,
λ ≥Rn+
0.
Conic Programming 29 / 46
![Page 195: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/195.jpg)
Algorithm (QCQP)
STEP 1: Given a QCQP problem, solve the corresponding CoD1’ problem andget the optimal value σ∗. If failed, then stop and the problem cannot besolved by the current scheme.
STEP 2: Solve CoD2’ to get the optimal λ∗.STEP 3: Compute x∗ = −A+(λ∗)b(λ∗).STEP 4: If (x∗, λ∗) is a KKT solution, then return x∗ as a global optimal solution
of QCQP with the objective value f(x∗) = σ∗. Otherwise, return σ∗ asa lower bound of QCQP.
Here A+ is the Moore-Penrose generalized inverse for a given squarematrix A ∈ Rn×n.
Conic Programming 30 / 46
![Page 196: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/196.jpg)
Example
Min f(x) = xTA0x+ 2bT0 x
s.t. g1(x) = xTA1x+ 2bT1 x+ c1 ≤ 0,
g2(x) = xTA2x+ 2bT2 x+ c2 ≤ 0,
where
A0 =
−2 10 210 4 12 1 −7
, b0 =
−12−656
,A1 =
2 0 00 −2 00 0 8
, b1 =
0−2−64
, c1 = 512,
A2 =
2 0 00 2 00 0 2
, b2 =
−20−16
, c2 = 128.
Conic Programming 31 / 46
![Page 197: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/197.jpg)
Example
• The feasible domain is a subset of R3+.
• If we choose K4 = S4+ (SDP relaxation), then the optimal value of the relaxed problem
is 222.88.• Instead, choose K4 = S4
+ +N4.
• x∗ = (0, 0, 8)T with f(x∗) = 224 and λ∗ = (1, 2)T .
M(x∗, λ∗) =
320 −16 −8 −40−16 4 10 2−8 10 6 1−40 2 1 5
=
320 −16 −8 −40−16 4 0 2−8 0 6 1−40 2 1 5
+
0 0 0 00 0 10 00 10 0 00 0 0 0
∈ S4+ +N4,
A0 + λ∗1A1 + λ
∗2A2 =
4 10 210 6 12 1 5
/∈ S3+.
Conic Programming 32 / 46
![Page 198: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/198.jpg)
Summary
• Equivalent conic programming problems.
• The copositiveness condition which ensures a KKT solution beingoptimal.
• The maximal property of the Lagrangian multiplier vector under theLICQ condition and the copositiveness condition.
• Conic programming relaxation.• A way to design polynomial time algorithm to solve a class of QCQP.
Conic Programming 33 / 46
![Page 199: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/199.jpg)
Adaptive Approximation to QCQP
Consider a box constrained quadratic program
Min xTA0x+ 2bT0 x+ c0
s.t. x ∈ G = [0, 1]n(BQP)
This problem is in general NP-hard.
Is there any polynomial time approximation?
Find an ellipsoid cover F1 ∪ · · · ∪ Fk ⊃ G such that the problem
Min xTA0x+ 2bT0 x+ c0
s.t. x ∈ F1 ∪ · · · ∪ Fk(BQPR)
satisfies opt(BQP) ≤ opt(BQPR) + ε.
Conic Programming 34 / 46
![Page 200: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/200.jpg)
Conic Reformulation
Let H =
[c0 bT0b0 A0
], BQP becomes
Min H • Y
s.t. Y11 = 1
Y ∈ D∗G
(BCoP)
and BQPR becomes
Min H • Y
s.t. Y11 = 1
Y ∈ D∗F1∪···∪Fk
(BCoPR)
Max σ
s.t. σ
[1 00 0
]+ S = H
S ∈ DF1∪···∪Fk
Conic Programming 35 / 46
![Page 201: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/201.jpg)
Conic Reformulation
Lemma
DF1∪···∪Fk= DF1 ∩ · · · ∩ DFk
and
D∗F1∪···∪Fk= D∗F1
+ · · ·+D∗Fk
Rewrite the relaxation BCoPR as
Min H • Y
s.t. Y11 = 1
Y = Y1 + · · · , YkYi ∈ D∗Fi
, i = 1, . . . , k
(BCRP)
What is DF1∪···∪Fkfor the dual problem of BCRP?
Conic Programming 36 / 46
![Page 202: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/202.jpg)
Conic Reformulation: Dual Problem
Lemma (Linear Matrix Inequalities (LMI))Suppose each ellipsoid Fi = {x ∈ Rn|xTAix+ 2bTi x+ ci 6 0} with someAi � 0 and Int(Fi) 6= ∅. Then, M ∈ DF1∪···∪Fk
if and only if the followingsystem is feasible:
M + λi
[ci bTibi Ai
]� 0, λi > 0 for i = 1, 2, ..., k.
Dual of BCoRP:Max σ
σ
[1 00 0
]+ S = H
S + λi
[ci bTibi Ai
]� 0, λi ≥ 0,
i = 1, . . . , k.
(BCRD)
Conic Programming 37 / 46
![Page 203: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/203.jpg)
Optimality Condition
(Y ∗, Y ∗1 , ..., Y∗k ) is feasible to BCRP,
(σ∗, λ∗, S∗) is feasible to BCRD,
S∗ · Y ∗i = 0 and λ∗i
[bi cTici Qi
]• Y ∗i = 0, i = 1, 2, ...k.
Conic Programming 38 / 46
![Page 204: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/204.jpg)
Some Properties
LemmaS∗ is on the boundary of DF1∪···∪Fk
.
Lemma (Rank-one decomposition)Y ∗ can be decomposed in the form of
Y ∗ =k∑i=1
ni∑j=1
µij
[1xij
] [1xij
]T
with xij ∈ Fi, µij > 0 and∑ki=1
∑ni
j=1 µij = Y ∗11 = 1.Moreover, [
1xij
]TS∗[
1xij
]= 0,∀xij
Conic Programming 39 / 46
![Page 205: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/205.jpg)
Improving The Relaxation
TheoremIf any xij decomposed from Y ∗ is feasible for BQP, then it is an optimalsolution of BQP.
However, in general, they are not feasible.
Let x = xij = arg mini,j
[1xij
]TH
[1xij
]with corresponding Fi.
Notice that for Fi, the feasible part is Fi ∩ G. If we could cover Fi ∩ G bysmaller sets, whose union does not contain x, then the solution of thenew cover may be improved.
Conic Programming 40 / 46
![Page 206: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/206.jpg)
Improving The Relaxation
For simplicity, the ellipsoids have to be constructed carefully.
DefinitionLet T = {x ∈ Rn|ui 6 xi 6 vi} be a rectangular set in Rn, define aquadratic function
gT (x) =
n∑i=1
(2xi − vi − ui)2
(vi − ui)2− n for x ∈ Rn,
and an ellipsoidal set FT = {x ∈ Rn|gT (x) 6 0}. Then we say FT is thecorresponding ellipsoid of T .
Conic Programming 41 / 46
![Page 207: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/207.jpg)
Algorithm
STEP 1: Set T = G and Nmax. Set k = 1 and let the cover be F1 = FT .STEP 2: Solve BCRP and BCRD defined by the cover F1 ∪ · · · ∪ Fk to obtain Y ∗
and (σ∗, λ∗, S∗). Set lk = σ∗.STEP 3: Decompose Y ∗ to obtain x and Fi.STEP 4: If x ∈ G, then x is optimal. Stop and return.
Otherwise, set T be the rectangle covered by Fi.Find i∗ = arg maxi{max(xi − 1,−xi)}.Partition T into T1 = {x ∈ T |xi∗ ≤ ui∗+vi∗
2} and
T2 = {x ∈ T |xi∗ ≥ ui∗+vi∗2}.
Drop Fk from the cover, and add FT1 and FT2 into the cover.Set k = k + 1.
STEP 5: If k < Nmax, then go to STEP 2. Otherwise return max{l1, ..., lk} as alower bound for BQP.
Conic Programming 42 / 46
![Page 208: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/208.jpg)
Convergence
TheoremLet l∗k = maxi=1,...,k{li} be the largest lower bound up to the kth iteration,then limk→+∞ l∗k = opt(BQP ).
Conic Programming 43 / 46
![Page 209: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/209.jpg)
Example
Min xTA0x+ 2bT0 x
s.t. x ∈ [0, 1]10
with
A0 =
54 24 −79 88 11 −109 −76 −14 250 8124 303 −24 −61 24 −6 −5 112 85 76−79 −24 72 28 40 −9 64 −25 25 −388 −61 28 −81 −132 16 71 7 148 4411 24 40 −132 −86 −2 69 −7 14 22−109 −6 −9 16 −2 153 −34 −52 24 65−76 −5 64 71 69 −34 −149 −108 −11 113−14 112 −25 7 −7 −52 −108 49 −21 −106250 85 25 148 14 24 −11 −21 22 −8081 76 −3 44 22 65 113 −106 −80 −179
and
b0 =1
2[84 − 89 10 − 54 30 − 60 49 74 171 − 19]T .
Conic Programming 44 / 46
![Page 210: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/210.jpg)
Example
The optimal value is -611.
0 10 20 30 40 50−670
−660
−650
−640
−630
−620
−610
Iterations
Low
er B
ound
s
Adaptive BoundSDP Bound
Conic Programming 45 / 46
![Page 211: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/211.jpg)
Summary
• Developed an adaptive relaxation for BQP.• It applies to convex constrained QCQP too.• It may be applied to general QCQP, but needs well-designed covers.• Other techniques such as RLT can be involved in the relaxation.
Conic Programming 46 / 46
![Page 212: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/212.jpg)
LCoP Part VII – Practical LCoP
Shu-Cherng Fang
Department of Industrial and Systems EngineeringGraduate Program in Operations Research
North Carolina State University
December 2017
Conic Programming 1 / 40
![Page 213: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/213.jpg)
Practical LCoP
Content• Convex Cones and Properties• Different Forms of LCoP• SOCP Formulation• SDP Formulation
Conic Programming 2 / 40
![Page 214: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/214.jpg)
Convex Cones and Properties
K, K1, K2 are convex cones in Rn.• (K∗)∗ = cl(K).• K1 ⊆ K2 ⇒ K∗2 ⊆ K∗1 .• K1 ∩K2, K1 ∪K2, K1 +K2, K1 ×K2 are all cones.• (K1 +K2)∗ = K∗1 ∩K∗2 .• K1, K2 closed⇒ K1 +K2 and K1 ×K2 are closed.• ri(K1 +K2) = ri(K1) + ri(K2).• ri(K1 ×K2) = ri(K1)× ri(K2).• The supporting hyperplane of K always contains the origin• If K is solid, then K∗ is pointed.• If K is pointed, then K∗ is solid.• If K is proper, then K∗ is proper.
Conic Programming 3 / 40
![Page 215: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/215.jpg)
Convex Cones and Properties
K, K1, K2 are convex cones in Rn.• Rn+ ⊆ Rn, Ln ⊆ Rn, Sn+ ⊆ Rn×n are pointed, closed, convex and
solid cones, i.e., proper cones.• (Rn+)∗ = Rn+, (Ln)∗ = Ln, (Sn+)∗ = Sn+.
• Rn1+ ×R
n2+ × · · · ×R
nk+ is a proper cone in R
k∑i=1
ni
.
• Ln1 × Ln2 × · · · × Lnk is a proper cone in Rk∑
i=1ni
.
• Sn1+ × S
n2+ × · · · × S
nk+ is a proper cone in R
k∑i=1
ni×ni
.• Rn1
+ × Ln2 × Sn3+ is a proper cone in Rn1+n2+n3×n3 .
Conic Programming 4 / 40
![Page 216: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/216.jpg)
Different Forms of LCoP
• LP (Given A ∈ Rm×n,b ∈ Rm, ai ∈ Rn, bi ∈ R and c ∈ Rn)
Min cTxs.t. Ax = b
x ≥ 0(LP)
Max bT ys.t. AT y ≤ c
y ∈ Rm(LD)
Min c • xs.t. ai • x = bi, i = 1, . . . ,m
x ∈ Rn+(ai ∈ Rn, bi ∈ R)
(LP)Max bT ys.t. AT y + s = c
y ∈ Rm, s ∈ Rn+(LD)
Max b • ys.t. c−AT y ∈ Rn+
y ∈ Rm(LD)
Conic Programming 5 / 40
![Page 217: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/217.jpg)
Different Forms of LCoP
• SOCP (Given A ∈ Rm×n, b ∈ Rm, c ∈ Rn, ai ∈ Rn, bi ∈ R)
Min cTxs.t. Ax = b
x ∈ Ln(SOCP)
Max bT ys.t. AT y + s = c
y ∈ Rm, s ∈ Ln(SOCD)
Min c • xs.t. ai • x = bi, i = 1, . . . ,m
x ≥Ln 0(SOCP)
Max bT ys.t. c−AT y ∈ Ln
y ∈ Rm(SOCD)
Conic Programming 6 / 40
![Page 218: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/218.jpg)
Different Forms of LCoP
• SDP (Given Ai ∈ Rn×n, bi ∈ R and C ∈ Rn×n)
Min C •Xs.t. Ai •X = bi, i = 1, · · · ,m
X � 0(SDP)
Max bT y
s.t.m∑i=1
yiAi + S = C
y ∈ Rm, S ≥Sn+
0
(SDD)
Min C •Xs.t. Ai •X = bi, i = 1, · · · ,m
X ∈ Sn+(SDP)
Max bT y
s.t. C −m∑i=1
yiAi � 0
y ∈ Rm(SDD)
Max bT y
s.t. C −m∑i=1
yiAi ∈ Sn+yi ∈ R, i = 1, . . . .m
(SDD)
Conic Programming 7 / 40
![Page 219: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/219.jpg)
Different Forms of LCoP
• Combinations
Min pTx+ qT ys.t. Ax+By = b
x ≥ 0, y ∈ Lm
Max pTx+ qT y + rT z (linear objective)s.t. Ax+By ≤ b (linear inequalities)
C −k∑i=1
ziAi � 0 (linear matrix inequalities)
x ≥ 0, y ∈ Lm, z ∈ Rk
Conic Programming 8 / 40
![Page 220: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/220.jpg)
Second Order Cone Program (SOCP)Formulation
• Combinations
Min c • xs.t. ai • x = bi, i = 1, . . . ,m
x ∈ K
where K = Ln1 × · · · × Lnr = {x ∈ Rn |r∑i=1
ni = n, (x1, · · · , xn1) ∈
Ln1 , · · · , (xn−nr+1, · · · , xn) ∈ Lnr}, c, ai ∈ Rn, bi ∈ R.
Conic Programming 9 / 40
![Page 221: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/221.jpg)
Second Order Cone Program (SOCP)Formulation
• For easy thinking, take r = 1 and n = n1.
Min cTxs.t. Ax = b
x ∈ K[Conic variables]
(SOCP)
Max bT ys.t. AT y + S = C
y ∈ Rm, S ∈ K(i.e., C −AT y ∈ K)[Conic constraints]
(SOCD)
Conic Programming 10 / 40
![Page 222: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/222.jpg)
SOCP Formulation
• Generalization:Since linear constraints are allowed, instead of considering x ∈ Ln,we may consider
y = Mx+ q ∈ Lm
i.e., a linear transformation of x in an SOC.
• Denote M =
(Am−1×ncT1×n
), q =
(bm−1×1d1×1
), we have
y =
(AcT
)x+
(bd
)=
(Ax+ bcTx+ d
)∈ Lm
⇔ ‖Ax+ b‖ ≤ cTx+ d
Conic Programming 11 / 40
![Page 223: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/223.jpg)
SOCP Formulation
• (SOCP) becomes
Min cTxs.t. ‖Ax+ b‖ ≤ cTx+ d
x ∈ Rn
where A ∈ R(m−1)×n, b ∈ Rm−1, c ∈ Rn, d ∈ R.• More generally, (SOCP) may be represented as
Min cTxs.t. ‖Aix+ bi‖ ≤ cTi x+ di, i = 1, · · · , r
x ∈ Rn
where Ai ∈ R(ni−1)×n, b ∈ Rni−1, ci ∈ Rn, di ∈ R, and x ∈ Rn with
n =r∑i=1
ni.
Conic Programming 12 / 40
![Page 224: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/224.jpg)
Questions
• Are the following problems SOCP?
Minp∑i=1
1/(aTi x+ bi)
s.t. aTi x+ bi > 0, i = 1, · · · , pcTj x+ dj ≥ 0, j = 1, · · · , q
Min xTP0x+ 2qT0 x+ r0s.t. xTPix+ 2qTi x+ ri ≤ 0, i = 1, · · · , p
P0, P1, · · · , Pp ∈ Sn++
Conic Programming 13 / 40
![Page 225: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/225.jpg)
SOC-Representable Sets and Functions
• Definition (SOC-Representable Set)
A set S ⊆ Rn is SOC-representable if there exists a linear transformation
L(x) = Mx+ q for x ∈ Swith M = (·)m×n and q ∈ Rm, such that
L(S) =
K∏i=1
Lmi for some K > 0 andK∑i=1
mi = m.
• Note that S is SOC-representable if and only if S is the x-spaceprojection of the solution set of K(> 0) SOC inequalities:
Mi
(xµ
)+ qi ∈ Lmi , i = 1, ...,K,
i.e., S = {x ∈ Rn|Mi
(xµ
)+ qi ∈ Lmi , i = 1, ...,K}.
Conic Programming 14 / 40
![Page 226: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/226.jpg)
SOC-Representable Sets and Functions
• Theorem: If S1,S2 ⊆ Rn are SOC-representable sets, then
αS1(α ≥ 0),S1 ∩ S2,S1 × S2 are SOC-representable.
• Definition (SOC-Representable Function)
A function g(·) : Rn → R is SOC-representable if its epigraph
Epi(g) = {(x, t) ∈ Rn+1 | g(x) ≤ t} is an SOC representable
set in Rn+1.
Conic Programming 15 / 40
![Page 227: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/227.jpg)
SOC-Representable Sets and Functions
• Theorem
If a function g(·) : Rn → R is SOC-representable, then any of its
lower level sets Gα = {x ∈ Rn | g(x) ≤ α} is an SOC-representable set.
Conic Programming 16 / 40
![Page 228: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/228.jpg)
SOCP for NLP
min f(x)s.t. gj(x) ≤ 0 j = 1, ..., J (NLP )
x ∈ Rn
⇐⇒min ts.t. f(x) ≤ t
gj(x) ≤ 0 j = 1, ..., Jx ∈ Rn, t ∈ R
Conic Programming 17 / 40
![Page 229: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/229.jpg)
SOCP for NLP
• TheoremIf f(·) and gj(·), j = 1, 2, ..., J , are all SOC-representable functions, then(NLP) becomes an SOCP.
Conic Programming 18 / 40
![Page 230: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/230.jpg)
Identify SOC-Representable Functions
• TheoremIf f1 and f2 are SOC-representable functions, then αf1(α ≥ 0); f1 + f2and max {f1, f2} are SOC-representable.
• TheoremIf f1 and f2 are convex and SOC-representable with f1(x) beingmonotone nondecreasing, then g(x) = f1(f2(x)) is convex andSOC-representable.
Conic Programming 19 / 40
![Page 231: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/231.jpg)
Identify SOC-Representable Functions
• Hyperbolic Constriants
Theorem: For w, x, y ∈ R,
w2 ≤ xy, x ≥ 0, y ≥ 0⇔∥∥∥∥( 2wx− y
)∥∥∥∥ ≤ x+ y ⇔
2wx− yx+ y
∈ L3.
Theorem: For w ∈ Rn, x, y ∈ R,
wTw ≤ xy, x ≥ 0, y ≥ 0⇔∥∥∥∥( 2wx− y
)∥∥∥∥ ≤ x+ y ⇔
2wx− yx+ y
∈ Ln+2.
• Quadratic Constraints
Theorem: For Q ∈ Sn++, q ∈ Rn, r ∈ R,
xTQx+ 2qTx+ r ≤ 0⇔∥∥∥Q1/2x+Q−1/2q
∥∥∥ ≤ (qTQ−1q − r)1/2.
Conic Programming 20 / 40
![Page 232: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/232.jpg)
SOC-Representable Functions
• Constant function
g(x) ≡ c, x ∈ Rn, c ∈ R
Consider the epigraph{(
xt
)∈ Rn+1 | c ≤ t
}.
Set A = (0)m×n, then ||Ax|| ≤ t− c, i.e.,(
Axt− c
)∈ Lm+1.
Conic Programming 21 / 40
![Page 233: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/233.jpg)
SOC-Representable Functions
• Linear function
g(x) = aTx+ b, a ∈ Rn, b ∈ R
Consider the epigraph{( x
t
)| aTx+ b ≤ t
}.
Set C = (0)p×n, then the set can be represented by||Cx|| ≤ t− aTx− b.
Conic Programming 22 / 40
![Page 234: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/234.jpg)
SOC-Representable Functions
• Square Root of Homogeneous Convex Quadratic Function
g(x) =√xTAx,A ∈ Sn+
The epigraph is{( x
t
)|√xTAx ≤ t
}.
Since A ∈ Sn+, there exists matrix B satisfying A = BTB, we have√(Bx)T (Bx) ≤ t, i.e., ‖Bx‖ ≤ t.
Conic Programming 23 / 40
![Page 235: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/235.jpg)
SOC-Representable Functions
• Convex Quadratic Function
g(x) = xTAx+ bTx+ c, A ∈ Sn+, b ∈ Rn, c ∈ R
g(x) ≤ t is equivalent to√(Bx)TBx+
(t− bTx− c− 1)2
4≤ t− bTx− c+ 1
2,
where B is the matrix satisfying A = BTB, i.e.,∥∥∥∥ Bxt−bT x−c−1
2
∥∥∥∥ ≤ t− bTx− c+ 1
2.
Conic Programming 24 / 40
![Page 236: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/236.jpg)
SOC-Representable Functions
• Reciprocal Function
g(x) = 1x , x > 0
g(x) ≤ t is equivalent to xt ≥ 1, x > 0, hence we have√1 +
(x− t)24
≤ x+ t
2, x > 0, t ≥ 0.
i.e., ∥∥∥∥ 1x−t2
∥∥∥∥ ≤ x+ t
2.
Conic Programming 25 / 40
![Page 237: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/237.jpg)
SOC-Representable Functions
• Maximum Function
g(x) = max{x1, x2, ..., xn}
The epigraph{( x
t
)| max{x1, x2, ..., xn} ≤ t
}={( x
t
)|
t− x1 ≥ 0, ..., t− xn ≥ 0}
.
Conic Programming 26 / 40
![Page 238: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/238.jpg)
SOC-Representable Functions
• Sum of the norms
g(x) =∑pi=1 ||Fix+ gi||, Fi ∈ Rni×n, gi ∈ Rni(
xt
)satisfies g(x) ≤ t is equivalent to ∃ t1, t2, ..., tp such that
||Fjx+ gj || ≤ tj , j = 1, ..., p and t1 + ...+ tp ≤ t.
• Max of the norms
g(x) = maxi=1,...,p ||Fix+ gi||, Fi ∈ Rni×n, gi ∈ Rni
Conic Programming 27 / 40
![Page 239: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/239.jpg)
SOC-Representable Functions
• Sum of the fractional functions
g(x) =∑pi=1 1/(aTi x+ bi), where aTi x+ bi > 0, i = 1, ..., p(
xt
)satisfies g(x) ≤ t is equivalent to ∃ t1, t2, ..., tp such that
ti(aTi x+ bi) ≥ 1, i = 1, ..., p and t1 + ...+ tp ≤ t, i.e., ∃ t1, ..., tp such
that∥∥∥∥( 2
aTi x+ bi − ti
)∥∥∥∥ ≤ aTi x+ bi + ti, i = 1, ..., p and
t1 + ...+ tp ≤ t.
Conic Programming 28 / 40
![Page 240: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/240.jpg)
SOC-Representable Functions
• Sum of the fractional functions
g(x) =∑pi=1
||Fix+gi||2aTi x+bi
, where aTi x+ bi > 0, i=1,...,p(xt
)satisfies g(x) ≤ t is equivalent to ∃ t1, t2, ..., tp such that
(Fix+ gi)T (Fi + gi) ≤ ti(aTi x+ bi), i = 1, ..., p and t1 + ...+ tp ≤ t,
i.e., ∃ t1, ..., tp such that∥∥∥∥( Fix+ giaTi x+ bi − ti
)∥∥∥∥ ≤ aTi x+ bi + ti, i = 1, ..., p and t1 + ...+ tp ≤ t.
Conic Programming 29 / 40
![Page 241: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/241.jpg)
SOC-Representable Functions
• Geometrical mean
g(x) = −(x1...xn)1/n, x ∈ Rn+, where n = 2q, q > 0
• Geometrical mean of nonnegative affine functions
g(x) = −
(p∏i=1
(aTi x+ bi)
)1/p
with aTi x+ bi ≥ 0
Conic Programming 30 / 40
![Page 242: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/242.jpg)
Semidefinite Programming (SDP)Formulation
• Let C,Ai ∈ Sn×n, b ∈ Rm, bi ∈ R.
Min C •Xs.t. Ai •X = bi, i = 1, · · · ,m
X ∈ Sn+(SDP)
Max bT y
s.t.m∑i=1
yiAi + S = C
y ∈ Rm, S ∈ Sn+
(SDD)
Min C •Xs.t. Ai •X = bi, i = 1, . . . ,m
X � 0(SDP)
Max bT y
s.t. C −m∑i=1
yiAi � 0
y ∈ Rm(SDD)
Conic Programming 31 / 40
![Page 243: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/243.jpg)
SDP Formulation
• SDD is often easier to “see” and “work” on, since it has only mvariables. We may have different interpretations of SDD.
- Linear Matrix Inequality Constraint
C −m∑i=1
yiAi � 0
y ∈ Rm
- PSD conic constraint
C −m∑i=1
yiAi ∈ Sn+
y ∈ Rm
Conic Programming 32 / 40
![Page 244: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/244.jpg)
SDP Formulation
• SDD is often easier to “see” and “work” on, since it has only mvariables. We may have different interpretations of SDD.
- Linear transformation of variables in the cones11 = c11 −A1(1, 1)y1 −A2(1, 1)y2 − · · · −Am(1, 1)ym
sij = cij −A1(i, j)y1 −A2(i, j)y2 − · · · −Am(i, j)ym
snn = cnn −A1(n, n)y1 −A2(n, n)y2 − · · · −Am(n, n)ym
S = (sij) ∈ Sn+
Conic Programming 33 / 40
![Page 245: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/245.jpg)
SDP Formulation
• Useful Facts
1.(xt
)∈ Ln+1 ⇔
[t xT
x tIn
]∈ Sn+1
+
Any SOCP is an SDP!
2. Let M =
(P vvT d
), P � 0, v ∈ Rn, d ∈ R.
M � (�)0 if and only if d− vTP−1v ≥ (>) 0.
3. Let M =
(P BBT C
), P � 0, B,C proper size.
M � (�)0 if and only if S = C −BTP−1B � (�) 0.
Conic Programming 34 / 40
![Page 246: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/246.jpg)
SDP Formulation
• Useful Facts
4. For x ∈ Rn, M = xxT ∈ Sn+.
5. For x ∈ Rn and W ∈ Sn, then(
1 xT
x W
)∈ Sn+1
+ if and only if
W � xxT .
Conic Programming 35 / 40
![Page 247: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/247.jpg)
SDP Identification
• Types of constraints that can be modeled as SDP
1. Convex quadratic constraints (Q ∈ Sn+, q ∈ Rn, c ∈ R)
xTQx+ qTx+ c ≤ 0⇔ xT (MTM)x+ qTx+ c ≤ 0
⇔(
I MxxTMT −qTx− v
)� 0
• Convex QCQPMin xTQ0x+ qT0 x+ c0s.t. xTQix+ qTi x+ ci ≤ 0 i = 1, . . . ,m
x ∈ Rn
Conic Programming 36 / 40
![Page 248: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/248.jpg)
SDP Identification
• Convex QCQP
Min θs.t. xTQ0x+ qT0 x+ c0 ≤ θ
xTQix+ qTi x+ ci ≤ 0 i = 1, . . . ,mx ∈ Rn
Min θ
s.t.
(I M0x
xTM0 −qTx− c0 + θ
)� 0(
I MixxTM0 −qTx− ci
)� 0, i = 1, . . . ,m
x ∈ Rn, θ ∈ R
Conic Programming 37 / 40
![Page 249: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/249.jpg)
SDP Identification
2. Binary constraints (x ∈ Rn)
xj ∈ {−1, 1}, j = 1, . . . , n
Y = xxT � 0 (rank-one matrix constraint)
Yjj = 1
3. Distance constraints
‖x− a‖2 ≤ 1
⇔ (x− a)T (x− a) ≤ 1
⇔(
I x− a(x− a)T 1
)� 0
Conic Programming 38 / 40
![Page 250: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/250.jpg)
SDP Identification
4. Elliposid constraints (Q � 0)
(x− a)TQ(x− a) ≤ 1
⇔ (x− a)T√Q√Q(x− a) ≤ 1
⇔(
I√Qx−
√Qa
(√Qx−
√Qa)T 1
)� 0
5. Eigenvalue constraints
λI � S � µI (µ > λ ≥ 0)
a psd with eigenvalues between λ and µ.
Conic Programming 39 / 40
![Page 251: Linear Conic Programming Theory andLinear Conic Programming Theory and Applications1 Shu-Cherng Fang Department of Industrial and Systems Engineering Graduate Program in Operations](https://reader035.vdocument.in/reader035/viewer/2022062505/5ed5ed0038044f6a1120c9aa/html5/thumbnails/251.jpg)
SDP Identification
6. Fractional constraints
(cTx)2
dTx≤ t
⇔(
t cTxcTx dTx
)� 0
Conic Programming 40 / 40