yue yu and ufuk topcu
TRANSCRIPT
Proportional-Integral Projected Gradient Method for InfeasibilityDetection in Conic Optimization
Yue Yu and Ufuk Topcu
Abstract— A constrained optimization problem is primal in-feasible if its constraints cannot be satisfied, and dual infeasibleif the constraints of its dual problem cannot be satisfied. Wepropose a novel iterative method, named proportional-integralprojected gradient method (PIPG), for detecting primal anddual infeasiblity in convex optimization with quadratic objectivefunction and conic constraints. The iterates of PIPG eitherasymptotically provide a proof of primal or dual infeasibility,or asymptotically satisfy a set of primal-dual optimality condi-tions. Unlike existing methods, PIPG does not compute matrixinverse, which makes it better suited for large-scale and real-time applications. We demonstrate the application of PIPG inquasiconvex and mixed-integer optimization using examples inconstrained optimal control.
I. INTRODUCTION
Conic optimization is the minimization of a quadraticobjective function subject to conic constraints:
minimizez
12zJPz ` qJz
subject to Hz ´ g P K, z P D,(1)
where z P Rn is the optimization variable, symmetric posi-tive semidefinite matrix P P Rnˆn and vector q P Rn definethe quadratic objective function, matrix H P Rmˆn andvector g P Rm are constraint parameters, cone K Ď Rm andset D Ď Rn are nonempty, closed, and convex. Optimization(1) includes many common convex optimization problems asspecial cases, including linear, quadratic, second-order cone,and semidefinite programing [1, 2].
Optimization (1) is primal infeasible if there exists noz P D such that Hz ´ g P K, it is dual infeasible ifthe constraints of its dual problem cannot be satisfied. Ifoptimization (1) is not primal infeasible, the infeasibility ofits dual problem implies that the primal optimal value isunbounded from below [3].
Given optimization (1), infeasibility detection is the prob-lem of providing a proof of primal or dual infeasibility if itis the case [3]. In particular, a proof of primal infeasibilityis the existence of a hyperplane that separates cone K fromthe set tHz ´ g|z P Du. A proof of dual infeasibility isthe existence of a vector z P Rn that satisfies the followingconditions for all z P D [4]:
Pz “ 0, qJz ă 0, Hz P K, z ` z P D.
Infeasibility detection is necessary for adjusting the pa-rameters in pathological optimization problems [5], and an
The authors are with the Oden Institute for Computational Engineeringand Sciences, The University of Texas at Austin, Austin, TX, 78712, USA(e-mails: [email protected],[email protected]).
integral subproblem in quasiconvex optimization [2, 6] andmixed-integer optimization [7, 8].
Infeasibility detection via Douglas-Rachford splittingmethod (DRS) has recently attracted increasing attention[9, 10, 11, 3]. DRS detects infeasibility of optimization (1) bycomputing either a nonzero solution of the homogeneous self-dual embedding [10], or the minimal-displacement vector[9, 11, 3]. Compared with traditional infeasibility detectionmethods [12, 13], DRS has empirically achieved up to three-orders-of-magnitude speedups in computation for relativelylarge-scale problems [10].
A challenge of DRS-based infeasibility detection is com-puting matrix inverse [10, 9, 11, 3]. Such computationbecomes expensive as the size of optimization (1) increases,or when optimization (1) is solved repeatedly with differentconstraint parameters in real-time [14, 15]. While suchmatrix inverse is not necessary for problems with only linearconstraints [16, 17], whether it is necessary in general foroptimization (1) is, to our best knowledge, still an openquestion.
We propose a novel infeasibility detection method for opti-mization (1), named proportional-integral projected gradientmethod (PIPG). The iterates of PIPG either asymptoticallysatisfy a set of primal-dual optimality conditions or diverge.In the latter case, the difference between the consecutiveiterates converges to a nonzero vector, known as the minimaldisplacement vector, that proves primal or dual infeasibility.
PIPG is the first infeasibility detection method for op-timization (1) that avoids computing matrix inverse. Allexisting methods compute matrix inverse as a subroutine ofeither interior point methods [12, 13] or DRS [9, 10, 11, 3].In contrast, PIPG only computes matrix multiplication andprojections onto the set D and the cone K in optimization (1).As a result, PIPG allows implementation for large-scale andreal-time problems using limited computational resources.
PIPG provides an efficient method for quasiconvex andmixed-integer optimization problems. We demonstrate theapplication of PIPG in these problems using two examplesin constrained optimal control: minimum-time landing andnonconvex path-planning of a quadrotor.
The rest of the paper is organized as follows. After somebasic results in convex analysis and monotone operatortheory, Section II reviews the existing results on DRS-basedinfeasibility detection. Section III introduces PIPG alongwith its convergence properties. Section IV demonstrates theapplication of PIPG in constrained optimal control problems.Finally, Section V concludes and comments on future workdirections.
arX
iv:2
109.
0275
6v2
[m
ath.
OC
] 1
1 Se
p 20
21
II. PRELIMINARIES AND RELATED WORK
We will review some basic results in convex analysisand monotone operator theory, and some existing resultson infeasibilty detection using Douglas-Rachford splittingmethod.
A. Notations and preliminaries
We let N, R and R` denote the set of positive integer, realnumbers, and non-negative real numbers, respectively. For avector z P Rn and integers i, j P N (1 ď i ă j ď n), we letrzsj denote the j-th element of vector z, and rzsi:j P Rj´i`1
denote the vector composed of the entries of z between (andincluding) the i-th entry and the j-th entry. For two vectorsz, z1 P Rn, we let xz, z1y denote their inner product, and‖z‖ :“
a
xz, zy denote the `2 norm of z. We let In denote thenˆn identity matrix and 0mˆn denote the mˆn zero matrix.When the dimensions of these matrices are clear from thecontext, we omit their subscripts. For a matrix H P Rmˆn,HJ denotes its transpose, |||H||| denotes its largest singularvalue. For a square matrix A P Rnˆn, exppAq denotes thematrix exponential of A. For a set S, we let ´S :“ tz|´ z PSu. Given two sets S1 and S2, we let S1 ˆ S2 denote theirCartesian product. We say function f : Rn Ñ RY t`8u isconvex if fpαz` p1´αqz1q ď αfpzq ` p1´αqfpz1q for allα P r0, 1s. We say set D Ď Rn is convex if αz`p1´αqz1 P Dfor any α P r0, 1s and z, z1 P D. We say set K Ď Rm is aconvex cone if K is a convex set and αw P K for any w P Kand α P R`.
Let f : Rn Ñ R Y t`8u be a convex function. Theproximal operator of function f is a function given by
proxf pzq :“ argminz1
fpz1q `1
2
∥∥z1 ´ z∥∥2(2)
for all z P Rn. Let D Ď Rn be a nonempty closed convexset. The projection of z P Rn onto set D is given by
πDrzs :“ argminz1PD
∥∥z ´ z1∥∥ . (3)
The point-to-set distance from z to set D is given by
dpz|Dq :“ minz1PD
∥∥z ´ z1∥∥ . (4)
The normal cone of set D at z is given by
NDpzq :“ ty P Rn|xy, z1 ´ zy ď 0,@z1 P Du. (5)
The recession cone of set D is a nonempty closed convexcone given by
recD :“ ty P Rn|y ` z P D,@z P Du. (6)
The indicator function of set D is defined as
δDpzq :“
#
0, if z P D,`8, otherwise.
(7)
for all z P Rn. The support function of set D is given by
σDpzq :“ supyPD
xy, zy (8)
for all z P Rn. Let K Ď Rm be a nonempty closed convexcone. The recession cone of cone K is itself, i.e., recK “ K[18, Cor. 6.50]. The polar cone of K is a closed convex conegiven by
K˝ :“ tw P Rm|xw, yy ď 0,@y P Ku. (9)
One can verify that pK˝q˝ “ K [19, Cor. 6.21]. We will usethe following results on polar cone.
Lemma 1. [18, Thm. 6.30] If K Ă Rm is a nonemptyclosed convex cone, then w “ πKrws ` πK˝rws andxπKrws, πK˝rwsy “ 0 for all w P Rm.
We say function T : Rp Ñ Rp is a γ-averaged operatorfor some γ P p0, 1q if and only if the following conditionholds for all ξ1, ξ2 P Rp [18, Prop. 4.35]:
‖T pξ1q ´ T pξ2q‖2´ ‖ξ1 ´ ξ2‖2
ďγ ´ 1
γ‖ξ1 ´ T pξ1q ´ ξ2 ` T pξ2q‖2
.(10)
We will use the following result on averaged operators.
Lemma 2. [3, Lem. 5.1] Let T : Rp Ñ Rp be a γ-averagedoperator for some γ P p0, 1q. Let ξ1 P Rp and ξj`1 :“T pξjq for all j P N. Then there exists ξ P Rp, known as theminimal-displacement vector of operator T , such that
limjÑ8
ξj
j“ limjÑ8
ξj`1 ´ ξj “ ξ.
B. Infeasibility detection via Douglas-Rachford splittingmethod
Douglas-Rachford splitting method (DRS) is an iterativemethod for optimization problems of the following form:
minimizeξ
`pξq ` rpξq, (11)
where ` : Rp Ñ R Y t`8u and r : Rp Ñ R Y t`8u areconvex functions. DRS uses the following iteration, wherej P N is the iteration counter and α P p0, 2q is the step size:
ξj`1 “ ξj `αpprox`p2proxrpξjq ´ ξjq ´ proxrpξ
jqq. (12)
One can show that equation (12) is equivalent to ξj`1 “
T pξjq for some α2 -averaged operator T [20].
In the following, we will review two different ways ofinfeasibilty detection using DRS. For simplicity, we assumethat DRS is terminated when a maximum number of iter-ations, denoted by k is reached. We also let ε P R` besuch that numbers within the interval r´ε, εs are treated asapproximately zero.
1) Homogeneous self-dual embedding method: The firstway of infeasibility detection via DRS considers the follow-ing special case of optimization (1):
minimizez
qJz
subject to Hz ´ g P K.(13)
The homogeneous self-dual embedding of optimization(13) is given by the following set of linear equations andconic inclusions [12, 13, 10]:
v “ Su, u P K, v P ´K˝, (14)
where
S “
»
–
0 HJ q´H 0 g´qJ ´gJ 0
fi
fl , K “ Rn ˆ p´K˝q ˆ R`. (15)
Here ´K˝ is also known as the dual cone of K. Noticethat the conditions in (14) are satisfied if and only if ξ ““
uJ vJ‰J
is an optimal solution of optimization (11) with
`pu, vq “δKˆp´K˝qpu, vq, rpu, vq “ δv“Supu, vq, (16)
where δv“Su is the indicator function of set tpu, vq|v “ Suu.If u, v P Rm`n`1 satisfy (14) without both being zero
vectors, then one can detect the primal and dual infeasibilityof optimization 13 as follows. Let
z “ rus1:n, w “ rusn`1:n`m,
τ “ rusm`n`1, κ “ rvsm`n`1.
If τ ą 0 and κ “ 0, then 1τ z satisfy the optimality conditions
of optimization (13). If τ “ 0 and xg, wy ă 0, then one canconstruct a proof of primal infeasibility. If τ “ 0 and xq, zy ă0, then one can construct a proof of dual infeasibility. See[10, Sec. 2.3] for a detailed discussion.
Algorithm 1 summarizes the above infeasibility detectionmethod, where the optimization (11) defined by (16) issolved using DRS. In this case, the DRS iteration in (12)is equivalent to line 2-4 of Algorithm 1 [10, Sec. 3.2].Further, with a proper choice of initial values of u1, v1 anda sufficiently large iteration number k, the vectors uk and vk
approximately satisfy (14) without being both zero vectors[10, Sec. 3.4]. Notice that line 2 computes the inverse of asymmetric matrix in Rpm`n`1qˆpm`n`1q.
Algorithm 1 DRS for optimization (13) via homogeneousself-dual embedding
Input: k, ε, initial values u1, v1.Output: z‹, “Primal infeasible”,“Dual Infeasible”
1: for j “ 1, 2, . . . , k ´ 1 do2: uj`1 “ pI ` Sq´1puj ` vjq3: uj`1 “ πKru
j`1 ´ vjs4: vj`1 “ vj ´ uj`1 ` uj`1
5: end for6: if ruksm`n`1 ą ε, rvksm`n`1 ď ε then7: return z‹ “ 1
ruksm`n`1ruks1:n.
8: else if ruksm`n`1 ď ε then9: if xg, ruksn`1:n`my ă ´ε then
10: return “Primal infeasible”11: end if12: if xq, ruks1:ny ă ´ε then13: return “Dual infeasible”14: end if15: end if
2) Minimal-displacement vector method: Another way todetects the primal and dual infeasibility of optimization (1)
via DRS is to compute the minimal-displacement vector ofDRS. In particular, let
H ““
HJ I‰J, D “ tg ` y|y P Ku ˆ D.
Then one can rewrite optimization (1) by as an instance ofoptimization (11) with ξ “
“
zJ yJ‰J
and
`pz, yq “1
2zJPz ` qJz ` δHz“ypz, yq, rpz, yq “ δDpyq,
where δHz“ypz, yq is the indicator function of settz, y|Hz “ yu. In addition, when DRS is applied to theabove instance of optimization (11), the difference betweenthe consecutive iterates converges to a limit, known as theminimal-displacement vector of DRS. If this limit is zero, theiterates of DRS asymptotically satisfies a set of primal-dualoptimality conditions. If this limit is nonzero, the iterates ofDRS asymptotically provide a proof for either primal or dualinfeasibility [3, 21].
Algorithm 2 summarizes the above infeasibility detectionmethod, where the DRS iteration in (12) simplifies to line 2-6 [21, Sec. 4]. With a sufficient large iteration number k,the vectors wk ´ wk´1 and vk ´ vk´1 together give anapproximation of the aforementioned minimal-displacementvector. Notice that line 4 computes the inverse of a symmetricmatrix in Rnˆn.
Algorithm 2 DRS for optimization (1) via minimal-displacement vector
Input: k, α, ε initial values z1, y1.Output: z‹, “Primal infeasible”, “Dual infeasible”.
1: for j “ 1, 2, . . . , k ´ 1 do2: yj “ πDry
js
3: wj “ yj ´ yj
4: zj “ pI ` P `HJHq´1pzj ´ q `H
Jp2yj ´ yjqq
5: zj`1 “ zj ` αpzj ´ zjq6: yj`1 “ yj ` αpHzj ´ yjq7: end for8: if maxt
∥∥wk ´ wk´1∥∥ ,∥∥zk ´ zk´1
∥∥u ď ε then9: return z‹ “ zk
10: else11: if
∥∥wk ´ wk´1∥∥ ą ε then
12: return “Primal infeasible”13: end if14: if
∥∥zk ´ zk´1∥∥ ą ε then
15: return “Dual infeasible”16: end if17: end if
III. PROPORTIONAL-INTEGRAL PROJECTED GRADIENTMETHOD
We now introduce proportional-integral projected gradientmethod (PIPG), a primal-dual infeasibility detection methodfor optimization (1). Provided that an optimal primal-dualsolution exists, PIPG ensures that both the primal-dual gapand the constraint violation converge to zero along certain
sequences of averaged iterates [22, 23]. In the following, wewill show that the iterates of PIPG can also provide proofof primal or dual infeasibility.
Algorithm 3 summarizes PIPG for infeasibility detection,where k P N is the maximum number of iteration, α ą 0is the step size, and ε P R` is chosen such that numbersin interval r´ε, εs are treated as approximately zero. UnlikeAlgorithm 1 and Algorithm 2, Algorithm 3 does not computematrix inverse. Furthermore, using Lemma 1, one can showthat the projections in Algorithm 3 are no more difficult tocompute than those in Algorithm 2.
Algorithm 3 PIPG for optimization (1)
Input: k, α, ε, initial values z1, v1.Output: z‹, “Primal infeasible”, “Dual infeasible”.
1: for j “ 1, 2, . . . , k ´ 1 do2: wj`1 “ πK˝rv
j ` αpHzj ´ gqs3: zj`1 “ πDrz
j ´ αpPzj ` q `HJwj`1qs
4: vj`1 “ wj`1 ` αHpzj`1 ´ zjq5: end for6: if maxt
∥∥wk ´ wk´1∥∥ ,∥∥zk ´ zk´1
∥∥u ď ε then7: return z‹ “ zk
8: else9: if
∥∥wk ´ wk´1∥∥ ą ε then
10: return “Primal infeasible”11: end if12: if
∥∥zk ´ zk´1∥∥ ą ε then
13: return “Dual infeasible”14: end if15: end if
The name proportional-integral projected gradient method(PIPG) is due to the fact that line 2-4 of Algorithm 3 combineprojected gradient method together with proportional-integralfeedback of constraints violation [23]. Similar ideas werefirst introduced in distributed optimization [24, 25, 26] andlater extended to general conic optimization [22, 23].
In the following, we will show that line 2-4 in Algorithm 3is the fixed-point iteration of an averaged operator. Further,the minimal displacement vector (see Lemma 2) of saidaveraged operator yield information regarding optimalityand infeasibility. We will use the following assumptions onoptimization (1) and the step size α in Algorithm 3.
Assumption 1. 1) Matrix P P Rnˆn is symmetric andpositive semidefinite.
2) Set D Ď Rn and cone K Ď Rm are both nonempty,closed and convex.
Assumption 2. The step size α in Algorithm 3 satisfies
0 ă α ď8´ 4
γ?λ2 ` 16ν2 ` λ
for some γ P p 12 , 1q, λ ě |||P |||, and ν ě |||H|||.
As our first step, the following proposition shows that, un-der Assumption 1 and Assumption 2, line 2-4 in Algorithm 3is the fixed-point iteration of an averaged operator.
Proposition 1. Suppose that Assumption 1 and Assumption 2hold. Then line 2-4 in Algorithm 3 implies that
„
zj`1
vj`1
“ T
ˆ„
zj
vj
˙
, (17)
where T : Rm`n Ñ Rm`n is a γ-averaged operator, whereγ P p 1
2 , 1q is given in Assumption 2.
Proof. See Appendix I.
Remark 1. In order to satisfy Assumption 2, one needs toestimate the values of |||H||| and |||P |||. The following poweriteration ensures that
∥∥zj∥∥ quickly converges to |||H|||2 asthe number of iteration j increases:
zj`1 “1
‖zj‖HJHzj ,
where the entries in vector z1 P R are randomly generated.Notice that the above power iteration does not computematrix inverse or decomposition; see [27] for further details.The computation of |||P ||| is similar.
Due to Proposition 1 and Lemma 2, one can show that both∥∥zk ´ zk´1∥∥ and
∥∥wk ´ wk´1∥∥ computed in Algorithm 3
converge to certain limits as k increases. To show that theselimits yield information regarding optimality and infeasibil-ity, we first introduce the following results.
Lemma 3. Suppose that Assumption 1 holds and the se-quence twj , zj , vjujPN is computed recursively using line 2-4 in Algorithm 3 where α ą 0. Let λ ě |||P ||| and ν ě |||H|||.Then, for all j ě 3,
dp´Pzj ´ q ´HJwj |NDpzjqq ď
ˆ
1
α` λ
˙∥∥zj ´ zj´1∥∥ ,
dpHzj ´ g|NK˝pwjqq
ď1
α
∥∥wj ´ wj´1∥∥` ν ∥∥zj ´ 2zj´1 ` zj´2
∥∥ .(18)
Proof. See Appendix II.
Lemma 4. Suppose that Assumption 1 and Assumption 2hold and the sequence twj , zj , vjujPN is computed recur-sively using line 2-4 in Algorithm 3. Then there existsz P recD and w P K˝ such that lim
jÑ8zj ´ zj´1 “ z and
limjÑ8
wj ´ wj´1 “ w. Further,
Hz P K, Pz “ 0, xq, zy “ ´1
α‖z‖2
,
σDp´HJwq ` xg, wy “ ´
1
α‖w‖2
.(19)
Proof. See Appendix III.
Equipped with Lemma 3 and Lemma 4, we summarizeour main theoretical contribution in the following theorem.
Theorem 1. Suppose that Assumption 1 and Assumption 2hold and the sequence twj , zj , vjujPN is computed recur-sively using line 2-4 in Algorithm 3. Then there exist z P
recD and w P K such that
limjÑ8
zj ´ zj´1 “ z, limjÑ8
wj ´ wj´1 “ w. (20)
Further, the following three statements hold.1) If ‖z‖ “ ‖w‖ “ 0, then
limjÑ8
dp´Pzj ´ q ´HJwj |NDpzjqq “ 0,
limjÑ8
dpHzj ´ g|NK˝pwjqq “ 0.
(21)
2) If ‖w‖ ą 0, then
infzPDxHz ´ g, wy ą 0 “ sup
yPKxy, wy. (22)
3) If ‖z‖ ą 0, then
Hz P K, Pz “ 0, xq, zy ă 0. (23)
Proof. Lemma 2 implies that the limits in (20) hold forsome z P recD and w P K˝. Further, the first statementis due to Lemma 3 and (20). The second and the thirdstatement are due to (19) and the fact that σDp´HJwq “´ infzPD xHz,wy and supyPK xy, wy “ 0 when w P K˝. Thelatter fact is due to the definition of polar cone in (9).
We now discuss the implications of the three statementsin Theorem 1. First, the two limits in (21) imply that a setof primal-dual optimality conditions are satisfied asymptot-ically. To see this implication, let z‹ P D and w‹ P K˝ besuch that
dp´Pz‹´q´HJw‹|NDpz‹qq “ dpHz‹´g|NK˝pw
‹qq “ 0.
Since the point-to-set distance is nonnegative, the above con-ditions are equivalent to the following primal-dual optimalityconditions of optimization (1) [19, Ex.11.52]:
´Pz‹ ´ q ´HJw‹ PNDpz‹q, (24a)
Hz‹ ´ g PNK˝pw‹q. (24b)
Provided that the constraints in optimization (1) satisfy cer-tain qualification, the conditions in (24) imply that pz‹, w‹qis an optimal primal-dual solution of optimization (1) [19,Cor. 11.51]. We note that even if the conditions in (21) hold,optimization (1) can still be infeasible, i.e., its constraintscannot be satisfied. In this case, however, an infinitesimalperturbation of vector g will make optimization (1) eitherfeasible or infeasible. See [3, Sec. 6.3] for an example.
Second, the strict inequality in (22) implies that xw, yy “0 is a hyperplane that separates cone K from the settHz ´ g|z P Du. See Fig. 1 for an illustration. As a result,optimization (1) is infeasible.
Third, if there exists z P D such that Hz ´ g P K,then the conditions in (23) imply that the optimal value ofoptimization (1) is unbounded. In particular, let y “ z ` z,then (23) and the fact that z P recD and Hz P K imply thefollowing:
y P D, Hy ´ g P K,1
2yJPy ` qJy ă
1
2zJPz ` qJz.
KtHz ´ g|z P Du
xw, yy “ 0
Fig. 1: An illustration of the seperating hyperplane.
Hence the value of the objective function in optimization(1) strictly decreases along direction z without violating theconstraints. By repeating a similar process we can show thethe optimal value of optimization (1) is indeed unboundedfrom below. Furthermore, the dual problem of optimization(1) is infeasible [3, 21].
IV. APPLICATIONS IN CONSTRAINED OPTIMAL CONTROL
We demonstrate the application of PIPG in constrainedoptimal control problems. In particular, we will show howto use PIPG to compute minimum-time landing trajectory inSection IV-A, and reduce the number of binary variables innonconvex path-planning problem in Section IV-B.
Fig. 2: A custom-made quadrotor designed by the Au-tonomous Control Laboratory.
We consider the optimal control of the custom-madequadrotor designed by the Autonomous Control Laboratory(ACL) at University of Washington, which we referred to asthe ACL quadrotor. See Fig. 2 for an illustration and [28] fora detailed description of the ACL quadrotor. For simplicity,we let all problem parameters in the following (e.g., mass ofthe quadrotor, maximum thrust magnitude) be unitless.
The dynamics of an ACL quadrotor is given by
d
dsxpsq “ Acxpsq `Bcupsq ` hc (25)
for all time s P R`, where xpsq P R6 denotes the positionand velocity of the center of mass of the quadrotor at times, upsq P R3 denotes the thrust vector provided by thepropellers of the quadrotor at time s. In addition,
Ac “
„
03ˆ3 I303ˆ3 03ˆ3
, Bc “1
0.35
„
03ˆ3
I3
, hc “
„
05ˆ1
´9.8
,
where 0.35 is the total mass of the quadrotor, 9.8 is thegravity constant.
We can further simplify dynamics (25) by assuming thethrust vector does not change value within each samplingtime period of length ∆ P R`. In particular, suppose that
upsq “ upt∆q, t∆ ď s ă pt` 1q∆ (26)
ρ12θ1ρ2
Fig. 3: An illustration of the set of feasible thrust vectors.
for all t P N. Let xt :“ xpt∆q and ut :“ upt∆q for all t P N.Then dynamics equation (25) is equivalent to
xt`1 “ Axt `But ` h (27)
for all t P N, where
A “ exppAc∆q, B “´
ş∆
0exppAcsqds
¯
Bc,
h “´
ş∆
0exppAcsqds
¯
hc.(28)
Due to the physical limits of its onboard motors, we considerthe following constraints on the thrust vector of the quadro-tor. The magnitude of the thrust vector is lower bounded byρ1 “ 2 and upper bounded by ρ2 “ 5. Further, the directionof the thrust vector is confined within an icecream cone withhalf-angle angle θ “ π{4. These constraints can be writtenas ut P U for all t P N, where
U :“ tu P R3| ‖u‖ cos θ ď rus3, rus3 ě ρ1, ‖u‖ ď ρ2u.(29)
Notice that we approximate the lower bound on ‖u‖ usinga linear inequality constraint rus3 ě 2 so that set U remainsconvex. See Fig. 3 for an illustration of set U. We also notethat the state of system (27) can be transferred from anyinitial value to any final value using a finite-length sequenceof inputs, where each input is chosen from set U.
In the following, we will consider optimal control prob-lems with dynamics constraints (27) (where we fix ∆ “ 0.2)and input constraint set (29). We will solve these optimalcontrol problems using Algorithm 2 (where α “ 1) andAlgorithm 3 (where α satisfies Assumption 2 with γ “
0.9) and compare their performance. We will also use thefollowing notation:
ur0,τ´1s :““
uJ0 uJ1 . . . uJτ´1
‰J,
xr1,τ´1s :““
xJ1 xJ2 . . . xJτ´1
‰J.
(30)
A. Minimum-time landing via quasiconvex optimization
We consider the problem of landing a quadrotor on acharging station in the minimum amount of time possiblesubject to various state and input constraints. Let x0 P R6
denote the initial state of the quadrotor, and suppose that thecharging station is located at the origin of the position coordi-nate system. Then the minimum landing time is given by i∆,where ∆ is the sampling time in (26), and i is the smallestinteger that makes the following optimization feasible (i.e.,
there exists one solution that satisfies its constraints):
minimizeur0,τ´1s,xr1,τ´1s
12
řτ´1t“0 ‖ut‖2
subject to xt`1 “ Axt `But ` h, 0 ď t ď τ ´ 1,ut P U, 0 ď t ď τ ´ 1,xt P X, 1 ď t ď τ ´ 1,xt “ 0, i ď t ď τ ´ 1,
(31)where τ P N is large enough such that optimization (31) isfeasible if i “ τ ´ 1, set U is given by (29), and set X isgiven by
X “ tr P R3| ‖r‖ cosβ ď rrs3u ˆ tr P R3| ‖r‖ ď ηu,
where β “ π{4, η “ 5. The quadratic objective function in(31) penalizes large-magnitude inputs. The constraint xt P Xupper bounds the speed of the quadrotor and limits thedirections from which the quadrotor approach the chargingstation. The latter is also known as the approach coneconstraint, which is widely used in spacecraft landing [15].See Fig. 4 for an illustration.
approach
cone
charging station
Fig. 4: An illustration of the quadrotor landing.
One can compute the smallest integer i that makes opti-mization (31) feasible by solving a quasiconvex optimizationproblem. To this end, we say pur0,τ´1s, xr1,τ´1sq is feasiblefor (31) if pur0,τ´1s, xr1,τ´1sq satisfy the constraints inoptimization (31). We also define function f using (32) (seethe bottom of the next page). Then the smallest integer i thatmakes optimization (31) feasible is the optimal value of thefollowing optimization:
minimizeur0,τ´1s,xr1,τ´1s
fpur0,τ´1s, xr1,τ´1sq. (33)
We will show that optimization (33) is a quasiconvexoptimization problem and can be solved using bisectionmethod and infeasibility detection as follows. First, givenany ε P R`, one can verify that
tpur0,τ´1s, xr1,τ´1sq|fpur0,τ´1s, xr1,τ´1sq ď εu
“
#
pur0,τ´1s, xr1,τ´1sq
ˇ
ˇ
ˇ
ˇ
ˇ
pur0,τ´1s, xr1,τ´1sq isfeasible for (31) with i “ tεu
+
,
(34)where tεu is the largest integer less or equals to ε. Since theconstraints in (31) are all convex with respect to ur0,τ´1s andxr1,τ´1s, we conclude that set (34) is convex. In other words,all sublevel sets of function f are convex sets. Therefore,function f is quasiconvex and one can solve optimization
(33) by checking whether the set in (34) is empty, i.e.,whether optimization (31) is feasible when i “ tεu, fordifferent choices of ε P R` [2, Sec. 4.2.5].
In the following, we consider optimization (33) where τ “40, i “ t24, 26u, and x0 “
“
6 6 15 2 2 2‰J
.To solve the above optimization, we apply Algorithm 2
and Algorithm 3 to the corresponding optimization (31).In particular, Appendix IV gives the transformation fromoptimization (31) to optimization (1). Fig. 5 illustrates theconvergence of
∥∥wj ´ wj´1∥∥ and
∥∥zj ´ zj´1∥∥ computed
in Algorithm 2 and Algorithm 3, where the two algorithmsshow similar convergence. These convergence results showthat both
∥∥wj ´ wj´1∥∥ and
∥∥zj ´ zj´1∥∥ converge to zero
if i “ 26, and∥∥wj ´ wj´1
∥∥ does not converge to zero ifi “ 24. As a result of Theorem 1, we conclude that theoptimal value of optimization (33) is between 24 and 26(solving another instance of optimization (31) will confirmthat it is 25).
(a) i “ 24, Algorithm 2 (b) i “ 24, Algorithm 3
(c) i “ 26, Algorithm 2 (d) i “ 26, Algorithm 3
Fig. 5: Convergence of∥∥wj ´ wj´1
∥∥ and∥∥zj ´ zj´1
∥∥ inAlgorithm 2 and Algorithm 3 when applied to optimization(31) with different values of i.
We note that the quasiconvex optimization approach pre-sented in this section has also been used for the minimumtime control of overhead cranes [29].
B. Path-planning via mixed-integer optimizationWe consider the problem of flying a quadrotor from a
initial position to a final position, both located in a L-shaped
corridor. See Fig. 6 for an illustration. To avoid collisionwith the boundary of the corridor, we consider the followingstate constraints
r1 ď Txt ď r1 or r2 ď Txt ď r2, (35)
for all t, where T ““
I3 03ˆ3
‰
,
r1 ““
0 ´2 0‰J, r1 “
“
2 9 3‰J,
r2 ““
2 ´2 0‰J, r1 “
“
12 0 3‰J.
Given the initial and final state of the quadrotor (denoted byx0 P R6 and xτ P R6, respectively), the above problem canbe formulated as follows:
minimizeur0,τ´1s,xr1,τ´1s
br1,τ´1s
12
řτ´1t“0 ‖ut‖2
subject to xt`1 “ Axt `But ` h, 0 ď t ď τ ´ 1,ut P U, 0 ď t ď τ ´ 1,xt P X, 1 ď t ď τ ´ 1,r1 ` btpr
2 ´ r1q ď Txt, 1 ď t ď τ ´ 1,r1 ` btpr
2 ´ r1q ě Txt, 1 ď t ď τ ´ 1,bt P t0, 1u, 1 ď t ď τ ´ 1,
(36)where the notation br1,τ´1s is similar to those in (30), Uis given by (29), and X “ R3 ˆ tr P R3| ‖r‖ ď ηu withη “ 5. The quadratic objective function in (36) penalizeslarge-magnitude inputs. The constraint xt P X upper boundsthe speed of the quadrotor. We assume τ is large enoughsuch that optimization (36) has an optimal solution.
corridor1
corridor 2
x0
xτ
Fig. 6: An illustration of path-plannning in an L-shapedcorridor.
Optimization (36) is challenging to solve if the numberof binary variables is large. For example, if τ “ 22, thenoptimization (36) contains 21 binary variables, and solvingit naively requires considering 221 (more than 2 millions)different values of vector br0,τ´1s, each value corresponds toa convex optimization problem.
One can reduce the number of binary variables in op-timization (36) using infeasibility detection as follows.Let b P t0, 1u and pu‹
r0,τ´1s, x‹r1,τ´1s, b
‹r1,τ´1sq be an
optimal solution of optimization (36). If b‹1 “ b, then
fpur0,τ´1s, xr1,τ´1sq :“ min
i P NY t8uˇ
ˇpur0,τ´1s, xr1,τ´1sq is feasible for (31)(
, where minH :“ 8. (32)
pu‹r0,τ´1s, x
‹r1,τ´1s, b
‹r1,τ´1sq satisfies the constraints of the
following optimization:
minimizeur0,τ´1s,xr1,τ´1s
br1,τ´1s
12
řτt“0 ‖ut‖
2
subject to xt`1 “ Axt `But ` h, 0 ď t ď τ ´ 1,ut P U, 0 ď t ď τ ´ 1,xt P X, 1 ď t ď τ ´ 1,r1 ` btpr
2 ´ r1q ď Txt, 1 ď t ď τ ´ 1,r1 ` btpr
2 ´ r1q ě Txt, 1 ď t ď τ ´ 1,
b1 “ b, bt P r0, 1s, 1 ď t ď τ ´ 1.(37)
Compared with (36), here the constraints on br1,τ´1s arecontinuous instead of discrete, and the value of b1 is fixedinstead of being a variable.
If optimization (37) is infeasible (i.e., no solution satisfiesits constraints), then pu‹
r0,τ´1s, x‹r1,τ´1s, b
‹r1,τ´1sq does not
satisfy the constraints in (37). Consequently, one must haveb‹1 ‰ b. In other words, we can eliminate the binary variableb1 in optimization (36) by fixing its value to be 1´ b.
In the following, we consider optimization (36) where τ “22 and
x0 ““
1 9 2.5 0 0 0‰J,
xτ ““
12 ´1 0.5 0 0 0‰J.
.To eliminate binary variable b1 in the above optimiza-
tion, we apply Algorithm 2 and Algorithm 3 to the corre-sponding optimization (37). Particularly, Appendix IV givesthe transformation from optimization (37) to optimization(1). Fig. 7 illustrates the convergence of
∥∥wj ´ wj´1∥∥
and∥∥zj ´ zj´1
∥∥ computed in Algorithm 2 and Algo-rithm 3, where the two algorithms show similar convergence.These convergence results show that both
∥∥wj ´ wj´1∥∥ and∥∥zj ´ zj´1
∥∥ converge to zero if b “ 0, and∥∥wj ´ wj´1
∥∥does not converge to zero if b “ 1. As a result of Theorem 1,we conclude that optimization (37) is infeasible if b “ 1 andfeasible if b “ 0. Therefore we can eliminate the binaryvariable b1 in optimization (36) by fixing its value to be 0.
By repeating a similar process, we can further reduce thenumber of binary variables, and conclude that bi “ 0 forall i “ 1, 2, . . . , 7 and bi “ 1 for all i “ 12, 13, . . . , 21in optimization (36). In other words, by solving at most 42convex optimization problems, each similar to optimization(37), we can reduce the number of possible values of binaryvariables br1,21s from 221 (more than 2 millions) to merely24 (less than 20)!
V. CONCLUSIONS
We introduce a novel method, named proportional-integralprojected gradient method (PIPG), for infeasibility detectionin conic optimization. The iterates of PIPG either asymp-totically provide a proof of primal or dual infeasibility,or asymptotically satisfy a set of primal-dual optimalityconditions. We demonstrate the application of PIPG in qua-siconvex and mixed integer optimization.
There are several questions about PIPG that still remainopen. For example, does PIPG allow larger step sizes like
(a) b “ 0, Algorithm 2 (b) b “ 0, Algorithm 3
(c) b “ 1, Algorithm 2 (d) b “ 1, Algorithm 3
Fig. 7: Convergence of∥∥wj ´ wj´1
∥∥ and∥∥zj ´ zj´1
∥∥ inAlgorithm 2 and Algorithm 3 when applied to optimization(37) for different values of b.
the primal-dual method in [30]? If the projections in PIPGare computed approximatedly rather than exactly, how willthe results in Theorem 1 change? We aim to answer thesequestions in our future work.
APPENDIX IPROOF OF PROPOSITION 1
We will use the following result on projection.
Lemma 5. [18, Thm. 3.16] Let D Ă Rn be a nonemptyclosed convex set. Then y “ πDrzs if and only if y P D andxy ´ z, z1 ´ yy ě 0 for any z1 P D.
Proof. Let zj P D, vj P Rm and
w`j :“πK˝rvj ` αpHzj ´ gqs, (38a)
z`j :“πDrzj ´ αpPzj ` q `HJw`j qs, (38b)
v`j :“w`j ` αHpz`j ´ zjq, (38c)
for j “ 1, 2. Inorder to show that line 2-4 in Algorithm 3 isthe fixed point iteration of a γ-averaged operator, it sufficesto show the following inequality∥∥z`2 ´ z`1 ∥∥2
`1´γγ
∥∥z1 ´ z`1 ´ z2 ` z
`2
∥∥2
`∥∥v`2 ´ v`1 ∥∥2
`1´γγ
∥∥v1 ´ v`1 ´ v2 ` v
`2
∥∥2
ď ‖z2 ´ z1‖2` ‖v2 ´ v1‖2
(39)
To this end, first we apply Lemma 5 to (38a) and (38b) forboth j “ 1 and j “ 2 and obtain the following inequalities:
0 ďxw`1 ´ v1 ´ αpHz1 ´ gq, w`2 ´ w
`1 y, (40a)
0 ďxw`2 ´ v2 ´ αpHz2 ´ gq, w`1 ´ w
`2 y, (40b)
0 ďxz`1 ´ z1 ` αpPz1 ` q `HJw`1 q, z
`2 ´ z
`1 y, (40c)
0 ďxz`2 ´ z2 ` αpPz2 ` q `HJw`2 q, z
`1 ´ z
`2 y. (40d)
Summing up the right hand side of (40a), (40b), (40c), and(40d) gives the following
0 ď xz`1 ´ z`2 , z1 ´ z
`1 ´ z2 ` z
`2 y
` αxP pz2 ´ z1q, z`1 ´ z
`2 y
` xw`1 ´ w`2 , w
`2 ´ v2 ´ w
`1 ` v
1y
` αxw`1 ´ w`2 , Hpz1 ´ z2 ´ z
`1 ` z
`2 qy.
(41)
Second, by using (38c) for j “ 1 and j “ 2 we can showthe following two identities
αxw`1 ´ w`2 , Hpz1 ´ z2 ´ z
`1 ` z
`2 qy
“ xw`1 ´ w`2 , w
`1 ´ v
`1 ` v
`2 ´ w
`2 y,
(42)
xw`1 ´ w`2 , v1 ´ v
`1 ´ v2 ` v
`2 y
“ xv`1 ´ v`2 , v1 ´ v
`1 ´ v2 ` v
`2 y
` αxHpz1 ´ z`1 ´ z2 ` z
`2 q, v1 ´ v
`1 ´ v2 ` v
`2 y.
(43)
By summing up both sides of (41), (42), and (43) we obtainthe following
0 ď xz`1 ´ z`2 , z1 ´ z
`1 ´ z2 ` z
`2 y
` xv`1 ´ v`2 , v1 ´ v
`1 ´ v2 ` v
`2 y
` αxP pz2 ´ z1q, z`1 ´ z
`2 y
` αxHpz1 ´ z`1 ´ z2 ` z
`2 q, v1 ´ v
`1 ´ v2 ` v
`2 y.
(44)
Our next step is to further simplify the inner product termsin (44) as follows. First, by completing the square, we canshow the following inequality
2xHpz1 ´ z`1 ´ z2 ` z
`2 q, v1 ´ v
`1 ´ v2 ` v
`2 y
ďγα
2γ´1
∥∥Hpz1 ´ z`1 ´ z2 ` z
`2 q
∥∥2
`2γ´1γα
∥∥v1 ´ v`1 ´ v2 ` v
`2
∥∥2.
(45)
Second, by completing the square we can also show thefollowing identities∥∥z1 ´ z
`1 ´ z2 ` z
`2
∥∥2`∥∥z`1 ´ z`2 ∥∥2
´ ‖z1 ´ z2‖2
“ ´2xz`1 ´ z`2 , z1 ´ z
`1 ´ z2 ` z
`2 y,
(46)
∥∥v1 ´ v`1 ´ v2 ` v
`2
∥∥2`∥∥v`1 ´ v`2 ∥∥2
´ ‖v1 ´ v2‖2
“ ´2xv`1 ´ v`2 , v1 ´ v
`1 ´ v2 ` v
`2 y.
(47)
Third, since ν ě |||H|||, we have∥∥Hpz1 ´ z`1 ´ z2 ` z
`2 q
∥∥2ď ν2
∥∥z1 ´ z`1 ´ z2 ` z
`2
∥∥2.
(48)If λ “ 0 ě |||P |||, then we necessarily have P “ 0. In thiscase, by multiplying both sides of (44) by 2, both sides of(45) by α, both sides of (48) by γα2
2γ´1 , then summing themup together with the both sides of (46), (47), we obtain (39)if γα2ν2
2γ´1 ď2γ´1γ .
If λ ą 0, we need the following two additional inequali-ties. First, since λ ě |||P ||| one can verify the following
´ xP pz1 ´ z2q, z1 ´ z2y ď ´1λ ‖P pz1 ´ z2q‖2
. (49)
Second, by completing the square we can show the following
2xP pz2 ´ z1q, z`1 ´ z1 ´ z
`2 ` z2y
ď 2λ ‖P pz2 ´ z1q‖2
` λ2
∥∥z`1 ´ z1 ´ z`2 ` z2
∥∥2.
(50)
Finally, by multiplying both sides of (44) by 2, both sidesof (45) by α, both sides of (48) by γα2
2γ´1 , both sides of(49) by 2α, both sides of (50) by α, then summing them uptogether with both sides of (46) and (47), we obtain (39) ifγα2ν2
2γ´1 `αλ2 ď
2γ´1γ .
Combining the above two case, we conclude that (39)holds for all λ P R` if
γα2ν2
2γ´1 `αλ2 ď
2γ´1γ .
One can verify that the above inequality holds under As-sumption 2, which completes the proof.
APPENDIX IIPROOF OF LEMMA 3
We will again use Lemma 5.
Proof. Using line 2-4 in Algorithm 3 we can show thefollowing
zj “πDrzj´1 ´ αpPzj´1 ` q `HJwjqs,
wj “πK˝rwj´1 ` αpHp2zj´1 ´ zj´2q ´ gqs.
Applying Lemma 5 to the above two projections and usingthe definition of normal cone in (5) we can show
1α pz
j´1 ´ zjq ´ Pzj´1 ´ q ´HJwj P NDpzjq, (51a)
1α pw
j´1 ´ wjq `Hp2zj´1 ´ zjq ´ g P NK˝pwjq. (51b)
Hencedp´Pzj ´ q ´HJwj |NDpz
jqq
ď∥∥ 1α pz
j ´ zj´1q ´ Pzj ` Pzj´1∥∥
ď 1α
∥∥zj ´ zj´1∥∥` ∥∥P pzj ´ zj´1q
∥∥where the first inequality is due to the definition of distancefunction in (4) and (51a), the second inequality is due to thetriangle inequality. Finally, combining the above inequalitywith the fact that λ ‖z‖ ě ‖Pz‖ for any z P Rn, we obtainthe first inequality in Lemma 3.
Similarly, we can show the following
dpHzj ´ g|NK˝pwjqq
ď∥∥ 1α pw
j ´ wj´1q `Hpzj ´ 2zj´1 ` zj´2q∥∥
ď 1α
∥∥wj ´ wj´1∥∥` ∥∥Hpzj ´ 2zj´1 ` zj´2q
∥∥ ,where the first inequality is due to the definition of distancefunction in (4) and (51b), and the second inequality isdue to the triangle inequality. Finally, combining the aboveinequality with the fact that ν ‖z‖ ě ‖Hz‖ for any z P Rn,we obtain the second inequality in Lemma 3.
APPENDIX IIIPROOF OF LEMMA 4
We will use Lemma 1, Lemma 5 and the following result.
Lemma 6. [21, Lem. 3.2] Let tyjujPN be a sequence suchthat yj P Rn for all j P N and there exists y P Rn such thatlimjÑ8
yj
j “ y. Let D Ă Rn be a closed convex set.
1) yj ´ πDryjs P precDq˝.2) lim
jÑ8
πDryjs
j “ πrecDrys.
3) limjÑ8
yj´πDryjs
j “ πprecDq˝rys.
4) limjÑ8
xπDryjs,yj´πDry
jsy
j “ σDpπprecDq˝rysq.
We will also use the following fact: given sequencestyjujPN and tzjujPN where xj , yj P Rn for all j, if the limitslimjÑ8 y
j and limjÑ8 zj both exists and are finite-valued,
thenlimjÑ8
xyj , zjy “ x limjÑ8
yj , limjÑ8
zjy. (52)
Proof. We start by showing the two limits. From Proposi-tion 1 and Lemma 2 we know that there exists z P Rn andw P Rm such that
limjÑ8
zj
j “ limjÑ8
zj ´ zj´1 “ z, (53a)
limjÑ8
vj
j “ limjÑ8
vj ´ vj´1 “ w, (53b)
In addition, using line 4 in Algorithm 3 one can show thefollowing
vj ´ vj´1 “wj ´ wj´1 ` αHpzj ´ 2zj´1 ` zj´2q,
vj
j “wj
j `αjHpz
j ´ zj´1q,
Letting j Ñ8 in the above two equations, then using (53a)and (53b) we can show the following
limjÑ8
wj
j “ limjÑ8
wj`1 ´ wj “ w. (54)
The above equation and (53a) together give the two limitsin Lemma 4.
Next, we show that z P recD, w P K˝ and the conditionsin (19) hold. To this end, let
zj :“zj´1 ´ αpPzj´1 ` q `HJwjq (55a)
wj :“wj´1 ` αpHp2zj´1 ´ zj´2q ´ gq (55b)
Then using (53a) and (54) we can show that the followingtwo limits
limjÑ8
zj
j “ z ´ αpPz `HJwq (56a)
limjÑ8
wj
j “ w ` αHz (56b)
Further, using line 2-4 in Algorithm 3 we can show that
zj “ πDrzjs, wj “ πK˝rw
js. (57)
Hence, by applying Lemma 6 to sequences tzjujPN andtwjujPN we can show that
zj ´ zj P precDq˝, wj ´ wj P K, (58a)
limjÑ8
zj
j “ z “ πrecDrz ´ αpPz `HJwqs, (58b)
limjÑ8
zj´zj
j “´ αpPz `HJwq
“πprecDq˝rz ´ αpPz `HJwqs,
(58c)
limjÑ8
wj
j “ w “ πK˝rw ` αHzs, (58d)
limjÑ8
wj´wj
j “ αHz “ πKrw ` αHzs, (58e)
limjÑ8
xzj ,zj´zjyj “ σDp´αpPz `H
Jwqq, (58f)
limjÑ8
xwj ,wj´wjyj “ σK˝pαHzq “ 0, (58g)
where we used (53a), (54), and the fact that recK˝ “ K˝and pK˝q˝ “ K. Further, (58g) is due to (58e) and thedefinition of support function and polar cone in (8) and (9),respectively. Notice that (58b), (58d) and (58e) imply thefollowing
z P recD, w P K˝, Hz P K. (59)
Further, by applying Lemma 1 to the projections in (58b),(58c), (58d) and (58e), we obtain the following
xz, Pz `HJwy “ 0, xw,Hzy “ 0. (60)
Combining the above two equalities and the fact that P issymmetric and positive semidefinite, we obtain the following
Pz “ 0 (61)
Equipped with (59) and (61), our emaining task is to provethe last two equalities in (19). To this end, using (58a), (58b),(58d) and the definition of polar cone we can show that
xzj´1 ´ zj ´ αpq `HJwjq, zy ď 0,
xwj´1 ´ wj ` αpHp2zj´1 ´ zj´2q ´ gq, wy ď 0,(62)
where we used the definition of zj and wj in (55), and theequality in (61). By multiplying both inequalities in (62) by1αj and letting j Ñ8 we obtain the following
1α ‖z‖2
` xq, zy ě ´ limjÑ8
xwj , Hzy ě 0, (63a)
1α ‖w‖2
` xg, wy ě limjÑ8
xzj´1, HJwy ě ´σDp´HJwq,
(63b)
where the last step in (63a) is due to (57), (58e), and thedefinition of polar cone in (9). Further, the last step in (63b)is due to the definition of support function in (8). Summingup the two inequalities in (63) we obtain the following
σDp´HJwq` xq, zy` xw, gy` 1
α ‖z‖` 1α ‖w‖2
ě 0. (64)
We will show that the above inequality actually holds asan equality as follows. First, by using (52), (53a), and (58c)we can show the following
limjÑ8
1j xz
j ´ zj , zj ´ zj´1y “ x´αpPz `HJwq, zy ď 0,
(65)where the last step is due to (58c) and the definition of polarcone in (9). In addition, by applying Lemma 5 to the firstprojection in (57) we can show
xzj ´ zj , zj ´ zj´1y ě 0 (66)
Combining (65) and (66) gives the following
0 “ limjÑ8
1j xz
j ´ zj , zj ´ zj´1y. (67)
Second, one can show the following
limjÑ8
1j xz
j ´ zj , zj´1y
“ limjÑ8
xzj ´ zj´1 ` αpPzj´1 ` q `HJwjq, zj´1
j y
“ ‖z‖2` αxq, zy ` α lim
jÑ8
1j xPz
j´1 `HJwj , zj´1y
ě ‖z‖2` αxq, zy ` α lim
jÑ8
1j xH
Jwj , zj´1y,
(68)where the first step is due to (55a), the second step is dueto (52) and (53a) and the last step is due to the positivesemidefiniteness of matrix P . Third, we can show thatfollowing
0 “ limjÑ8
xwj´wj ,wjyj
“ limjÑ8
xwj ´ wj´1 ´ αpHp2zj´1 ´ zj´2q ´ gq, wj
j y
“ ‖w‖2` xw, gy ´ xHz,wy ´ α lim
jÑ8
1j xH
Jwj , zj´1y,
(69)where the first equality is due to (58g), the second equalityis due to (55b), and the last equality is due to (52), (53a) and(54). By summing up both sides of (58f), (67), (68) and (69)then multiplying the resulting inequality by 1
α , we obtain thefollowing
σDp´HJwq ` xq, zy ` xw, gy ` 1
α ‖z‖` 1α ‖w‖2
ď 0,
where we also used (60), (61), and the fact thatασDp´H
Jwq “ σDp´αHJwq when α ą 0. Combining the
above inequality with the one in (64), we conclude that theinequality in (64) holds as an equality. As a result, bothinequalities in (63) also hold as equalities, which completesthe proof.
APPENDIX IVTRANSFORMATION FROM OPTIMAL CONTROL PROBLEMS
TO CONIC OPTIMIZATION PROBLEMS
We let 1n and 0n denote the n-dimensional vectors ofall 1’s and all 0’s, respectively. Let diagpcq denote thediagonal matrix whose diagonal elements are given by vectorc. Further, we let
A “
„
I6pτ´1q
06ˆ6pτ´1q
´
„
06ˆ6pτ´1q
Iτ´1 bA
,
B “ ´Iτ bB, C “ Iτ b“
0 0 1‰
,
D “ Iτ´1 b
„
I3 03ˆ3
´I3 03ˆ3
, E “ Iτ´1 b
„
r1 ´ r2
r2 ´ r1
,
and
H1 “
„
A B0τˆ6pτ´1q C
,
g1 “ r pAx0`hqJp1τ´2bhq
Jp´xτ`hq
J ρ11Jτ sJ,
H2 ““
D 06pτ´1qˆ3τ E‰
, g2 “ 1τ´1 b
„
r1
´r1
P1 “ diag
ˆ
”
0J6pτ´1q 1J3τ
ıJ˙
, q1 “ 09τ´6,
P2 “ diag
ˆ
”
0J6pτ´1q 1J3τ 0Jτ´1
ıJ˙
, q2 “ 010τ´7.
Further, we let
X1 “tr P R3| ‖r‖ cosβ ď rrs2u ˆ tr P R3| ‖r‖ ď ηu,
X2 “R3 ˆ tr P R3| ‖r‖ ď ηu,
U1 “tu P R3| ‖u‖ cos θ ď rus3, ‖u‖ ď ρ2u,
K1 “t06τu ˆ Rτ`.
With the above definitions, we are ready to transform theoptimal control problems in Section IV into space cases ofconic optimization (1). Optimization (31) is the special caseof (1) where
z ““
xJ1 . . . xJτ´1 uJ0 . . . uJτ´1
‰J,
P “ P1, q “ q1, H “ S1H1, g “ S1g1, K “ K1,
D “ pX1qj´1 ˆ txτu
τ´j ˆ pU1qτ ,
where S1 is a diagonal matrix with positive diagonal entriessuch that the rows in matrix H have unit `2 norm. Andoptimization (31) is the special case of (1) where
z ““
xJ1 . . . xJτ´1 uJ0 . . . uJτ´1 b1 . . . bτ´1
‰J
H “ S2
„“
H1 07τˆpτ´1q
‰
H2
, g “ S2
„
g1
g2
,
P “ P2, q “ q2, K “ K1 ˆ R6pτ´1q` ,
D “ pX2qτ´1 ˆ pU1q
τ ˆ t1´ bu ˆ r0, 1sτ´2,
where S2 is a diagonal matrix with positive diagonal entriessuch that the rows in matrix H have unit `2 norm. Noticethat, in both problems, we use matrix S1 and S2 to ensurethe constraints parameters H and g are well-conditioned.
REFERENCES
[1] A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimiza-tion: Analysis, Algorithms, and Engineering Applications. SIAM,2001.
[2] S. P. Boyd and L. Vandenberghe, Convex Optimization. CambridgeUniversity Press, 2004.
[3] G. Banjac, P. Goulart, B. Stellato, and S. Boyd, “Infeasibility detectionin the alternating direction method of multipliers for convex optimiza-tion,” J. Optim. Theory Appl., vol. 183, no. 2, pp. 490–519, 2019.
[4] G. Banjac, “On the minimal displacement vector of the Douglas–Rachford operator,” Ope. Res. Lett., vol. 49, no. 2, pp. 197–200, 2021.
[5] S. Barratt, G. Angeris, and S. Boyd, “Automatic repair of convexoptimization problems,” Optim. Eng., vol. 22, pp. 247–259, 2021.
[6] A. Agrawal and S. Boyd, “Disciplined quasiconvex programming,”Optim. Lett., vol. 14, no. 7, pp. 1643–1657, 2020.
[7] M. Conforti, G. Cornuejols, G. Zambelli, et al., Integer programming.Springer, 2014, vol. 271.
[8] L. A. Wolsey, Integer programming. John Wiley & Sons, 2020.
[9] A. U. Raghunathan and S. Di Cairano, “Infeasibility detection in alter-nating direction method of multipliers for convex quadratic programs,”in Proc. IEEE Conf. Decision Control. IEEE, 2014, pp. 5819–5824.
[10] B. O’Donoghue, E. Chu, N. Parikh, and S. Boyd, “Conic optimizationvia operator splitting and homogeneous self-dual embedding,” J.Optim. Theory Appl., vol. 169, no. 3, pp. 1042–1068, 2016.
[11] Y. Liu, E. K. Ryu, and W. Yin, “A new use of Douglas–Rachfordsplitting for identifying infeasible, unbounded, and pathological conicprograms,” Math. Program., vol. 177, no. 1, pp. 225–253, 2019.
[12] Y. Ye, M. J. Todd, and S. Mizuno, “An Op?nLq-iteration homoge-
neous and self-dual linear programming algorithm,” Math. Oper. Res.,vol. 19, no. 1, pp. 53–67, 1994.
[13] Y. Nesterov, M. J. Todd, and Y. Ye, “Infeasible-start primal-dual meth-ods and infeasibility detectors for nonlinear programming problems,”Math. Program., vol. 84, no. 2, pp. 227–268, 1999.
[14] D. Malyuta, T. P. Reynolds, M. Szmuk, T. Lew, R. Bonalli, M. Pavone,and B. Acikmese, “Convex optimization for trajectory generation,”arXiv preprint arXiv:2106.09125 [math.OC], 2021.
[15] D. Malyuta, Y. Yu, P. Elango, and B. Acikmese, “Advances intrajectory optimization for space vehicle control,” arXiv preprintarXiv:2108.02335 [math.OC], 2021.
[16] Y. Malitsky, “The primal-dual hybrid gradient method reduces to aprimal method for linearly constrained optimization problems,” arXivpreprint arXiv:1706.02602 [math.OC], 2017.
[17] D. Applegate, M. Dıaz, H. Lu, and M. Lubin, “Infeasibility detectionwith primal-dual hybrid gradient for large-scale linear programming,”arXiv preprint arXiv:2102.04592 [math.OC], 2021.
[18] H. H. Bauschke and P. L. Combettes, Convex analysis and monotoneoperator theory in Hilbert spaces. Springer, 2017, vol. 408.
[19] R. T. Rockafellar and R. J.-B. Wets, Variational analysis. SpringerScience & Business Media, 2009, vol. 317.
[20] P. Giselsson and S. Boyd, “Linear convergence and metric selection forDouglas-Rachford splitting and ADMM,” IEEE Trans. Autom. Control,vol. 62, no. 2, pp. 532–544, 2016.
[21] G. Banjac and J. Lygeros, “On the asymptotic behavior of theDouglas–Rachford and proximal-point algorithms for convex opti-mization,” Optim. Lett., pp. 1–14, 2021.
[22] Y. Yu, P. Elango, and B. Acıkmese, “Proportional-integral projectedgradient method for model predictive control,” IEEE Control Syst.Lett., 2020.
[23] Y. Yu, P. Elango, U. Topcu, and B. Acikmes, e, “Proportional-integralprojected gradient method for conic optimization,” arXiv preprintarXiv:2108.10260 [math.OC], 2021.
[24] J. Wang and N. Elia, “Control approach to distributed optimization,”in Proc. Allerton Conf. Commun. Control Comput. IEEE, 2010, pp.557–561.
[25] Y. Yu, B. Acıkmese, and M. Mesbahi, “Mass–spring–damper networksfor distributed optimization in non-Euclidean spaces,” Automatica, vol.112, p. 108703, 2020.
[26] Y. Yu and B. Acıkmese, “RLC circuits-based distributed mirror descentmethod,” IEEE Control Syst. Lett., vol. 4, no. 3, pp. 548–553, 2020.
[27] J. Kuczynski and H. Wozniakowski, “Estimating the largest eigenvalueby the power and lanczos algorithms with a random start,” SIAM JMatrix Anal. Appl., vol. 13, no. 4, pp. 1094–1122, 1992.
[28] M. Szmuk, “Successive convexification & high performance feedbackcontrol for agile flight,” Ph.D. dissertation, Dept. Aeronatu. & Astro-naut., Univ. Washington, 2019.
[29] X. Zhang, Y. Fang, and N. Sun, “Minimum-time trajectory planningfor underactuated overhead crane systems with state and controlconstraints,” IEEE Trans. Ind. Electron., vol. 61, no. 12, pp. 6915–6925, 2014.
[30] Z. Li and M. Yan, “New convergence analysis of a primal-dualalgorithm with large stepsizes,” Adv. Comput. Math., vol. 47, no. 1,pp. 1–20, 2021.