ccqo poly regularknowledge, only a few results on polynomially solvable cases of ccqo problem are...

A polynomial case of cardinality constrained quadratic

optimization problem∗

Jianjun Gao † Duan Li ‡

August 27, 2010

Abstract

We investigate in this paper a fixed parameter polynomial algorithm for the cardinalityconstrained quadratic optimization problem, which is NP-hard in general. More specifi-cally, we prove that, given a problem of size n, the number of decision variables, and s,the cardinality, if, for some 0 < k ≤ n, the n − k largest eigenvalues of the coefficientmatrix of the problem are identical, we can construct a solution algorithm with computa-tional complexity of O

(n2k

), which is independent of the cardinality s. Our main idea is

to decompose the primary problem into several convex subproblems, while the total num-ber of the subproblems is determined by the cell enumeration algorithm for hyperplanearrangement in R

k space.

Keywords: Cardinality constrained quadratic optimization, cell enumeration, nonconvexoptimization, fixed parameter polynomial algorithm

1 Introduction

We consider in this paper the following cardinality constrained quadratic optimization problem(CCQO),

(P) : minx

f(x) =1

2x′Qx+ q′x

Subject to: x ∈ ∆(s) ,

x ∈ Rn |

n∑

t=1

δ(xi) ≤ s < T

, (1)

where Q ∈ Rn×n is positive definite, q ∈ R

n \ 0, and the indicator function δ(·) : R→ 0, 1is defined such that δ(a) = 1 if a is non-zero and δ(a) = 0 otherwise.

∗This work was supported by Research Grants Council of Hong Kong, under grants 414207 and 414808.†Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong,

Shatin, Hong Kong ([email protected])‡Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong,

Shatin, Hong Kong.([email protected]).

1

Polynomially solvable case of CCQP 2

This class of quadratic optimization problems with a cardinality constraint arises naturallyfrom various applicationsThe exact and approximate solution approaches for cardinality con-strained optimization problems are studied in the literatures, e.g., [2], [3], [9], [12]. To our bestknowledge, only a few results on polynomially solvable cases of CCQO problem are reportedin the literature. In the context of subset selection problem, Das and Kempe [5] proposed anapproximate algorithm for the case where the covariance possesses a constant bandwidth anddeveloped an exact algorithm for the case where the covariance graph is of a tree structure.Recently, Donoho and Candes [4][6] showed that, under some conditions, using l1 norm toreplace the cardinality constraint in the sparse signal reconstruction problem yields the exactsolution with an overwhelming probability.

We focus in this paper on a class of CCQO problems with a special structure. Morespecifically, we consider situations where the n−k largest eigenvalues of matrix Q are identical,0 ≤ k ≤ n, which we term as a matrix Q with a k-degree freedom (see Definition 2.1). Weprove that, for fixed k, this class of CCQO problems is polynomially solvable. Motivatedby the geometrical characteristics of CCQO problem, we decompose the problem (P) intoseveral convex quadratic programming subproblems and the number of these sub-problems isdetermined by a cell enumeration algorithm for the hyperplane arrangement in R

k space [1].From the complexity point of view, if k is fixed, the solution scheme is a polynomial-timealgorithm. To certain extent, our result in this paper is similar to a polynomially solvable casein binary quadratic program, where the rank of coefficient matrix is fixed (see [8]).

This paper is organized as following. After the introduction in this section, we develop thesolution scheme for CCQO problem with a k-degree freedom coefficient matrix in Section 2.As the derived solution scheme depends heavily on a distance function between the cardinalityfeasible set and an affine space, we develop a scheme for identifying such a distance functionin Section 3 using cell enumeration of hyperplane arrangement in discrete geometry. Afterpresenting an illustrative example in Section 4, we conclude the paper in Section 5.

Throughout the paper, we use v(·) to denote the optimal value of problem (·), S ≻ 0 apositive definite matrix, Sn++ the set of positive definite matrices, diaga ∈ R

n×n the diagonalmatrix with a ∈ R

n being its diagonal, 0 the vector with all elements being 0 and ‖ · ‖ the l2norm. Furthermore, we denote the ellipsoid and the ball in R

n, respectively, by

E(P, p, ρ) ,y ∈ R

n | (y − p)′P (y − p) ≤ ρ, P ≻ 0, ρ ≥ 0, (2)

B(p, r2) ,y ∈ R

n | ‖y − p‖2 ≤ r2. (3)

2 Solution scheme of problem (P)

2.1 Preliminary

Problem (P) has been proved to be, in general, NP-hard (see the proof in [11]). Here we givean alternative proof for the NP-hardness of problem (P), which appears to be much simplerthan the one in [11]. Let us construct the following problem,

(G) : minx∈Rn

f := M‖x− 1‖2 + ‖Ax‖2 | x ∈ ∆(s)

,


where M > 0 is a large number, 1 is the vector with all elements being 1, and A ∈ Rl×n

with l ≤ n. Note that any instance of problem (G) is polynomially reducible to an instance ofproblem (P). That is to say, solving problem (G) is no more difficult than solving problem (P).Since x ∈ ∆(s), minimizing the first term of (G) enforces xi to take either 0 or 1 for i = 1, · · · , n.More specifically, at least n− s of xi’s are zero. Thus, the optimal value of problem G is lowerbounded, i.e., v(G) ≥ M(n − s). Answering the question “whether equality v(G) = M(n − s)holds or not ” turns out to find the integer (binary) solution of linear systems Ax = 0 suchthat x ∈ 0, 1n and

∑

i=1 xi ≤ s, which is a known NP-complete decision problem [7]. Ourconclusion for the NP-hardness of problem (P) follows the simple reduction method [7].

Geometrically, the objective contour of (P) is an ellipsoid in Rn space,

E(Q,h, ρ) ,x ∈ Rn | f(x) ≤ τ

=x ∈ Rn| (x− h)′Q(x− h) ≤ ρ,

where

h , −Q−1q, (4)

C , −1

2q′Q−1q, (5)

ρ , 2τ − 2C. (6)

Clearly, we must have τ ≥ C. Minimizing f(x) under constraint (1) is now equivalent tofinding the minimum ellipsoid that touches the set ∆(s), or equivalently, we can reformulateproblem (P) as follows,

(P1) : minx,ρ

1

2ρ+ C, (7)

Subject to: x ∈ E(Q,h, ρ), (8)

x ∈ ∆(s).

In the following, we choose to deal with problem formulation (P1), instead of problem formu-lation (P). Figure 1 illustrates a case where n = 2 and s = 1 and the feasible set ∆(s) consistsof both x-axis and y-axis in R

2 plane. It is clear from the figure that the optimal contour isthe minimum ellipsoid that touches the x-axis. Note that the number of feasible subspacescould be as large as

∑sj=1 C

jn, where C

jn = n!

j!(n−j)! .

Let the spectral decomposition of matrix Q be Q = Γ′ΛQΓ, where matrix Γ is unitary and

ΛQ , diagλQ1 , λ

Q2 , · · · , λ

Qn . (9)

Without loss of generality, we assume the eigenvalues of matrix Q to be arranged in an as-cending order,

0 < λQ1 ≤ λQ

2 ≤ · · · ≤ λQn .

Definition 2.1. Matrix H ∈ Sn+ is said to be of k-degree freedom, if there exists k such that

0 ≤ k ≤ n and 0 ≤ λH1 ≤ λH

2 · · ·λHk < λH

k+1 = λHk+2 = · · · = λH

n , where λHi is the i-th smallest

eigenvalue of H.


−1 0 1 2 3 4 5 6 7 8 9 10−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

3

3.5

4

Figure 1: The optimal contour and ∆(s) when n = 2 and s = 1

In plain words, a positive definite matrix is of k-degree freedom, if the n− k largest eigen-values are identical. If k = 0, the 0-degree freedom matrix is a diagonal matrix with identicaleigenvalues. If k = n, all the eigenvalues of the matrix have their full freedom. It is veryinteresting to investigate the cases when 0 < k < n, as we will demonstrate in the followingthat any CCQO problems can be always approximated by such a matrix of k-degree freedom.For problem (P), we can chose a k and construct an auxiliary problem ,

(A) : minx

f(x) =1

2(x− h)′A(x− h) + C,

Subject to: x ∈ ∆(s),

where A ∈ Sn+ is a matrix of k-degree freedom specified by A = Γ′ΛkΓ with

Λk = diag(λQ1 , λ

Q2 , · · · , λ

Qk , λ

Qk , · · · , λ

Qk

).

Lemma 2.1. If Q ≻ 0, the following relationships hold,

v(A) ≤ v(P) and v(P) − v(A) ≤ (λQn − λQ

k )Φ,

where Φ is a parameter dependent on problem (P).

Proof. Since Q A, we have f(x) ≤ f(x) for all x ∈ Rn, which implies that v(A) ≤ v(P). Let

x be the optimal solution of problem (A), thus giving rise to

v(P) − v(A) ≤ f(x)− f(x) =1

2(x− h)′(Q−A)(x− h)

≤1

2(λQ

n − λQk )‖x− h‖2. (10)

As the following holds true for any x ∈ ∆(x) in problem (A),

λQ1 ‖x− h‖2 + C ≤ f(x) ≤ λQ

k ‖x− h‖2 + C, ∀x ∈ ∆(x). (11)


We can minimize the upper bound λQk ‖x−h‖2 in (11) by taking x = (x1, · · · , xn) ∈ ∆(s) with

xi = hi for i ∈ I and xi = 0 for i 6∈ I, where I is the index set consisting of the first s largestelements of |hi|. Then, inequality (11) becomes

‖x− h‖2 ≤λQk

λQ1

∑

i 6∈I

h2i . (12)

Combining (10) and (12) yields the conclusion in the lemma.

Lemma 2.1 suggests that increasing the order of k may reduce the gap between the auxiliaryand the primary problems. However, the computational burden of solving auxiliary problem(A) may increase for a larger k at the same time.

2.2 A decomposition approach

In this section, we develop an efficient solution scheme for problem (P) with Q being of ak-degree freedom. Our main idea for solving such a class of problems is to decompose (P) intoseveral convex quadratic subproblems. Before we state our main results, we introduce someresults on the decomposition of ellipsoid E(Q,h, ρ) when Q is of a k-degree freedom.

Theorem 2.1. Let E(Λ,0, γ) be an ellipsoid with γ > 0, where Λ = diagλi|ni=1 is of k-degree

freedom with λi being the i-th smallest eigenvalue, 1 < k < n.

(i) For any α ∈ Rn such that

α ∈ Ek ,

(u1, · · · , un)′ |

k∑

i=1

λiλ2k+1u

2i

(λk+1 − λi)2≤ γ, and uj = 0, j = k + 1, · · · , n

,

the following holds,

r2(α) ,γ

λk+1−

k∑

i=1

λiα2i

λk+1 − λi≥ 0. (13)

(ii) The ellipsoid E(Λ,0, γ) is the union of the balls expressed as follows,

E(Λ,0, γ) =⋃

α∈Ek

B(α, r2(α)).

Proof. (i) Since Λ is of k-degree freedom, we have λk+1 > λi > 0 for i = 1, · · · , k, whichfurther implies λk+1/(λk+1 − λi) > 1, for i = 1, · · · , k. Then, for any α ∈ Ek, thefollowing inequality holds,

k∑

i=1

λiλk+1α2i

(λk+1 − λi)≤

k∑

i=1

λiλ2k+1α

2i

(λk+1 − λi)2≤ γ.

Dividing both sides by λk+1 gives rise to the result in (i).


(ii) For any y∗ ∈ E(Λ,0, γ), we have

k∑

i=1

λi(y∗i )

2 + λk+1

n∑

j=k+1

(y∗j )2 ≤ γ. (14)

Let α∗ = (α∗1, α

∗2, · · · , α

∗n)

′ be defined such that α∗i = y∗i (λk+1−λi)/λk+1 for i = 1, · · · , k

and α∗j = 0 for j = k + 1, · · · , n. Then, we have

k∑

i=1

λiλ2k+1(α

∗i )

2

(λk+1 − λi)2=

k∑

i=1

(y∗i )2λi ≤ γ − λk+1

n∑

i=k+1

(y∗i )2 ≤ γ,

which implies that α∗ ∈ Ek. Define

r2(α∗) =γ

λk+1−

k∑

i=1

λi(α∗i )

2

λk+1 − λi.

From the result in (i), we have r2(α∗) ≥ 0. We can further conclude that y∗ ∈B(α∗, r2(α∗)) by checking the following inequality,

‖y∗ − α∗‖22 − r2(α∗) =

k∑

i=1

(y∗i − y∗iλk+1 − λi

λk+1)2 +

n∑

j=k+1

(y∗j )2

− (γ

λk+1−

k∑

i=1

(y∗i )2λi(λk+1 − λi)

λ2k+1

),

=k∑

i=1

(y∗i )2λi

λk+1+

n∑

j=k+1

(y∗j )2 − (

γ

λk+1) ≤ 0,

where the last inequality is implied by (14). Thus, we conclude that, for any y∗ ∈E(Λ,0, γ), there exists α∗ ∈ Ek such that y∗ ∈ B(α∗, r2(α∗)), which further impliesE(Λ,0, γ) ⊆

⋃

α∗∈EkB(α∗, r2(α∗)).

On the other hand, for any y ∈⋃

α∈EkB(α, r2(α)), there exists α ∈ Ek such that y ∈

B(α, r2(α)) with

r2(α) =γ

λk+1−

k∑

i=1

α2i λi

λk+1 − λi≥ 0.

As, for i = 1, · · · , k,

λk+1 − λi

λk+1(yi −

αiλk+1

λk+1 − λi)2 ≥ 0,

we have

y2i λi

λk+1≤ (yi − αi)

2 +α2i λi

λk+1 − λi, i = 1, · · · , k, (15)


−4 −3 −2 −1 0 1 2 3 4−3

−2

−1

0

1

2

3

Figure 2: Decomposition of ellipsoid in R2

which further gives rise to

k∑

i=1

λiy2i

λk+1+

n∑

j=k+1

y2j ≤

k∑

i=1

[(yi − αi)

2 +α2i λi

λk+1 − λi

]+

n∑

j=k+1

y2j . (16)

Since y ∈ B(α, r2(α)), we have

k∑

i=1

[(yi − αi)

2] +

n∑

j=k+1

y2j ≤γ

λk+1−

k∑

i=1

α2i λi

λk+1 − λi. (17)

Combining inequalities (16) and (17) yields

k∑

i=1

λiy2i +

n∑

j=k+1

λk+1y2j ≤ γ,

which implies y ∈ E(Λ,0, γ). We finally conclude that⋃

α∈EkB(α, r2) ⊆ E(Λ,0, γ).

Theorem 2.1 actually provides a parameterized representation of the ellipsoid E(Λ,0, γ) byinfinite number of balls, when Λ is of k-degree freedom. Figure 2 illustrates this decompositionscheme for an ellipsoid in R

2 with 1-degree freedom. It is obvious that the center of the ballsis along the longer radius of the ellipsoid and the union of infinite such balls is nothing but theellipsoid itself.

We now proceed to extend the decomposition scheme in Theorem 2.1 to the ellipsoidE(Q,h, ρ) in constraint (8), when Q is of k-degree freedom. For any ρ ≥ 0, we first decomposeellipsoid E(ΛQ,0, ρ) as follows,

E(ΛQ,0, ρ) =⋃

α∈EQ

k

B(α, r2(α)),


where

EQk ,

u ∈ Rn |

k∑

i=1

κiu2i ≤ ρ, uj = 0, j = k + 1, · · · , n

, (18)

r2(α) , (ρ−

k∑

i=1

(ιiα2i ))/λ

Qk+1, (19)

κi ,λQi (λ

Qk+1)

2

(λQk+1 − λQ

i )2, for i = 1, · · · , k, (20)

ιi ,(λQ

i λQk+1)

λQk+1 − λQ

i

, for i = 1, · · · , k. (21)

As the shape and size of ellipsoid E(ΛQ, 0, ρ) are coordinate independent, the affine transfor-mation x = Γ′y + h maps y ∈ E(Λ, 0, ρ) to x ∈ E(Q,h, ρ). Thus, the constraint (8) in problem(P) can be expressed as follows according to Theorem 2.1,

x ∈ E(Q,h, ρ) =⋃

α∈EQ

k

B(Γ′α+ h, r2(α)),

where EQk and r2(α) are defined by (18) and (19), respectively. Furthermore, problem (P1)

can be reformulated as,

(P2) : minα,ρ

1

2ρ+ C,

Subject to: x ∈⋃

α∈EQk

x | λk+1‖x− Γ′α− h‖22 +

k∑

i=1

ιiα2i ≤ ρ

, (22)

x ∈ ∆(s). (23)

The formulation of problem (P2) is still hard to solve, as the constraint in (22) involves aninfinite number of balls. As we pointed out before, minimizing f(x) under constraint (1) isequivalent to finding the minimum ellipsoid that touches the set ∆(s). We further recognizehere that, among infinite balls which together make up the minimum ellipsoid, one specificball offers the tangent point to achieve this task. Please refer to Figure 1 again, in which thered ball serves exactly this purpose. Based on this argument, we now construct the followingproblem to identify this particular ball in order to solve problem (P2), albeit indirectly,

(P) : minρ,β

1

2ρ+ C,

Subject to: λk+1dis(β) +

k∑

i=1

ιiβ2i ≤ ρ, (24)

k∑

i=1

κiβ2i ≤ ρ, (25)


where decision variables are β ∈ Rk, ρ ∈ R and the distance function dis(·) : Rk → R+ defined

as

dis(β) , minx∈Rn

‖x−Hβ − h‖22 | x ∈ ∆(s)

, (26)

with H ∈ Rn×k being formed by taking the first k columns of Γ′. Let Hi ∈ R

1×k be the i-throw of matrix H and hi be the i-th element of h.

Define b(β) = (b1(β), b2(β), · · · , bn(β))′, where

bi(β) , |Hiβ + hi|, for i = 1, · · · , n. (27)

Definition 2.1. We define an index set I(β) ⊂ 1, 2, · · · , n as the set that includes indicesof the first n − s smallest elements in b(β) and the complementary set of I(β) as I(β) =1, · · · , n \ I(β).

Note that for any fixed β, I(β) may not be unique. When we have multiple candidates ofbi(β) to be chosen as the (n− s)th smallest element in b(β) or the last element in I(β), we cantake an arbitrary choice and this does not affect our discussion.

Theorem 2.2. If solution-pair (β, ρ) solves problem (P), then solution-triple (α, ρ, x) solvesproblem (P2), where α = ((β)′,0′n−k)

′, x = (x1, · · · , xn)′ with

xi =

Hiβ + hi i ∈ I(β),

0 i ∈ I(β),(28)

and index sets I(β) and I(β) are defined in Definition 2.1.

Proof. Substituting (α, ρ, x) into (22) and (23) confirms the feasibility of (α, ρ, x) in problem(P2), thus giving rise to v(P2) ≤ v(P). Now we assume that solution-triple (α, ρ, x) solvesproblem (P2) with v(P2) =

12 ρ+C < v(P) = 1

2 ρ+C. Since (α, ρ, x) is the solution of problem

(P2), (α, ρ) satisfies constraint (25) in problem (P). On the other hand, note that the followinginequality always holds,

λk+1(minx‖x−Hβ − h‖22) +

k∑

i=1

ιiβ2i ≤ λk+1‖x− Γ′α− h‖22 +

k∑

i=1

ιiα2i ≤ ρ, (29)

where α and β satisfy α = (β′, 0′n−k)′. Inequality (29) implies that (α, ρ) satisfies constraint

(24). From our assumption that 12 ρ + C < 1

2 ρ + C, we find a better solution (α, ρ, x) for

problem (P), which contradicts the optimality of solution (α, ρ) for problem (P). Note thatonce optimal β is fixed, the optimal x can be found by minimizing (26), i.e., we have

dis(β) = minx∈∆(s)

n∑

i=1

(xi −Hiβ − hi)2. (30)

Since x ∈ ∆(s), we choose xi = Hiβ+hi, for i ∈ I and xi = 0 for i ∈ I which minimizes dis(β)in (30).


Theorem 2.2 reveals an equivalence between problems (P) and (P2). While we will focuson problem (P) in the following, the key issue we are facing is how to handle function dis(β)defined in (26). Clearly, function dis(β) measures the distance between the affine space, y ∈Rn | y = Hβ + h and the set ∆(s). We will carry out detailed discussion on how to identify

function dis(β) in Section 3.

3 Distance Evaluation

Given h ∈ Rn and H ∈ R

n×k with rank(H) = k, the distance function, dis(·), is defined in(26). In this section, we focus on distance function dis(β), which plays a key role in solvingproblem (P). In particular, we show that dis(β) is a piece-wise continuous convex quadraticfunction and all the coefficients can be found explicitly by implementing our proposed algo-rithm. Geometrically, dis(β) identifies the minimum distance between an affine space and thefeasible set ∆(s),

dis(β) = minx∈Rn

‖y − x‖2 | y ∈ Yk(H,h), x ∈ ∆(s)

,

where Yk(H,h) =y ∈ R

n | y = Hβ + h, β ∈ Rk. Although the number of feasible subspaces

in ∆(s) is of a combinatorial nature, the function dis(β) can be still characterized efficiently.We now borrow some concepts from the discrete geometry [1] by considering the following

hyperplane arrangements generated by the following hyperlanes in Rk,

p1i,j , β ∈ Rk | (Hi +Hj)β + (hi + hj) = 0, (31)

p2i,j , β ∈ Rk | (Hi −Hj)β + (hi − hj) = 0, (32)

for (i, j) ∈ I , (i, j)|i = 1, · · · , n − 1, j = i + 1, · · · , n. Note that the total number of suchhyperplanes is n(n − 1). A cell E of the hyperplane arrangements corresponding to p1i,j and

p2i,j is a k-dimensional polyhedral set formed by the half spaces induced by hyperlanes p1i,j and

p2i,j, (i, j) ∈ I.

We characterize the positive and negative half spaces of p1i,j and p2i,j, respectively, by

w1i,j =

+ if (Hi +Hj)β + (hi + hj) ≥ 0− if (Hi +Hj)β + (hi + hj) < 0

, for (i, j) ∈ I, (33)

w2i,j =

+ if (Hi −Hj)β + (hi − hj) ≥ 0− if (Hi −Hj)β + (hi − hj) < 0

, for (i, j) ∈ I. (34)

Thus, any cell can be characterized by an (n× n) up-triangular sign matrix w,

sign(E) = w =

0 w1,2 w1,3 · · · w1,n

0 w2,3 · · · w2,n

· · · · · · · · ·wn−1,n

0

, (35)


where wi,j is specified bywi,j = (w1

i,j w2i,j) (36)

and the operator “” is defined such that (+ +) = +, (+ −) = −, (− +) = − and(− −) = +.

Lemma 3.1. In each cell E, induced by hyperplane arrangements p1i,j and p2i,j, (i, j) ∈ I, theorder of functions bi(β)

ni=1 is invariant within the cell, i.e., for a permutation of index set

1, 2, · · · , n, i1, i2, · · · , in, the following holds true when β varies within cell E,

bi1(β) ≤ bi2(β) · · · ≤ bin(β).

Proof. Clearly, the order of bi(β)ni=1 is determined by comparing whether bi(β)− bj(β) ≥ 0

or not, for all pair (i, j) ∈ I. Since bi(β) ≥ 0 for all i, checking whether bi(β) − bj(β) ≥ 0 ornot is equivalent to checking whether the difference bi(β)

2− bj(β)2 is nonnegative or not. Note

that, for any (i, j) ∈ I, we have

(bi(β))2 − (bj(β))

2 =((Hi +Hj)β + (hi + hj)

)((Hi −Hj)β + (hi − hj)

), (37)

which further implies

(bi(β))2 − (bj(β))

2

≥ 0 if (w1

i,j = +, w2i,j = +) or (w1

i,j = −, w2i,j = −),

≤ 0 if (w1i,j = −, w

2i,j = +) or (w1

i,j = +, w2i,j = −).

As any point β in cell E possesses the same sign vector sign(E), thus the order of bi(β)|ni=1

is invariant within each cell E.

When k = 0, the affine space Yk(H,h) degenerates to a singleton h, i.e., Y0 = y ∈ Rn |y = h, and b|ni=1 = |hi||

ni=1. As the index set I(β), in such a case, includes the indices

corresponding to the first n − s smallest elements of |h|i|ni=1, the distance function dis(β)

becomes a constant which can be explicitly expressed.

Lemma 3.2. When k = 0, the projection of h on ∆(s) is x∗ = x∗i |ni=1 = argminx∈∆(s) ‖h−

x‖22 with

x∗i =

hi i ∈ I(β)0 i ∈ I(β)

and dis(β) =∑

i∈I(β)(hi)2.

Note that the projection point may not be unique. We continue to prove that, whenk ≥ 1, the distance function is a piece-wise quadratic function. Since each cell of a hyperplanearrangement is a polyhedra, we use a unified expression Ψtβ ≤ ηt, where Ψt ∈ R

m×k andηt ∈ R

m, for cell t. While m is always bounded as m ≤ n(n − 1), it is bounded from abovemore tightly by the number of hyperplanes which are active for the concerned cell. We willdescribe in details our algorithm in Section 3.1 and Section 3.2 to search for all cells.


Theorem 3.1. The distance function dis(β) is a piece-wise continuous quadratic function,with the following quadratic form with respect to β for each cell indexed by t ∈ 1, · · · , N,i.e.,

dis(β) = β′Dtβ + dtβ + ct, ∀β satisfying Ψtβ ≤ ηt, (38)

where Dt ∈ Sk++, dt ∈ R

k and ct ∈ R. Furthermore, the total number of cells, N , is boundedfrom above by O

((n2 − n)k

).

Proof. For any fixed β, applying Lemma 3.2 gives rise to

dis(β) = minx∈∆(s)

‖Hβ + h− x‖22 = minx∈∆(s)

n∑

i=1

(H ′iβ + hi − xi)

2

=∑

i∈I(β)

bi(β)2 = β′Dβ + dβ + c, (39)

where

D =∑

i∈I(β)

H ′iHi, d = 2

∑

i∈I(β)

hiHi, c =∑

i∈I(β)

h2i . (40)

While the sets I(β) and I(β) are completely determined by the order of bi(β)ni=1, the order

of functions bi(β)ni=1 is invariant within each cell E induced by hyperplane arrangements in

(31) and (32) according to Lemma 3.1. It has been known that the upper bound on the numberof cells of the hyperplane arrangement generated by (31) and (32) is in order of O

((n2 − n)k

)

(see [1]).Now we prove the continuity of function dis(β). Since dis(β) is a continuous function in

the interior of each cell E, we only have to check the boundary between cells. Without loss ofgenerality, we consider two neighboring cells, E1 and E2, which are separated by hyperplanep1i∗,j∗. There exist two different cases: i) If E1 and E2 define a same index set I(β), then dis(β)is the same for both E1 and E2, which implies the continuity of dis(β). ii) Assume that cellsof E1 and E2 define two different index sets I1(β) and I2(β). Since hyperplane p

1i∗,j∗ separates

these two cells, we know that bi∗(β) and bj∗(β) change order from the proof of Lemma 3.1.More specifically, the index sets I1(β) and I2(β) are different in two indices, i∗ and j∗,

I1(β) = i∗ ∪ (I1(β) ∩ I2(β)),

I2(β) = j∗ ∪ (I1(β) ∩ I2(β)).

Thus, we have

dis(β) =∑

i∈I1(β)∩I2(β)

bi(β)2 + b2i∗(β), β in cell E1,

dis(β) =∑

i∈I1(β)∩I2(β)

bi(β)2 + b2j∗(β), β in cell E2


Clearly, on the hyperplane p1i∗,j∗, which is the boundary between cells E1 and E2, we have

b2i∗(β) = b2j∗(β) from the definition (31) and relationship (37), which further implies that

dis(β) is continuous on the boundary p1i∗,j∗.

From Theorem 3.1, we know that the distance function dis(β) is a piece-wise quadraticfunction defined on the cells of hyperplane arrangements, which takes the form in (38) andthe total number of the pieces, N , is bounded by O((n2 − n)k). Although problem (P) is notconvex, it can still be solved by evaluating N subproblems Pt, t = 1, · · · , N , separately, asfollows,

(Pt) : min1

2ρ+ C, (41)

Subject to: λQk+1(β

′Dtβ + dtβ + ct) +

k∑

i=1

ιiβ2i ≤ ρ,

k∑

i=1

κiβ2i ≤ ρ, (42)

Ψtβ ≤ ηt.

Once we solve all these subproblems, we can identify the optimal solution (β∗, ρ∗) by solvinga particular sub-problem (Pt∗), where

t∗ = arg mint=1,··· ,N

v(Pt).

Note that when β∗ is fixed, we simply use formulation (28) to identify the optimal solution ofproblem (P) with optimal objective v(P) = 1

2ρ∗ + C.

Remark 3.1. From Corollary 3.1 in Section 3, when k = 1, the total number of the quadraticpieces of dis(β) has a bound, N ≤ n(n − 1). Furthermore, the sub-problem (Pt) can besimplified to the following problem,

(Pt) : minβ,ρ

τ =1

2ρ+ C,

Subject to: λQ2 (Dtβ

2 + dtβ + ct) + ι1β2 ≤ ρ, (43)

κ1β2 ≤ ρ, (44)

It ≤ β ≤ It+1.

Compared with the general case with k > 1, a more efficient algorithm is devised in Section3.1 to identify the distance function dis(β) for k = 1.

Remark 3.2. Initial bounds of β are critical for identifying the distance function dis(β), asthey affect significantly the speed of the search procedure described in Section 3. Generallyspeaking, when Q ≻ 0, such bounds on β can be obtained from the following observation. Inproblem (P), the constraint in (42),

∑ki=1 κiβi ≤ ρ, implies

∑ki=1 κiβi ≤ 2v(P) + 2C, where

v(P) is an upper bound of problem (P). In other words, we are only interested in identifying


the distance function dis(β) on some bounded domain of β. From an algorithmic point of view,we prefer box-type of bound on β. Thus, we may assume β is confined in a box ,

β ∈ [ωl, ωu] ,

β ∈ Rk | ωl

i ≤ βi ≤ ωui , i = 1, · · · , k

,

where ωl = ωli|

ni=1 and ωy = ωb

i |ni=1. An upper bound v(P) of problem (P) can be easily

found by some heuristics, e.g., from the objective value of the incumbent (the best feasiblesolution obtained).

Although Theorem 3.1 shows that the function dis(β) has at most O((n2 − n)k) pieces, inreal application, the total number of the pieces is far less than this upper bound, especially,when we add box bound on β (see Remark 3.2). In the following subsections, we focus ondeveloping an algorithm to identify the function dis(β) in a bounded domain of β, i.e., toidentify the coefficients, Dt, dt, ct, t = 1, · · · , N , for β ∈ [wl, wu]. We separate our discussionfor cases of k = 1 and k > 1.

3.1 Identification of dis(β) for k = 1

When k = 1, both H and h are vectors in Rk.

Corollary 3.1. When k = 1, the distance function dis(β) consists of at most N ≤ n(n − 1)pieces of quadratic functions.

Proof. The proof of the corollary follows Theorem 3.1. However, when k = 1, cells of thehyperplane arrangement degenerate to intervals on a real line. An upper bound of the totalnumber of cells, N , can be calculated. Clearly, the set I(β) changes only when some bi(β)intersects with bj(β), with i ∈ I(β) and j ∈ I(β). That is to say, N ≤ Sn, where Sn isthe total number of intersection points between the functions bi(β) and bj(β) for i 6= j, andi, j = 1, · · · , n. The number Sn can be computed in a recursive way. When n = 2, it holdsthat S2 = 2, and, when n > 3, the recursion Sn = Sn−1 + 2(n − 1) holds. Solving such arecursion yields Sn = (n− 1)n. Thus, theoretically, we can divide the interval [−∞,∞] into atmost N ≤ n(n− 1) consecutive intervals [Ij , Ij+1], for j = 1, · · · , N .

From Corollary 3.1, we know that N ≤ n(n−1), where notation “O” is dropped. However,it is still expensive and unnecessary to compute all N quadratic functions directly.

To identify function dis(β) with β ∈ [ωl, ωu], we partition the interval [ωl, ωu] into severalsub-intervals, where functions bi(β)|

ni=1 are linear in each of there sub-intervals. Note that

such a partition always exists. Since bi(β) is a constant when Hi = 0, we assume that Hi 6= 0for all i = 1, · · · , n. Function bi(β) achieves its minimum point at −hi/Hi with bi(β) = 0, fori = 1, · · · , n. If −hi/Hi 6∈ [ωl, ωu], then bi(β) is linear in [ωl, ωu]. Without loss of generality,we assume that points −hi/Hi|

ni=1 are arranged in an ascending order and are all in interval

[ωl, ωu],

ωl < −h1H1≤ −

h2H2≤ · · · ≤ −

hnHn

< ωu.


Clearly, in each interval of [ωl,− h1

H1], · · · , [− hn

Hn, ωu], function bi(β) is linear with respect to β.

(see Figure 3).Now we concentrate on identifying dis(β) in each sub-interval within β ∈ [βl, βu]. Our

main scheme is to sequentially check the intersection point between bi(β), i ∈ I(β) and bj(β),j ∈ I(β) from βl to βu. Once an intersection point is identified, the index set I(β) is modifiedaccordingly. More specifically, we use the following Table, T, to store the data.

T I1 I2 · · · Is

I1 T1,1 T1,2 · · · T1,s

I2 T2,1 T2,2 · · ·...

......

... · · ·...

In−s Tn−s,1 · · · · · · Tn−s,s

In table T , the first column and the first row are corresponding to the index sets Iβ and Iβ,respectively, i.e., Ii ∈ I(β), i = 1, · · · , n − s and Ii ∈ I(β) for i = 1, · · · , s. Element T(i, j)in the table stores the intersection point of bIi(β) and bIj (β). Since any two linear functionsintersect at most once, each time we only need to modify one column and one row, which leadsto a linear time operation O(n). We present formally such an algorithm in Algorithm 1 anduse the following Example 3.1 to illustrate the procedure.

Algorithm 1 Procedure for identifying dis(β) when k = 1

Input: Interval [ωl, ωu], H and hOutput: All individual pieces of function dis(β)

(1) Let βl ← ωl, βu ← ωu, flag← 1. Sort bi(βl)|ni=1 and construct I(βl) and I(βl).

(2) Initialize Table T by filling first column and first row with index set I(βl) and I(βl).For i = 1, · · · , n− s and j = 1, · · · , s, set

Ti,j =

hIi

−hIj

HIj−HIi

if βl ≤hIi

−hIj

HIj−HIi

≤ βu,

+∞ otherwise,(45)

while flag = 1 do

Construct I(βl) and identify dis(β) as given in (39) and (40). Output dis(β) with [βl, βu].if Ti,j 6=∞ for all i = 1, · · · , n− s, j = 1, · · · , s, then

Find the minimum value Ti∗,j∗ in Table T . βl ← βu, βu ← Ti∗,j∗ and Ti∗,j∗ ← ∞.Exchange index Ij∗ and Ii∗ and update j∗-th column and i∗-th row by using (45).if βu = ωu, then

flag← 0end if

else

flag← 0end if

end while


Example 3.1. We consider an example with n = 6, k = 1, s = 3 and

H =(−0.5 0.5 1 −0.75 −2 −4

)′,

h =(1 2.5 4 3.4 3.5 5.2

)′.

We want to identify function dis(β) with β ∈ [−1, 1.3].For this example, functions bi(β)

6i=1 are specified as follows (also see Figure 3),

b1(β) = | − 0.5β + 1|, b2(β) = |0.5β + 2.5|,b3(β) = |β + 4|, b4(β) = | − 0.75β + 3.4|,b5(β) = | − 2β + 3.5|, b6(β) = | − 4β + 5.2|.

Interval [−5, 5] can be decomposed as follows,

[−5, 5] = ∪[−5,−4] ∪ [−4, 1.3] ∪ [1.3, 1.75] ∪ [1.75, 2] ∪ [2, 4.53] ∪ [4.53, 5],

such that in each of the sub-intervals, all bi(β), i = 1, · · · , 6, are linear.We demonstrate our algorithm, in particular, for the interval of [−1, 1.3] in which

b1(β) = −0.5β + 1, b2(β) = 0.5β + 2.5,b3(β) = β + 4, b4(β) = −0.75β + 3.4,b5(β) = −2β + 3.5, b6(β) = −4β + 5.2.

−6 −4 −2 0 2 4 60

1

2

3

4

5

6

7

8

The functions bi(β) in [−6, 6]

−2 −1.5 −1 −0.5 0 0.5 1 1.5 20

1

2

3

4

5

6

7

8

9

The functions bi(β) in [−1, 1.3]

β

β

gi(β)

b5(β)

b6(β)b4(β)

b1(β)

b2(β)

b3(β)

b6(β)

b4(β)

b2(β)

b5(β)b3(β)

b1(β)

Figure 3: Functions bi(β) in Example 3.1

We now use Algorithm 1 to identify function dis(β) in the interval of [−1, 1.3].


• In the first step, we compute bi(−1) for i = 1, · · · , 6, initialize the index sets as I(−1) =1, 2, 3 and I(−1) = 4, 5, 6 according to the values of bi(−1)|

6i=1, and construct

Table 1, where the element in the position corresponding to bi ∈ I and bj ∈ I representstheir intersection. We find the minimum value −0.342 in Table 1, which indicates that

Table 1: Table of Step 1 in Example 3.16 4 5

1 1.200 +∞ +∞2 0.600 0.720 0.4003 0.240 −0.343 −0.167

b4(β) and b3(β) intersecting at β = −0.342. We can thus conclude that, in the sub-interval of β ∈ [−1,−0.342], the index set I(β) = 1, 2, 3 remains unchanged. Wefurther identify the first piece of function dis(β) for β ∈ [−1,−0.342] as

dis(β) = 1.5β2 + 9.5β + 23.25, β ∈ [−1,−0.342],

where the coefficients are computed according to (39) with index set I([−1,−0.342]) =1, 2, 3. Since −0.342 is the intersection point of b3(β) and b4(β), we exchange thepositions of b3 by b4 in Table 1, which leads to Table 2.


1 1.200 +∞ +∞2 0.600 +∞ 0.4004 0.554 +∞ 0.008

• The minimum value in Table 2 is 0.08 and we derive the second piece of dis(β) as

dis(β) = 1.062β2 − 36β + 18.51, β ∈ [−0.342, 0.080], (46)

where all coefficients are computed according to index set I([−0.342, 0.08]) = 1, 2, 4.Updating row 3 and column 3 in Table 2 yields Table 3.


1 1.200 +∞ +∞2 0.600 +∞ 0.7205 0.850 +∞ +∞

• The minimum value of Table 3 is 0.60 and we derive the third piece of function dis(β) as

dis(β) = 4.5β2 − 12.5β + 19.5, β ∈ [0.08, 0.60], (47)


−1 −0.5 0 0.5 10

2

4

6

8

10

12

14

16

18

20

22

4.5β2− 12.5β + 19.5

1.5β2 + 9.5β + 23

0.08−0.3429 0.6

1.06β2− 3.6β + 18.5

dis(β)

β

20β2− 56.6β + 40.3

Figure 4: Functions bi(β)|6i=1

where all coefficients are computed according to index set I([0.08, 0.6]) = 1, 2, 5. Up-dating row 2 and column 1 in Table 3 yields Table 4.


1 +∞ +∞ +∞6 +∞ +∞ +∞5 +∞ +∞ +∞

• The minimum value of Table 4 is 1.3 and we get the fourth piece of function dis(β) as

dis(β) = 20.25β2 − 56.6β + 40.29, β ∈ [0.60, 1.3]. (48)

As all elements in Table 4 are ∞, we complete the characterization of the distancefunction of dis(β) in [−1, 1.3] (See Figure 4).

3.2 Identification of dis(β) for k > 1

We know from Theorem 3.1 that the distance function dis(β) can be constructed accordingto the cells of hyperplane arrangement. Enumerating the cells of the hyperplane arrangementhas been investigated in the literature. For example, the authors in [1] and [10] proposed a cellenumeration method by reverse searching method. Such a method consumes O((n2 − n)kClp)time to enumerate all the cells, where Clp is the time for a linear programming. Note that thecells are searched in the whole R

k space when adopting the reverse searching method. In ourstudy, we are only interested in enumerating all the cells within a bounded region β ∈ [ωl, ωu],which is also ready to be solved by the algorithm in [10].


As we have illustrated in Lemma 3.1, the order of all bi(β)|ni=1 is fixed in each cell induced

by hyperplanes p1i,j and p2i,j, for i, j ∈ I. Such cells are characterized by the sign matrix sign(E)in (35). Now we specify a mapping from a sign matrix to an order of functions bi(β)|

ni=1.

Given a sign matrix sign(E) in (35), we first construct matrix Ω(E) by letting Ωii(E) = 0,copying the upper-triangle of sign(E) into its upper-triangle and inserting the opposite of theupper-triangle of sign(E) into its lower-triangle.

From (37), (36) and (33), we can conclude that bi(β) is the (t + 1)-th smallest elementamong bi(β)

ni=1 for β in cell E if there are totally t of “+” in i-th row of Ω(E), i.e., we have

(+,−,+, · · · ,−)︸︷︷︸

There are t “+”s.

←→ bi(β) is (t+ 1)-th smallest elemetnt.

Example 3.2. We illustrate how to identify function dis(β) with k > 1 by considering thefollowing example of identifying function dis(β) with k = 2, n = 4, s = 2 and parameters setas follows,

H =

4 25 11 − 12 − 0.5

, h =

−2−6−12

.

Assume that β = (β1, β2)′ is confined in the box of −1 ≤ β1 ≤ 4 and −1.5 ≤ β2 ≤ 1. According

to (31) and (32), we introduce the following hyperplanes,

p11,2 : z11,2(β) = −β1 + β2 + 4 = 0, p21,2 : z21,2(β) = 9β1 + 3β2 − 8 = 0,

p11,3 : z11,3(β) = 3β1 + 3β2 − 1 = 0, p21,3 : z21,3(β) = 5β1 + β2 − 3 = 0,

p11,4 : z11,4(β) = 2β1 + 2.5β2 − 4 = 0, p21,4 : z21,4(β) = 6β1 + 1.5β2 = 0,

p12,3 : z12,3(β) = 4β1 + 2β2 − 5 = 0, p22,3 : z22,3(β) = 6β1 − 7 = 0,

p12,4 : z12,4(β) = 3β1 + 1.5β2 − 8 = 0, p22,4 : z22,4(β) = 7β1 + 0.5β2 − 4 = 0,

p13,4 : z13,4(β) = −β1 − 0.5β2 − 3 = 0, p23,4 : z23,4(β) = 3β1 − 1.5β2 + 1 = 0.

It can be verified that the region specified by −1 ≤ β1 ≤ 4 and −1.5 ≤ β2 ≤ 1 is on one sideof the following hyperplane,

z21,3(β) > 0, z21,4(β) > 0, z22,4(β) > 0, z13,4(β) < 0, z23,4(β) > 0.

Thus, for any β in this region, it holds true that ω21,3 = +, ω2

1,4 = +, ω22,4 = +, ω1

3,4 = −,

ω23,4 = +. We can enumerate the cells of the arrangements generated by these hyperplanes in

the box region, β1 ∈ [1, 4] and β2 ∈ [−1.5, 1]. Implementing the algorithm of cell enumeration[10] generates 15 cells. While the sign vectors of these 15 cells are listed in Table 5, all thehyperplanes and their arrangement are illustrated in Figure 5, in which the arrow indicatesthe positive side of the hyperplane. We can then figure out the order of the bi(β)

4i=1 in each

cell. For example, let us consider cell2 in Table 5 with

sign(cell2) =

(− +) (+ +) (+ +)(+ +) (+ +)

(− +)

.


Then we construct matrix Ω(cell2) from the sign matrix sign(cell2) as

Ω(cell2) =

0 − + ++ 0 + +− − 0 −− − + 0

.

which yields an order of bi(β), i.e., b3(β) < b4(β) < b1(β) < b2(β). Once the order of bi(β) isachieved, the distance function can be expressed by applying (39) and (40),

dis(β) = 5β21 + 1.25β2

2 − 8β1β2 + 6β1 + 5.

All the other pieces of the distance function can be derived in a similar fashion.

Table 5: The cells of hyperplanes in Example 3.2No (w1,2, w1,3, w1,4), (w2,3, w2,4), (w3,4)

1 (++,++,++), (++,++), (−+)2 (−+,++,++), (++,++), (−+)3 (−+,++,−+), (++,++), (−+)4 (−+,++,−+), (++,+−), (−+)5 (++,++,−+), (++,+−), (−+)6 (++,++,−+), (−+,+−), (−+)7 (++,+−,−+), (−+,+−), (−+)8 (+−,+−,−+), (−+,+−), (−+)9 (+−,+−,−+), (−−,+−), (−+)10 (+−,−−,−+), (−−,+−), (−+)11 (+−,−−,−+), (−−,++), (−+)12 (+−,−−,−+), (−+,++), (−+)13 (+−,−−,−+), (+−,++), (−+)14 (+−,−−,++), (+−,++), (−+)15 (+−,−−,++), (++,++), (−+)

4 Illustrative Example

We demonstrate in this section a complete implementation of our solution scheme developedin this paper via an illustrative example.


1 1.5 2 2.5 3 3.5 4−1.5

−1

−0.5

0

0.5

1

β1

β2

cell2

cell4

cell5

cell15

z124

(β)

z223

(β)

z212

(β)

z114

(β)

z112

(β)

cell1

z123

(β)

cell3

((−+)(++)(++), (++)(++), (−+))z113

(β)

Figure 5: The cells of hyperplane arrangement in Example 3.2

Example 4.1. Let us consider an example of problem (P) with n = 6, s = 2 and

Q =

27.171 −5.738 2.479 −2.768 4.931 1.725−5.738 18.358 5.030 −5.615 10.004 3.5002.479 5.030 27.827 2.426 −4.322 −1.512−2.768 −5.615 2.426 27.292 4.825 1.6884.931 10.004 −4.322 4.825 21.404 −3.0071.725 3.500 −1.512 1.688 −3.007 28.948

,

q =(37.745 −26.329 −80.284 34.905 7.296 −51.002

)′.

It can be verified that Q is of a k = 1- freedom with one eigenvalue of λ1 = 1 and fiveeigenvalues of λ2 = 30. For this simple example, adoption of an enumeration method identifiesits optimal solution of x∗ = (0, 0, 2.989, 0, 0, 1.918)′ and the corresponding optimal value ofv(P) = −168.91.

We compute H and h defined (26) as

H = ( 0.312, 0.634, − 0.274, 0.306, − 0.544, − 0.190 )′ ,

h = ( − 11.369, − 19.636, 11.539, − 11.057, 17.384, 7.867)′ .

Using the parameters defined in Section 2.2, we have κ1 = 1.0701, ι1 = 1.0344 and C =−749.43, which are defined in (20), (21) and (5), respectively. Since k = 1, we identify thedistance function dis(β) as follows in the interval [0, 37] by using the method discussed in


Section 3.1,

dis(β) =

9.063β2 − 695.19β + 13396.41, I(β) = 2, 5, β ∈ [0, 21.59],15.71β2 − 1073.53β + 18467.86, I(β) = 2, 3, β ∈ [21.59, 25.26],18.86β2 − 1252.13β + 20969.04, I(β) = 3, 5, β ∈ [25.26, 25.91],24.83β2 − 1606.95β + 26156.91, I(β) = 3, 1, β ∈ [25.91, 28.75],26.66β2 − 1730.09β + 28178.20, I(β) = 3, 6, β ∈ [28.75, 33.37],15.71β2 − 1073.53β + 18467.85, I(β) = 2, 3, β ∈ [33.37, 35.34],9.06β2 − 695.19β + 13396.41, I(β) = 2, 5, β ∈ [35.34, 36.60].

(49)

Note that function dis(β) consists of total N = 7 pieces of convex quadratic functions. Wecan now explicitly write out the left hand of constraint (43) for each of sub-problem (Pt),t = 1, · · · , 7,

λ2dis(β1) + ι1β21 =

g1(β) = 10.10β2 − 695.19β + 13396.41 β ∈ [0, 21.59],g2(β) = 16.74β2 − 1073.53β + 18467.86 β ∈ [21.59, 25.26],g3(β) = 19.89β2 − 1252.13β + 20969.04 β ∈ [25.26, 25.91],g4(β) = 25.86β2 − 1606.95β + 26156.91 β ∈ [25.91, 28.75],g5(β) = 27.70β2 − 1730.09β + 28178.20 β ∈ [28.75, 33.37],g6(β) = 16.74β2 − 1073.53β + 18467.85 β ∈ [33.37, 35.34],g7(β) = 10.10β2 − 695.19β + 13396.41 β ∈ [35.34, 36.60].

On the other hand, constraint (44) keeps the same form, g(β) = 1.07β2 ≤ ρ. The sub-problem(Pt), t = 1, · · · , 7, becomes

(Pt) : minβ,ρ

1

2ρ+ C | gt(β) ≤ ρ, g(β) ≤ ρ

.

We plot all the constraints in Figure 6. Solving all these sub-problems, we find that the optimalvalue is achieved in interval [25.91, 28.75] with index set I(β) = 3, 1, the optimal ρ∗ equalto 1161.053 and the optimal value v(P) = 1

2ρ∗ +C = −168.91. From Theorem 2.2, we further

get optimal solution x∗3 = 2.989, x∗6 = 1.918, x∗1 = 0, x∗2 = 0, x∗4 = 0, x∗5 = 0.

5 Conclusions

We have investigated a class of CCQO problems with their coefficient matrices being of ak-degree freedom. We have demonstrated that we can decompose such a CCQO probleminto several quadratic convex subproblems, where the total number of the subproblems isdetermined by the number of the cells generated by some R

k dimensional hyperplanes withina bounded region. It is interesting to note that such a number is bounded by O(n2k). Inparticular, for the case with k = 1, we have developed an efficient method to identify all thesubproblems. For cases with k ≥ 2, by modifying the reverse searching method proposed in [1],we have also developed an efficient algorithm to identify the subproblems. Once subproblemsare known, we can solve each subproblem individually and find the solution of the originalCCQO problem.


20 22 24 26 28 30 32 34 360

0.5

1

1.5

2

2.5

3

3.5

4

25.26 25.91 28.75 33.37 35.35

g2(β)

g3(β)

g4(β)

g5(β)g6(β)

g7(β)

21.59

β

ρ × 10−3

ρ∗ = 1.161

β∗ = 31.23

Figure 6: Functions gi(β)|6i=1 and g(β) of Example 4.1

References

[1] D. Avis and K. Fukuda. Reverse search for enumeration. Discrete Applied Mathematics,65:21–46, 1996.

[2] D. Bertsimas and R. Shioda. Algorithm for cardinality-constrained quadratic optimiza-tion. Computational Optimization and Applications, 2007.

[3] D. Bienstock. A computational study of a family of mixed-integer quadratic programmingproblems. Mathematical Programming, 74:121–140, 1996.

[4] E. J. Candes, J. Romberg, and T. Tao. Robust uncertainty principles: exact signal recon-struction from highly incomplete freguency information. IEEE Transcation on InformationTheory, 52(2):489–509, 2006.

[5] A. Das and D. Kempe. Algorithms for subset selection in linear regression. In Proceedingsof the 40th annual ACM symposium on Theory of computing, pages 45–54, 2008.

[6] D. L. Donoho. Compressed sensing. IEEE Transaction Information Theory, 52(4):1289,2006.

[7] M. R. Garey and D. S. Johnson. Computer and Intractability, A Guide to The Thoery ofNP-Completeness. W. H. Freeman Co, Francisco, 1979.

[8] T. M. Liebling E. Steiner K. Allemand, K. Fukuda. A polynomial case of unconstrainedzero-one quadratic optimization. Mathematical Programming, 91:49–52, 2001.

[9] D.X. Shawa, S. Liub, and L. Kopmanb. Lagrangian relaxation procedure for cardinality-constrained portfolio optimization. Optimization Methods and Software, 23:411–420, 2008.


[10] N. Sleumer. Output-sensitive cell enumeration in hyperplane arrangements. Nordic journalof computing, 6:137–161, 1999.

[11] W.J. Welch. Algorithm complexity: three np-hard problems in computational statistics.Journal of Statistical Computation and Simulation, 15:17–25, 1982.

[12] J. Xie, S. He, and S. Zhang. Randomized portfolio selection with constraints. PacificJournal of Optimization, 4:89–112, 2008.

ccqo poly regularknowledge, only a few results on polynomially solvable cases of ccqo problem are...

Documents