7/27/2019 05089058

1/4

A Fast Algorithm for Linearly Constrained Quadratic Programming Problems

with Lower and Upper Bounds

Yanwu LIU

School of Management,

Wuhan University of Technology,

Wuhan, P. R. China,430070,

Email: liuyanwu8818@126.com

Zhongzhen ZHANG

School of Management,

Wuhan University of Technology,

Wuhan, P. R. China,430070,

Email: zhangzz321@126.com

AbstractThere are many applications related to linearly

constrained quadratic programs subjected to upper and

lower bounds. Lower bounds and upper bounds are treated

as different constraints by common quadratic programming

algorithms. These traditional treatments significantly

increase the computation of quadratic programming

problems. We employ pivoting algorithm to solve quadraticprogramming models. The algorithm can convert the

quadratic programming with upper and lower bounds into

quadratic programming with upper or lower bounds

equivalently by making full use of the Karush-Kuhn-Tucker

(KKT) conditions of the problem and decrease the

computation. The algorithm can further decrease

calculation to obtain solution of quadratic programming

problems by solving a smaller linear inequality system

which is the linear part of KKT conditions for the quadratic

programming problems and is equivalent to the KKT

conditions while maintaining complementarity conditions of

the KKT conditions to hold.

Keywords-pivoting algorithm; Karush-Kuhn-Tuckerconditions; quadratic programming; lower and upper

bounds

I. INTRODUCTIONDue to its rich variety of applications, quadratic

programming problems play a core role in nonlinearprogramming problems[1, 2]. Dai and Fletch present annew projected gradient algorithm to solve such problems,

but their algorithm only consider the case where theproblems are subjected to one equality constraint[3]. Luand Wei employ a decomposition method for quadratic

programming problem with box constraints[4]. Theirmethod is similar to the iterative method for solving linearequation system. In [5], Madsen, Nielsen, and Pinardescribe an algorithm to solve bound constrainedquadratic programming via piecewise quadratic functions.Dostal and Ostrava propose a semimonotonic augmentedLagrangian algorithm for solving large convex bound andequality constrained quadratic programming problems[6],In this paper, we establish the quadratic programmingmodel that includes general linear equality and inequalityconstraints and lower and upper bounds. By making fulluse of relations in the Karush-Kuhn-Tucker (KKT)conditions for the lower bound and the upper bounds, QP

problems with upper and lower bounds can be converted

into corresponding QP problems with either upper boundsor lower bounds equivalently. The pivoting algorithm canapply these relations to simplify the calculation of QP

problem with upper and lower bounds conveniently.Furthermore, the pivoting algorithm obtain the solution ofthe problems by solving a smaller inequality system whichis the linear part of KKT conditions of the problems whilemaintaining the complementarity conditions of KKTconditions of the dual problem to hold.

This paper is organized as follows. We give thegeneral form of QP problems with lower and upper

bounds in Section II. Section III proves important relationsof KKT conditions for QP problem with upper and lower

bounds. Section IV emphasizes the methodology of thepivoting algorithm for QP problems and presents the stepsof pivoting algorithm to solve QP problem with upper andlower bounds by using those relations above. Wedemonstrate the pivoting algorithm to solve an example inSection V. Section VI summarizes the paper.

II. GENERAL FORM OF QP PROBLEMSIn this paper we consider the following quadratic

programming problemminimize xcHxxxf TT += 2/1)(

subject to iT

i bxa = , li ,,2,1 "= , (1)

i

T

i bxa , mlli ,,2,1 "++= ,

iii uxl , .,,2,1 ni "=

wherennRH is symmetric positive definite

matrix, ),,2,1( niai "= and c are vectors in Rn,

iii ulb ,, ),2,1( ni "= are scalars, ii ul <

),,2,1( ni "=

III. AN IMPORTANT CHARACTERISTIC OF KKTCONDITIONS OF THE MODEL

The KKT conditions for (1) are

= =

=++

n

j

m

j

iiijjijij caxh1 1

0 ,

ni ,,2,1 "= ,

,0)(,,0 = iT

iii

T

ii babxa

,,,2,1 mlli "++= (2)

0i , ii ux , 0)( =+ iii ux ,

,,,2,1 ni"

=

2008 International Conference on MultiMedia and Information Technology

978-0-7695-3556-2/08 $25.00 2008 IEEE

DOI 10.1109/MMIT.2008.97

58

7/27/2019 05089058

2/4

.,,2,1, liba iT

i "==

where i , i , i are Lagrangian multipliers.Theorem 1 The inequality system (4) is equivalent to

the following inequality system

= = =+n

j

m

j

iijjijij caxh1 1

, ii lx ,

0i , ,0)( = iii lx ii ux , 1Ni ;

= =

=+

n

j

m

j

iijjijij caxh1 1

, ii ux ,

0i , ,0)( =+ iii ux ii lx , 2Ni ; (3)

,0)(,,0 = iT

iii

T

ii babxa

,,,2,1 mlli "++=

.,,2,1, liba iT

i "==

whereN1 andN2 are the index sets ofx,N1 N2 = {1,, n},N1N2 = .

Proof: LetX1,X

2be feasible solution set for inequality

system (2) and (3) respectively. There are two cases to beconsidered.

Case 1. Let x be any feasible solution for inequalitysystem (3). That is, xX

2. If set i = 0 (iN1) and i =

0(iN2) in the inequality system (2), then it is obviousthatx is also a feasible solution for the inequality system(2). That is, xX

1. That means thatX

2 X

1.

Case 2. Let x be any feasible solution for inequalitysystem (2). That is,xX

1. The value ofxi can be equal to

li, orui, or neither li norui at any time. Ifxiis equal to li,

thenxi < ui and i = 0; Ifxi is equal to ui, then xi > uiandi= 0; Ifli

7/27/2019 05089058

3/4

Theorem 2 The upper bound inequality xiuimusthold when the lower bounds inequality xi li are basicinequality and vice versa. Deviation of xi ui plusdeviation of xi li is ui li when the upper boundinequality and the lower bound inequality are bothnon-basic inequalities.

Proof: Ifxil

iis basic inequality, thenx

i= l

iaccording

to the definition of basic inequality. Since li< ui andxi = li,xi> ui. Ifxiui is basic inequality, thenxi = ui. Sinceli< ui and xi = ui, xi > li. According to the deviation ofnon-basic inequality, deviation ofxiui plus deviationofxi liis

(xi+ ui) + (xili) = uili (6)Theorem 2 reveals that we need consider only one of

the couple inequalities. When one of couple inequalities isbasic inequality, the other one must hold and is notnecessary to consider. When the couple inequalities are

both non-basic inequalities, we need information of only

one inequality and can obtain the information aboutwhether the other inequality holds directly from (6).

B. Algorithmic StepsStep 1: Initialization. Set up the initial table as shown

by TABLE I for the basic system ofx1 0,x2 0, ,xn0, 1 0, , l 0, l+1M , , mM(M is largeenough positive number).

Obviously, x1u1, , xnun must hold and arenot necessary to consider. There are two deviationcolumns in table I. The first deviation column shows thedeviation of the non-basic inequalities, and the seconddeviation column records the deviation of the otherinequality which does not appear in the table directlywhen some pairs of the upper bound inequalities and thelower bound inequalities are both non-basic and only oneinequality of any pair appears in the table directly.

TABLE I. INITIAL TABLE

e1 en en+1 en+m i i

h1 h11 h1n a11 am1 1

hn hn1 hnn a1m amm n

a1 a11 a1m 0 0 n+1

am am1 amm 0 0 m+n

In TABLE I, sincex1 0,x2 0, ,xn 0, 1 0, ,l 0, l+1 M , , m M are basic system, theircorresponding coefficient vectors e1, e2, , en, , en+m areunit vectors in R

m+nand current basis and their

complementary inequalities whose corresponding

coefficient vectors are h1, h2, , hn, a1, , am are vectorsinRm+n

and non-basic inequalities.Step 2: Preprocessing. Convert non-basic equalities

into basic qualities. Since equalities must hold, they aremaintained in the basic system once they are changedfrom non-basic equalities to basic equalities. After

preprocessing, iM (i = l, l + 1, , m) is non-basicinequality and the value ofi does not affect the basicsolution. In order to calculate conveniently, we set Mequal to zero.

Step 3: Main iterations by the smallest deviation rule.(1) If all the deviations of non-basic vectors are

nonnegative, stop. Otherwise, go to step (2).(2) Select a vector with the most negative deviation to

enter the basis. If this vector is not listed in the table,conduct a vector substitution. If the diagonal entry in thatrow is positive, carry out a pivoting on that entry andreturn to (1); otherwise carry out a pivoting on the most

positive entry in that row and then carry out a pivoting onthe symmetrical entry, return to (1).

Vector substitution is the process that the vectors ofone of couple inequalities and its complementaryinequality which do not appear in the table directlysubstitute the vectors of the other of couple inequalitiesand its complementary inequality which appear in the

table directly. It is easily proven that values of theelements in the corresponding low and column isunchangeable but their signs are inverse, other elementskeep unchangeable, two deviation in the correspondingrow are swapped, and other deviations keep unchangeable

after a vector substitution.

V. AN EXAMPLEIn this part, we demonstrate the pivoting algorithm to

solve the following example.

minimize xcHxx TT +2/1

subject to 22 21 = xx ,

52 321 + xxx ,

60 1 x ,

40 2 x ,

40 3 x ,

Where

=

201

021

111

H , c = (1.25, 1, 3).

Let h1 = (1, 1, 1, 1, 1), h2 = (1, 2, 0, 2, 1), h3 =(1, 0, 2, 0, 2), a1 = (1, 2, 0, 0, 0), a2 =(1, 1, 2, 0, 0), e1= (1, 0, 0, 0, 0), e2 = (0, 1, 0, 0, 0), e3 = (0, 0, 1, 0, 0), e4 =(0, 0, 0, 1, 0), e5 = (0, 0, 0, 0, 1).

The initial table of the example above is as follow.

60

7/27/2019 05089058

4/4

TABLE II. INITIAL TABLE

e1 e2 e3 e4 e5 i i

h1 1.000 1.000 1.000 1.000 1.000 1.2500

h2 1.000 2.000 0.000 2.000 1.000 1.0000

h3 1.000 0.000 2.000 0.000 2.000 3.0000

a1 1.000 2.000 0.000 0.000 0.000 2.0000

a2 1.000 1.000 2.000 0.000 0.000 5.0000

TABLE III. THE RESULT OF PREPROCESSING

e1 a1 e3 h2 e5 i i

h1 0.500 0.000 1.000 0.500 0.500 1.7500

e4 0.000 0.500 0.000 0.500 0.500 0.5000

h3 1.000 0.000 2.000 0.000 2.000 3.0000

e2 0.500 0.500 0.000 0.000 0.000 1.0000

a2 0.500 0.500 2.000 0.000 0.000 6.0000 11.0000

TABLE IV. THE RESULT OF THE FIRST PIVOTING

e1 a1 a2 h2 h3 i i

h1 1.125 0.375 0.750 0.500 0.250 7.0000

e4 0.375 0.625 0.250 0.500 0.250 2.7500

e5 0.750 0.250 0.500 0.000 0.500 4.5000

e2 0.500 0.500 0.000 0.000 0.000 1.0000 3.0000

e3 0.250 0.250 0.500 0.000 0.000 3.0000 1.0000

TABLE V. THE RESULT OF THE SECOND PIVOTING

h1 a1 a2 h2 h3 i i

e1 0.889 0.333 0.667 0.444 0.222 6.2222 0.2222

e4 0.333 0.500 0.000 0.667 0.167 0.4167

e5 0.667 0.000 0.000 0.333 0.333 0.1667

e2 0.444 0.667 0.333 0.222 0.111 4.1111 0.1111

e3 0.222 0.167 0.333 0.111 0.056 1.4444 2.5556

TABLE VI. THEFINALRESULT

e1 a1 a2 h2 h3 i i

h1 1.125 0.375 0.750 0.500 0.250 0.2500

e4 0.375 0.625 0.250 0.500 0.250 0.5000

e5 0.750 0.250 0.500 0.000 0.500 0.0000 e2 0.500 0.500 0.000 0.000 0.000 4.0000 0.0000

e3 0.250 0.250 0.500 0.000 0.000 1.5000 2.5000

From TABLE VI, the deviations of all non-basicinequalities satisfy the optimal conditions, so the currentsolution is the optimal solution: x1=6,x2 = 4, x3 = 1.5, theoptimal objective function value is 3.75.

VI. SUMMARYWe introduce pivoting algorithm to solve QP problems

with upper and lower bounds. We prove an important

theorem which guarantees that we need consider only one

of the two inequalities explicitly in any case. That means

QP problem with upper and lower bounds can be

converted into QP with either upper bounds or lower

bounds. This conversation decreases calculation caused

by considering upper and lower bounds simultaneously.The linear part of KKT conditions of QP problem is

equivalent to the whole KKT conditions while the

complementarity conditions hold. The pivoting algorithm

can realize the goal above by making sure that one of any

pair of complementary inequalities is basic and the other

is non-basic. Therefore, the pivoting algorithm really

increases the efficiency of calculation of original problem

immensely by solving a much smaller size linear

inequality system which is equivalent to QP problem withupper and lower bounds under certain conditions.

REFERENCES

[1] D. Hush, P. Kelly, C Scovel, and I. Steinwart, QP algorithm withguaranteed accuracy and run time for support vector machines,Journal of Machine Learning Research, 2007, 7(12), pp. 733769.

[2] M. J. Best, and J. Hlouskova. An algorithm for portfoliooptimization with transaction costs. Management Science, 2005,51(11): 1676-1688.

[3] Y H Dai, and R Fletch. New algorithms for singly linearlyconstrained quadratic programs subjected to lower and upperbounds. Mathematical Programming, 2006, 403421.

[4] Z. J. Lu, and Z. L. Wei. Decomposition method for quadraticprogramming problem with box constraints. MathematicaNumberica Sinical, 1999, 21(4): 475482.

[5] K. Madsen, H. B. Nielsen, and M C Pinar. Bound constrainedquadratic programming via piecewise quadratic functions.Mathematical Programming, 1999, 85: 135156.

[6] Z. Dostal, and Ostrava. An optimal algorithm for bound andequality constrained quadratic programming problems with boundspectrum. Computing, 2006, 78: 311328.

[7] Z. Z. Zhang, Quadratic programming, Wuhan: WuhanUniversity Press, 2006.

61