Comput. Opns Res. Vol. 14, No. 6, pp. 449456, 1987 0305-0548/X7 $3.00+ 0.00

Printed in Great Britain. All rights reserved Copyright 0 1987 Pergamon Journals Ltd

A COMPUTATIONAL COMPARISON OF GOMORY AND KNAPSACK CUTS

LARRY JENKINS* and DAVE PETERS?

Department of Engineering Management, Royal Military College of Canada, Kingston, Ontario K7K 5L0, Canada

(Received October 1986; revised February 1987)

Scope and Purpose-We consider solving integer programs by a variety of Gomory cutting planes and by the recently developed method of knapsack cuts. By examining the performance of each method on a large number of test problems, we determine which method is best for general integer and for O-l problems.

Abstract-A common method of solving integer programs is to solve the problem first as a linear program (LP) then add constraints that cut off noninteger solutions from the set of LP feasible solutions. As soon as an optimal LP solution is all integer, then it is an optimal solution to the integer program. The method of Gomory can generate a variety of different cuts but there is a dearth of reports on systematic testing of the effectiveness ofdifferent cuts. We report extensivecomputational comparisons between a number ofdiNerent cuts, including a successful one not previously publicised. It has been known for some time that Gomory cuts can be unsuccessful because of slow convergence with the accompanying difficulties of computer round-off error. Recently a method has been proposed for generating, for cl integer problems, cuts that are usually tighter than Gomory cuts and thus give faster convergence. This method of knapsack cuts is tested in comparison with Gomory cuts for moderate size problems and is found to be superior for O-l problems having dense constraint matrices but only slightly better than Gomory cuts for problems with sparse matrices. On the other hand, knapsack cuts applied to general integer problems reformulated as O-l are found to be less successful than Gomory cuts applied to the original integer problem.

1. INTRODUCTION

Consider the pure integer linear program:

maximize j$, c.ix.i

”

subject to C aijXj 6 bi, i=l,...,m, j=l

Xj20 and integer j= l,...,n. (IP)

This coincides with the standard definition of a linear program (LP) with the added requirement that, in any feasible solution to IP, all decision variables xj must be integer. Define IP with all integer constraints relaxed as the original relaxation LP” of IP. If an optimal solution of LP” is all integer, then that solution is also an optimal solution to IP.

It will be convenient to talk of the convex hull, conv(IP) which is the smallest convex space that contains all feasible integer solutions to IP. A plane in n dimensions that forms a face of conv(IP) is called a.facet. Conv(IP) is of great interest because the optimal solution(s) of IP occur at a vertex (vertices) of conv(IP) so that, if conv(IP) could be identified, solving IP would be reduced to solving an LP with the objective function of IP and conv(IP) as the solution space.

Unless IP has a special structure it is not possible, with present knowledge, to define all the facets of conv(IP) from the constraint set of IP. Nevertheless, it is possible to define planes that cut off part of the space that is feasible for LP” but not for IP. This was first exploited extensively by Gomory [l]

*Larry Jenkins is an Associate Professor in the Department of Engineering Management, Royal Military College of Canada, Kingston. He holds an M.A. in Natural Science from Oxford, an M.B.A. from McGill and a Ph.D. in Industrial Engineering from the University of Toronto. He has worked as a research chemist, an industrial engineer, a systems analyst and as a financial analyst, as well as consulting to Ontario Ministry of the Environment and Transport Canada. His current research interests are in the areas of parametric integer programming and design of information systems.

tDave Peters was a Research Associate in the Department of Engineering Management, Royal Military College of Canada, Kingston. He has a B.Sc. in Mathematics from the University ofvictoria, and a M.Math. in Combinatorics and Optimization from the University of Waterloo. He is currently developing computer software for paint blending. CAOR 14:6-I)

449

450 LARRY JENKINS and DAVE PETERS

into a procedure for solving IP. Gomory showed how to derive, from the optimal basic solution to an LP relaxation of IP, inequalities that would cut off part of the LP solution space, including the current optimal basic LP solution, but would not cut off any feasible integer solution of IP. The total procedure consists of adding a Gomory cut to the current LPk that has a noninteger solution to form LPktl and re-optimizing. This is repeated until an integer solution is obtained.

Gomory’s method can generate a variety of possible cuts, some of which usually yield the optimal integer solution more quickly than others, though there is a dearth of reports on extensive testing of the eff~tiveness of the different cuts. The only well known study is that of Trauth and Woolsey [2J nearly 20 years ago working with rather small, very degenerate problems. Part of the goal of this study is to present comparative results on a large number of randomly generated problems of moderate size. Although Gomory proved that with careful selection of the cutting planes his algorithm would always converge, convergence in practical terms has often been found to be painfully slow.

Recently, for problems where the decision variables are constraint to be 0 or 1, workers have had considerable success in defining cuts that are facets of the IP. This approach was used very effectively in the work reported by Crowder et al. [3]. In this paper we compare their method with a variety of different Gomory cuts.

Unlike Gomory cutting planes where convergence to a global optimal is theoretically guaranteed (though the number of iterations may be inordinately large) the m~hanism of Crowder et ~2. for creating knapsack cuts does not always guarantee convergence, so that it may be necessary to use some method such as Gomory cutting planes or branch-and-bound to complete convergence to the integer optimum.

The next section defines the various types of Gomory cuts considered, then Section 3 describes forming knapsack cuts. Section 4 describes briefly the implementation and Section 5 talks of the test problems. Computations results are given in Section 6.

2. GOMORY CUTS

A Gomory cut is B constraint inequality defining a plane in n-space that cuts off from the feasible region of LPk a vertex that represents a basic feasible solution of LPk but is not integer. The plane is such that it will not cut off any feasible integer solutions to IP. While a Gomory cut can be formed from any basic noninteger solution to LPk, it is standard to work with an optimal solution to LPk, based on the reasoning that this is the LP vertex most likely to be close to an optimal vertex of conv(IP), that is, an optimal solution to IP.

Consider a basic solution to an LP relaxation of IP where

xai + C aijxj = bf, i= l...m jsNB

represents the constraints updated by simplex pivots. Then the basic solution is XBi = b,! for i= . ..m.x 1 = 0 for j E NB, where Xgi is the basic variable in the ith row and NB indicates nonbasic, For any row in which b; is not integer, a Gomory cut is defined by

,z, (ha;j - [hc&])~~ 2 hb; - [hb;], i =I 1. . . m,

where [ ] represents the integer part of an expression [4]. Different cuts are derived according to the source row (which i) and the value of k. It is desirable to

develop at each iteration a cut that removes as much of the noninteger solution space of LPk as possible. Unfortunately it is rare that the space cut off by onb selection of i and h will be completely contained within the space cut off by a different selection of i and h (see Taha [5] for a discussion of this). In practice it is necessary to choose a heuristic rule for selecting i and h and then see how well this rule performs on a large number of test problems.

In this study a number of different Gomory cuts are examined, six of them having h = 1 and selecting the row i that best satisfies the heuristic criterion considered, and five of them varying h as

A computational comparison of Gomory and knapsack cuts 451

well as i to maximize the criterion. We examine only integer values of h and in this case all possible cuts can be generated from a given row by values h = 1 through h = D - 1 where D represents the absolute value of the determinant of the current basis [4].

In all cases the rows are scanned lexicographically. If two or more rows give the same maximum value for the criterion, a cut is formed from the first of those rows. If h is varied and two or more values of h give the same maximum value for the criterion, then the first such value of h is used.

The different cuts are as follows: First fraction (FF). Form a cut with h = 1 from the first row where bi - [bi] > 0. ~uxi~~~ suction (MF). Form a cut with h = 1 from the first row that has maxi{& - [bi]). Maximum fruction varying h (MFH). Form a cut with the integer value of h and row i that gives

IIlaX,~{hb~ - [hbj]}.

Maximum ratio (MI?). Form a cut with h = 1 from the first row that has

maxi((h - [4]>/Cj (@ij - [“ij]>]f*

Muximum ratio varying h (MRH). Form a cut with the integer value of h and row i that gives

maxi,h{WJf - [Q])l[&(Wj - [ha;il)]l.

Maximum penalty (MP). Form a cut with h = 1 from the row that gives maxi mini{+& - [b~])/(Ub - [U~j])) w h ere c; denotes the reduced cost in the current basis. It is the unit reduction in the value of the objective function by bringing variable xj into the basis while minj{(& - [bi])/(a; - [U;])> is th e maximum extent to which xj can be brought into the basis on a single pivoting operation.

Maximum penalty vurying h (MPH). Form a cut with the integer value of h and row i that gives maXi,h minj~C~(~~~ - ~~~~~)/(~U~j - [hUij],l)).

Maximum distance (MD). Form a cut with h = 1 from the row that gives maxi = ((bf - [&I)/ JC (Uli - [Urj])“}. Th e maximization is on the perpendicular distance from the current basic solution to the generated cut.

Muxim~m ~istu~ce vuryi~g h (ADS). Form a cut with the integer value of h and row i that gives max,,,f(hbi - [h&)]/Jc (haij - [hU~j])>.

First fraction is the simplest rule for computational convenience. Maximum fraction is an attempt to find easily a constraint that will cut deeply into the solution space, while maximum ratio extends the approach of maximum fraction by relating the ratio of the fractional parts of the right hand side to the fractional parts of the left hand side. Maximum penalty attempts to maximize the diminution in the value of the LP solution at each iteration on the theory that this will yield a deep cut. Maximum. distance is the Euclidean distance from the current solution to the added cut.

3. KNAPSACK CUTS

Recently a number of researchers, for example Balas [6], have explored the mathematics of defining the facets of the O-l knapsack problem:

maximize j$1 ‘jxj

subject to i ujxj < b, xj=o, l,j= l)‘.., n. (KP) j=l

Considerable progress has been made toward determining, from the single knapsack constraint, a number of the constraint inequalities called minimal cover cuts that form the faces of conv(KP).

If IP has O-l variables, then each individual constraint is a knapsack constraint. Now any minims cover cut that can be developed from a knapsack constraint of IP must be valid for the original problem IP. Further, if there is little interaction between the constraints, as might happen if for example, the matrix A is sparse, then many of the minimal cover cuts will be faces of IP. Thus appending knapsack cuts to the constraint set of IP may lead to easy identi~cation of the optimal solution of IP.

A large number of cuts may be generated from each KP constraint, so that it is impractical to append them all to LP”. Instead the approach used is to solve LPk and, if 2’, the optimal solution to

452 LARRY JENKINS and DAVE PETERS

LPk, is not integer, attempt to identify for each row the minimal cover cut most violated by 2 and append this constraint to problem LP“. Thus in a single iteration from LPk to LPkf ’ may cuts may be added, each generated from a different constraint row. The method for identifying the most violated minimal cover cut for each row of LPk was taken from [3] to which the reader is referred for details of both the theory and the method of implementation.

In those cases where this approach faiied to generate more facets for a noninteger solution X’, convergence to the integer optimum was completed using Gomory cutting planes generated by the maximum ratio rule.

4. IMPLEMENTATION*

The added cuts were formulated in terms of the original decision variables [7] and no previously added cuts were deleted during the course of the procedure. The program with which we tested the various algorithms was developed by extending LINDO [S] with user-routines written in FORTRAN-77. For the purpose of testing knapsack cuts on general integer problems a subroutine was written for automatically reformulating integer problems in terms of O-l variables using the transformation X~ = 8xja + 4Xj4 -I- 2xj, + Xjl where xj8 represents 8 units Of Xi and SO on; xj can then have any value up to 15, All problems were run in double precision a~thmetic on a Honeywell DPS- 8/62 computer running under CP-6 operating system.

5. TEST PROBLEMS

Test problems were generated using the random problem generator RIP [9] that can generate either O-l or general integer problems. All coelficients U.ij and bj are integer but the Cj coefficients are rational fractions converted to their double precision arithmetic representation, The user can control the number of variables (columns) m, the number of constraint rows n and the density of the A-matrix. Roughly speaking, one expects the di~culty in solving an IP to increase as these parameters increase. A random number stream permits generation of many different problems with the same parameter setting.

6. COMPUTATIONAL RESULTS

Our reports of test results give the average number of Gomory cuts or knapsack facets required to move from the optimal solution of LP” to the optimal solution of IP. We found that for a particular size of problem IP, as measured by the dimensions of the A-matrix, the number of simplex iterations was approximately a linear function of the number of cuts and the CPU time was approximately linear with the number of simplex iterations.

In our tests we ran approx. 7000 problems varying in size from 10 variables and 6 constraints up to 100 variables and 100 constraints. Those cuts that had poor performance on the smaller problems were not tried on larger problems. Some typical results for all cuts with the smaller problems are shown in Tables 1 and 2. Results are for batches of 50 problems having the characteristics specified in the first three columns. Within these specifications, random values were generated for the elements of A, b and c.

Table 1 gives results for some small O-l problems. In our implementation of the Gomory algorithm, only one cut is added in moving from LPk to LPk+ ’ , so the number of major iterations is the same as the number of cuts. On the other hand, the method of Crowder et al. may create several cuts per major iteration, and it happened that all problems in Table 1 were completely solved in one major iteration of knapsack facets, and with no need to resort to Gomory cutting planes to converge to the optimal integer solution.

Table 2 gives results for general integer problems (integer with an allowable range xj=o, 1,. . ., 9,lO). This does not immediately allow for solution by knapsack cuts. However, expressing variables as a composite of binary variables does allow this by the transformation xj = 8XJg + 4X,4 + 2XJ2 + XJl.

*More complete details of the computer programs and test data used may be obtained from the first author.

Var

iabl

es

Con

stra

ints

Den

sity

of

A

(%

)

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cuts

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

by

fac

ets

Not

so

lved

Mea

n nu

mbe

r of

ite

ratio

ns

Mea

n nu

mbe

r of

cut

s

Var

iabl

es

Con

stra

ints

Den

sity

of

A (

%)

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

Mea

n nu

mbe

r of

cut

s

Not

so

lved

by

fac

ets

Not

so

lved

Mea

n nu

mbe

r of

ite

ratio

ns

5

Mea

n nu

mbe

r of

cut

s

454 LARRY JENKINS and DAVE PETERS

Table 3. Mean number of cuts to solve O-l problems of low density

MX RAT MX PEN MX DST CJP and MX RAT

30 10 10 2.90 30 20 10 3.14 30 25 10 1.72 30 30 10 2.00 30 40 10 2.48 30 50 10 2.16 40 20 10 5.10 40 30 10 4.30 40 40 10 4.82 50 20 10 2 7.19 50 30 10 2 6.41 50 40 10 5.78 50 50 10 1.26

3.34 1 2.86

1.36 1.06 1.32 1.06 5.22 2.90 1.16

5 6.51 3 6.65 1 2.88

1.12

3.14 1.32 4.04 3.49 1 1.67 4.94 2.00 1.08 2.66 2.40 1.00 3.42 3.56 1.00 4.80 3.46 1.00 5.64

1 6.20 3 2.31 8.02 6.02 1 1.64 10.62 5.06 1.00 12.36

3 6.60 7 1 2.65 12.09 1 a.52 3 2.28 17.39

6.20 1 2.31 21.94 7.22 1.00 25.40

Table 4. Mean number of cuts to solve O-l problems of high density

MX RAT MX FEN MX DST UP and MX RAT

30 10 50 18 9.19 30 20 50 1 9.11 30 25 50 4.04 30 30 50 7.32 30 40 XI 1 9.45 30 xl 50 2 9.08 40 20 50 21 8.05 40 30 50 1 6.54 40 40 SO 6.72 SO 20 50 41 7.50 50 30 M 32 12.00 504050 2 7.82 50 50 so 7.36

7 5.28 3 4.17

1.66 1.52 1.41 1.23

18 5.65 2.61 1.48 5.00 9.60

2 3.53 1.08

3 5.41 :

1.86 18.41 4.43 1.60 18.77 4.40 1.06 13.20 6.08 1.00 16.06 7.10 1.00 21.27 6.79 0.96 24.06

13 8.25 29 6 3.60 43.00 1 5.90 2 1.60 25.63

4.90 1.00 21.06 39 4.50 42 34 4.00 51.00 28 8.70 37 16 4.60 60.30

1 6.64 5 2 1.82 34.13 6.60 1.02 26.46

In reporting the number of cuts for problems solved successfully the spread, measured by range or standard deviation, was not great, so that the mean value is a representative measure.

From results with these smaller problems we concluded that there is no advantage in varying h over all integer values; the results with k = 1 are just as good. It was also decided that first fraction and maximum fraction are very poor criteria so these were dropped from subsequent tests.

In Tables 3,4 and 5 the more successful criteria are compared extensively. For O-l problems of low density (Table 3) many are solved by knapsack facets in a single major iteration. In the batches of 50 problems only a few were not solved completely by knapsack cuts and needed cutting planes to find the integer solution. Of the three cutting plane methods, judging firstly by the number of problems not solved and secondly by the required number of cuts for these problems that were solved

A computational comparison of Gomory and knapsack cuts

Table 5. Mean number of cuts to solve integer problems

455

MX RAT MX PEN MX DST CJP and MX RAT ~- ---

20 to 10 4.83 4 4.89 5.60 42 12 11.83 38.17 20 15 10 3.85 I 3.96 4.08 32 1 1.13 23.94 20 20 10 3.88 3.36 5.20 14 I .08 22.94 30 20 10 1 9.41 3 9.29 1 11.03 43 11 16.24 55.82 30 25 10 5.24 1 4.87 6.82 26 4 7.64 24.31 30 30 10 7.26 3 5.89 8.96 18 6.21 24.11 20 10 xl 5 11.17 8 11.50 3 10.08 48 37 16.42 67.00 20 15 50 6.19 5 4.51 5.73 30 9 11.57 44.57 20 20 50 10.28 8 4.97 7.95 30 3 13.64 53.56 30 20 50 16 63.22 I? 36.11 12 36.11 50 36 28.78 217.33 30 2.5 M 3 16.18 7 11.04 6 17.61 37 14 10.68 70.86 30 30 50 23.48 10 10.09 2 18.42 33 6 13.00 82.33

successfully, then maximum ratio is slightly better than maximum distance which in turn is slightly better than maximum penalty.

In the more dense O-l problems documented in Table 4, knapsack cuts followed if necessary by cutting planes is again most successful, though much less so than for sparse problems. Of the cutting plane methods, maximum distance is in this case marginally better than maximum penalty and maximum ratio is the least successful.

For the integer problems reported in Table 5, the reformulation and solution with knapsack cuts is the least successful. Of the three Gomory cuts, maximum distance is slightly more successful than maximum ratio and maximum penalty is the least successful.

If one considers Tables 3,4 and 5 together, it is generally true that problems with markedly more variables than constraints were much harder to solve than those with a similar number of constraints and variables. Problems with more constraints than variables were even easier to solve. Some observations can also be made on how performance varies with the density of the A-matrix. Not surprisingly more dense problems are more difficult to solve, but there appears to be a difference in performance of the Gomory cuts. Maximum ratio is best on problems of low density but poorer for problems of, high density.

7. CONCLUSION

In an extensive series of tests the approach of varying h over integer values was found to be no more effective than the corresponding Gomory cutting planes with h = 1. Of the commonly proposed criteria for developing Gomory cutting planes, the criteria of first fraction and maximal fraction peyote poorly. The three criteria of maximum ratio, m~imum penalty and maximum distance were approximately equal in performance. The little publicized criterion of maximum distance is effective, but of the three successful methods, maximum ratio is the simplest computationally, and is therefore recommended.

As claimed in [3], the method of knapsack cuts is very successful for sparse 0-l problems, though on such problems Gomory cuts also perform well. On more dense 0-l problems, knapsack cuts out- perform Gomory cuts. The approach of reformulating general integer problems as O-l problems and then solving the reformulated problem by knapsack cuts gives poor performance. Applying Gomory cutting planes to the original formulation is more successful.

We were left with the impression that Gomory cuts for general integer problems and knapsack cuts for O-l problems very successfully diminish the search space in the first few iterations, but then

456 LARRY JENKINS and DAVE PETERS

become ineffective. If the integer optimum has not been found in the first few iterations a switch to some implicit enumeration method would be best.

1.

2.

3.

4. 5. 6. I. 8. 9.

REFERENCES

R. E. Gomory, An algorithm for integer solutions to linear programs. In Recent Advances in Mathematical Programming (Edited by R. L. Graves and P. Wolfe), pp. 269-302. McGraw-Hill, New York (1963). C. A. Trauth Jr and R. E. Wooisey, Integer linear programming: A study in computational etliciency. Mgmt Sci. 15,481- 493 (1969). H. Crowder, E. L. Johnson and M. Padberg, Sotving large-scale zero-one hnear programm~g problems. Opns Res. 31, 803-834 (1983). R. S. Gartimkel and G. L. Nemhauser, Integer Programming. Wiley, New York (1972). H. A. Taha, Integer Programming. Academic Press, New York (1975). E. Balas, Facets of the knapsack polytope. Math. Progm. 8, 146-164 (1975). D. Klein and S. Helm, Integer programming post-optimal analysis with cutting planes. Mgmt Sci. 25,6-&72 (1979). L. Schrage, User’s Mununl for LINDO. Scientihc Press, Palo Alto, Calif. (1981). B. W. Y. Lin and R. L. Rardin, Development of a parametric generating procedure for integer prog~~~g test problems. JACM 24,465-472 (1977).