39

3. LP ANALYTIC SOLUTIONS--SIMPLEX METHOD

3.1 SOLUTION CONCEPT FOR LP PROBLEMS

The LP problem in standard form is:

Max Z = j = 1

n

 cj xj ...[3.1]

subject to:

j = 1

n

 aij xj = bi (i = 1, 2, ..., m) ...[3.2]

xj ≥ 0 (j = 1, 2, ..., n) ...[3.3]

The formulation of the problem, the variables, the constraint equations, and the objective func-tion has all been discussed. The problem of finding the optimal solution remains. Consider theconstraint equations. Where the number of variables n is equal to the number of equations m,one can solve this set of independent linear equations. One unique solution will be obtained.There is no problem in optimization here because there is only one solution to the system ofequations. The values obtained of the variables from the constraint equations are substituted inthe equation for the objective function, and the solution is obtained immediately. It is when thereare more variables than equations, i.e., when n > m, that a problem arises. In this situation, thereis not one unique solution but an infinite number of solutions. From all these solutions, onewishes to find the one that yields the best value of the objective function.

The method used is to set n - m variables equal to zero, i.e., the excess number of variables.Then there will be m variables left and m equations to solve. The object is to determine whichvariables to set equal to zero. By examining every possible way of selecting n - m variablesfrom n variables, it can be shown that the optimum solution lies among these many solutions.

The n - m variables set equal to zero are called non-basic variables; the remaining m variablesare known as basic variables, and the collection of them is called a basis. The solution of theequations involving the basic variables will yield nonnegative values for them, and will not vio-late any of the constraint equations. Such a solution is called a basic feasible solution. Theoptimal solution is to be found among the basic feasible solutions. Therefore, once a basicfeasible solution is obtained, the object is to find a new basic feasible solution wherein the valueof the objective function is improved. This process is continued until a point is reached at whichno further improvement is possible.

This process can be illustrated using the following problem:

40

Max Z = 3.5 X1 + 5 X2 (Objective Function) ...[3.4]

s.t.: 2 X1 + X2 ≤ 6000 (Constraint 1) ...[3.5]

X1 + 4 X2 ≤ 10000 (Constraint 2) ...[3.6]

Since we do not know how to solve a system of inequalities, the above problem can be convertedto a system of equalities by adding slack variables, X3 and X4:

Max Z = 3.5 X1 + 5 X2 (Objective Function) ...[3.7]

s.t.: 2 X1 + X2 + X3 = 6000 (Constraint 1) ...[3.8]

X1 + 4 X2 + X4 = 10000 (Constraint 2) ...[3.9]

Constraints 1 and 2 (in Equations [3.8] and 3.9]) indicate that there are four variables and twoequations. Two of these four variables must be set equal to zero, and then values can be foundfor the other two. For example, setting X1 and X2 equal to zero and solving for X3 and X4, oneobtains X3 = 6,000 and X4 = 10,000. How many ways are there to set two variables equal froma total of four? This is the well-known problem for the number of combinations of P thingstaken Q at a time, which is

PQ

Ê Ë Á ˆ

¯ ˜ = P!

Q! (P - Q)!...[3.10]

which, for P = 4 and Q = 2, gives

4!2! (4 - 2)!

= 4 (3) (2) (1)2 (1) (2) (1)

= 6 combinations

Table 3.1 shows all six solutions, and illustrates them in terms of where they can be found in theplot of the solutions space shown in Figure 3.1.

It will be noted that the four feasible solutions correspond to points O, A, B, and E of Figure 3.1.These points are called extreme points or vertices. In all LP problems these extreme points cor-respond to basic feasible solutions. In examining the six solutions, the non-feasible ones wereeliminated immediately; of the remaining basic feasible solutions, the optimal solution wasfound.

41

Table 3.1: Basic Solutions to the Example Problem

X1 X2 X3 X4 ZPoint in

Figure 3.10 0 6,000 10,000 0 O0 6,000 0 -14,000 infeasible D0 2,500 3,500 0 12,500 A

3,000 0 0 7,000 10,500 B10,000 0 -14,000 0 infeasible C2,000 2,000 0 0 17,000 E

10

5

5 10 15

5

3.5 ObjectiveSlopeD

A E

BC

feasibleregion

O

Figure 3.1: Graphical Illustration of Example Problem

3.2 LP DEFINITIONS AND THEOREMS

Basic definitions and theorems useful in discussing the solution procedure for LP problems are:

Convex set: This is a collection of points such that if P1 and P2 are any two points in the collec-tion, all points on the line segment joining them are also in the collection. For example, setssuch as those illustrated in Figure 3.2 are convex.

42

Convex Sets

Figure 3.2: Examples of Convex Sets

However, boundaries such as those illustrated in Figure 3.3 do not enclose convex sets because itis possible to choose at least one pair of points (as shown) such that not every point on the linesegment joining them belongs to the set.

P1

P2

P1

P2

Nonconvex Sets

Figure 3.3: Examples of Nonconvex Sets

Vertex (or extreme point): This is a point in the convex set that does not lie on a segment joiningtwo other points of the set. Thus, for example, every point on the circumference of a circle andeach vertex of the polygon satisfy the requirements for an extreme point.

Feasible solution: Any solution of the constraint equations satisfying the non-negativity condi-tions is a feasible solution. In the example given in the previous section, every point bounded bythe by the polygon OBEA (convex set) is a feasible solution.

Basic solution: This is a solution of the constraint equations obtained by setting the “excessnumber”, n - m, of variables to zero and solving the constraint equations simultaneously. In theexample of the previous section, each of the six solutions in Table 3.1 is a basic solution, butonly O, A, B, and E are basic and feasible.

Basis: The collection of variables not set equal to zero to obtain the basic solution is the basis.In Table 3.1, for example, for the solution at point B, the basis is the collection (X1, X4).

Basic feasible solution: This is a basic solution that satisfies the non-negativity conditions. Asnoted, solutions at points O, A, B, and E in Table 3.1 are basic feasible solutions.

43

Optimal solution: A feasible solution that satisfies the objective function is an optimal solution.Point E in Figure 3.1 is the optimal solution to the example problem.

Optimal basic solution: This is a basic feasible solution for which the objective function is opti-mal. From Table 3.1, it is clear that only the solution at point E is an optimal basic solution.

3.3 BASIC THEOREMS OF LP

The three basic theorems of linear programming can now be cited.

Theorem 3.1: The collection of feasible solutions constitutes a convex set whose extremepoints correspond to basic feasible solutions.

This theorem tells us that we need be concerned only with convex sets since the only solutions ofinterest to us must be contained in the class of feasible solutions. Also, basic feasible solutionscorrespond to vertex points.

Theorem 3.2: If a feasible solution exists, then a basic feasible solution exists.

Theorem 3.1 assured us that the convex set contains all the feasible solutions. Now, if we find afeasible solution (say, by trial and error), then there must be at least one vertex to the convex set.

Theorem 3.3: If the objective function possesses a finite minimum, then at least one optimalsolution is a basic feasible solution.

If the linear programming problem has been properly formulated, it will satisfy the hypothesis ofa finite minimum. This theorem assures us that at least one of the optimal solutions is a basicfeasible solution. Therefore, our search for an optimal solution can be confined to the extremepoints. This observation is of prime importance in the development of the simplex method tosolve linear-programming problems.

The proofs of these theorems are not difficult and can be found in several of the standard texts onlinear programming.

3.4 SIMPLEX ALGORITHM

Graphical solutions work only in two (or, with great difficulty, three) dimensions and, therefore,for real problems we clearly need a solution technique for n-dimensional space (for 120 Xijt vari-ables for instance). That technique is the simplex algorithm that is now available in very effi-cient forms for computer solutions. We will solve a problem manually to demonstrate themethod.

Consider two linear systems of equations:

44

(1) 2 X1 + 4 X2 = 803 X1 + 2 X2 = 60

and

(2) X1 + 0 X2 = 100 X1 + X2 = 15

Which is preferable? System 2 has a solution (by inspection of X1 = 10, X2 = 15). System 1 canbe solved using the Gauss Jordan technique as follows:

Convert system 1 to the general matrix notation, Ax = b, where A = 2 43 2

È Î Í

˘ ˚ ˙ , x is the column

vector X1X 2

Ï Ì Ó

¸ ˝ ˛

, and b is the column vector 8060

Ï Ì Ó

¸ ˝ ˛

. Now, use Gauss-Jordan pivoting to convert A to

the identity matrix, I = 1 00 1

È Î Í

˘ ˚ ˙ , as we had in system 2 above. At the same time, the appropriate

Gauss-Jordan manipulations should be made on the b vector. This process is illustrated in thecalculations shown in Table 3.2.

Table 3.2: Illustration of Gauss-Jordan PivotingTableau X1 X2 bi Comments

2 4 801st Tableau3 2 60

We desire to replace the first 2 by a 1; to do this, multiply the1st row by 0.5. Also, we desire a zero in place of the 3, so wemultiply the new 1st row by -3 and add it to the 2nd row.These steps generate the entries in the 2nd Tableau.

1 2 402nd Tableau0 -4 -60

We now desire a 1 in place of the -4; to do this, multiply therevised 2nd row by -0.25. We replace the 2 in the revised rowby 1 by multiplying the revised 2nd row by -2 and adding.This generates the entries in the 3rd Tableau.

1 0 103rd Tableau0 1 15

We see that the solution to both systems 1 and 2 are identical and that the identity matrix form isvery valuable. If we now add an objective function to the matrix given previously, we have thegeneral form of the LP problem.

Max or Min Z = CTX ...[3.11]

subject to (s.t.):

AX ≤=≥ b ...[3.12]

45

or in expanded form:

Max or Min Z = C1 X1 + C2 X2 + ... + Cn Xn ...[3.13]

s.t.: a11 X1 + a12 X2 + ... + a1n Xn ≤ b1 ...[3.14]

a21 X1 + a22 X2 + ... + a2n Xn ≤ b2

...

am1 X1 + am2 X2 + ... + amn Xn ≤ bm

We seek the values of Xj that maximize Z(Xj). The aij, bi, and Cj are all known constants and theXj are decision variables. The previous example problem (system 1) had only equalities and hada unique solution because n = m = 2. Our general LP problem can include ≤, ≥, or = types ofconstraints. How do we solve inequalities? We don’t. We convert them to equalities as follows:

Given a j = 1

n

 aij Xj ≤ bi constraint, use:

j = 1

n

 aij Xj + Si = bi (i = 1, 2, ..., m) ...[3.15]

For example, instead of:

2 X1 + 5 X2 ≤ 20use

2 X1 + 5 X2 + S1 = 20

where S1 is a "slack variable". Or, instead of:

2 X1 + 5 X2 ≥ 20

use

2 X1 + 5 X2 - S1 = 20

where S1 is now a “surplus variable”.

Therefore, we can convert AX ≤ b to AX' = b, but we will in general have more variables thanconstraints (n > m). If n > m, we have an infinite number of solutions. We can therefore, arbi-trarily set n - m of the Xj = 0 and solve an n-by-m linear system as before. However, we seek the

46

best solution among this infinite number. The simplex algorithm finds this optimal solution byiteratively solving the m-by-m systems, identifying one of the Xj that is currently equal to zero(i.e., which is currently non-basic) but which could improve the solution if it were to be broughtinto the basis. This will force one of the non-zero Xj to become equal to zero (forcing it out ofthe basis). Fortunately, we are guaranteed convergence to the optimal solution in a finite numberof iterations because the optimal solution is always at a “corner” in our m-dimensional solutionspace (an intersection of m of the constraints). Consider the LP problem::

Max Z = 6 X1 + 5 X2 ...[3.16]

s.t.: X1 ≤ 3 (constraint 1) ...[3.17]

X2 ≤ 4 (constraint 2) ...[3.18]

2 X1 + X2 ≤ 8 (constraint 3) ...[3.19]

Graphically we can identify the optimal solution as shown in Figure 3.4.

10

5

5 10

(2)

(1)

(3)feasibleregion

Z = 32

X 2

X1

Figure 3.4: Graphical Solution to Example Problem

We can get an analytic solution as follows: To get a starting point, we convert the constants toequalities by adding a new Sj to each one as slack variables. Our problem is now

Max Z = 6 X1 + 5 X2 + 0 S3 + 0 S4 + 0 S5 ...[3.20]

s.t.: X1 + S3 = 3 (constraint 1) ...[3.21]

47

X2 + S4 = 4 (constraint 2) ...[3.22]

2 X1 + X2 + S5 = 8 (constraint 3) ...[3.23]

We can write the first simplex tableau with the AX = b system first, followed by the objectivefunction as shown in Table 3.3. We will use simplex rules for minimizing, but our problem is tomaximize. Therefore, we use ˜ Z = -Z as our objective function because Max Z = Min ˜ Z . There-fore, ˜ Z = -6 X1 - 5 X2.

Table 3.3: Illustration of Simplex Tableau CalculationsTableau X1 X2 S3 S4 S5 b or Z bi/aij

1 0 1 0 0 3 3/1 fl Min.0 1 0 1 0 4 4/02 1 0 0 1 8 8/2

1st Tableau:S3, S4, and S5are basic

-6 -5 0 0 0 0 = - ˜ Z 1 0 1 0 0 3 3/00 1 0 1 0 4 4/10 1 -2 0 1 2 2/1 fl Min.

2nd Tableau:X1, S4, and S5are basic

0 -5 6 0 0 18 = - ˜ Z 1 0 1 0 0 3 3/10 0 2 1 -1 2 2/2 fl Min.0 1 -2 0 1 2 -1/1

3rd Tableau:X1, X2, and S4are basic

0 0 -4 0 5 28 = - ˜ Z 1 0 0 -0.5 0.5 20 0 1 0.5 -0.5 10 1 0 1 0 4

4th Tableau:X1, X2, and S3are basic

0 0 0 2 3 32 = - ˜ Z = Z*

Note in the first tableau of Table 3.3 that we already have an initial solution: X1 = 0, X2 = 0, S3= 3, S4 = 4, S5 = 8, and Z = 0. This is the “do-nothing” solution. The identity matrix we require(our basic variables) includes S3, S4, and S5. We now seek a better solution. Our first rule is thata negative coefficient in the objective row will identify a non-basic Xj that could improve ourbasic solution, and the most negative will improve it most. We therefore decide to bring X1 intothe basis. Our second rule relates to how large Z can become (i.e., which row will have a coeffi-cient of 1 in the X1 column). This means we need to know which constraint is most binding.Our rule is to select the constraint with the minimum positive ratio bi/aij. We see that 3/1 isminimum so we “pivot” on row 1, column X1 (i.e., we use Gauss-Jordan arithmetic to force thatcoefficient to 1 and other X1 coefficients to 0). By Gauss-Jordan operations our 2nd Tableau inTable 1 is developed. The pivot cell is marked in green in Table 3.3, and the pivot row and col-

48

umn are shaded in blue. Note that we treat the Z row identically to the constraint rows. We nowhave a solution X1 = 3, X2 = 0, S3 = 0, S4 = 4, S5 = 2, and Z has improved from 0 to 18.

We still have a negative Cj in the X2 column, so we bring X2 into the basis in the third Tableau.The most binding row is now 3 and so we force zeros into the X2 column except for row 3(where we need a 1 for our new identity matrix). The third solution is X1 = 3, X2 = 2, S3 = 0, S4= 2, S5 = 0, and Z = 28. Note that we forced S3 out of the basis in Tableau 2, but its coefficientis now -4. We can improve the solution further by bringing it back to the basis in the fourthsolution.

The fourth tableau has no negative Cj and, therefore, it is optimal. The final solution is: X1 = 2,X2 = 4, S3 = 1, S4 = S5 = 0, and Z* = 32, which matches the graphical solution in Figure 3.2.

By following the four iterations on the graphical version of this problem, we can observe ourjourney along various corners of the solution (refer to Figure 3.2). We began at the origin. Thenwe went along the X1 axis to X1 = 3, X2 = 0. Notice that this forced S3 to 0 because the slack = 0when we are on that constraint. We then move up to X1 = 3, X2 = 2. We are now touching thethird constraint, so S4 = 0 and S3 = 0, but we are still two units away from constraint two, so S4 =2. The final solution moves us away from constraint one again, so S3 > 0, etc. The simplex didnot take us directly to the best corner because it can only bring one vector in (and force one out)at each iteration. Geometrically this means it cannot move inside the solution space but mustremain on the surface.

We have a distinct advantage in this example because the three slack variables gave us aninitial solution as a gift. What if, however, the third constraint had the inequality reversed? This“greater-than” type of constraint would require the addition of an artificial variable that could beused to get us to a feasible solution--it would then disappear from the problem, and we wouldproceed as described in the following section.

3.5 “GREATER-THAN” CONSTRAINTS

The following repeats the previous problem, but with the third constraint revised to:

2 X1 + X2 ≥ 8 ...[3.24]

This new problem is shown in Figure 3.3. Now when we subtract the surplus variable we havethe equality:

2 X1 + X2 - S5 = 8 ...[3.25]

and the identity matrix we need for the basis does not exist. We are tempted to multiply Equa-tion [3.25] by -1 to get

49

-2 X1 - X2 + S5 = -8 ...[3.26]

but S5 = -8 is not allowed (what would be the meaning of a negative surplus?) because the sim-plex limits all variables to being non-negative. Therefore, we have to define a new “artificial”variable, T6, and use

2 X1 + X2 - S5 + T6 = 8 ...[3.27]

Now we have an identity matrix, as shown in the first tableau of Table 3.4, but we want to get ridof the artificial variable as quickly as possible in order to get out of artificial space and into fea-sible space of the real problem. Therefore, we add a new row, called the W row, in which W =T6. However, since we need T6 in the basis, we need a zero in the T6 column of the W row.Therefore, we express W in terms of the other variables:

T6 = W = 8 - 2 X1 - X2 + S5 ...[3.28]

If we had more than one greater-than constraint we would need more than one Tj variable, andthe definition of W would, in general, be W = Â Tj.

Table 3.4: Simplex Calculations for a “Greater-Than” ConstraintX1 X2 S3 S4 S5 T6 b or Z bi/aij-6 -5 0 0 0 0 0 = - ˜ Z -2 -1 0 0 1 0 -8 = -W1 0 1 0 0 0 3 3/1 fl Min.0 1 0 1 0 0 4 4/0

1st Tableau

2 1 0 0 -1 1 8 8/20 -5 6 0 0 0 18 = - ˜ Z 0 -1 2 0 1 0 -2 = -W1 0 1 0 0 0 3 3/00 1 0 1 0 0 4 4/1

2nd Tableau

0 1 -2 0 -1 1 2 2/1 fl Min.0 0 -4 0 -5 5 28 = - ˜ Z 0 0 0 0 0 1 01 0 1 0 0 0 3 3/00 0 2 1 1 -1 2 2/1 fl Min.

3rd Tableau

0 1 -2 0 -1 1 2 2/-10 0 6 5 0 38 = - ˜ Z 1 0 1 0 0 30 0 2 1 1 2

4th Tableau

0 1 0 1 0 4

50

In Table 3.4, the right side column coefficient equals -W just as that coefficient in the Z row is -˜ Z . We now proceed to the second tableau by the same rules, except we use the W row rather

than the Z row to select the most negative Cj until we have forced W to become equal to zero (inthe third tableau). We now have a feasible solution to the original problem. We now eliminatethe W row and T6 column and use the Z row for our most negative Cj selection. In the fourthtableau we see no negative Cj, so the optimal solution has been found: X1 = 3, X2 = 4, S5 = 2,and Z = 38.

Figure 3.3 shows graphically these steps leading to the optimal solution.

10

5

5 10

(2)

(1)

(3)feasibleregion

Z = 38

X 2

X1

Figure 3.3: A Linear Programming Problem with a “Greater-Than” Constraint

3.6 SHADOW PRICES

We are now in a position to relate the variables (the shadow prices) discussed in the Lagrangianapproach to the Cj coefficients in the simplex tableau. They are in fact identical. The coeffi-cients in the slack or surplus columns of the Z row equal the imputed values of the resourcesassociated with each row. The Z row coefficients in Table 3.4 are:

Row li Slack or Surplus1 6 S3 = 02 5 S4 = 03 0 S5 = 2

This implies that, for example, if constraint one represented a limitation on water, one additionalunit of water would be worth $6 (if the objective function is in expressed in dollars), since l1 =

51

C3 = ∂Z∂bi

. The shadow price of constraint three (l3) is zero because the constraint is not binding.

Note that always either li or Si = 0. This condition is called “complementary slackness”. Thereis no value in acquiring more of a resource if we are not using what we already have (if a slackor surplus variable is greater than zero in the optimal solution, its corresponding shadow pricewill be equal to zero).

3.7 CANONICAL FORM

The simplex algorithm notation and format presented here is referred to as the "canonical form".Stark and Nicholls (1972) give a more rigorous description of this approach. Modern proprietaryLP algorithms use many modifications of the simplex and other algorithms to improve effi-ciency. Problems with more than a thousand variables can now be easily solved in a few secondson a microcomputer.

52

3.8 PROBLEMS

Solve Problems 1, 2, and 3 from Section 2 using the simplex algorithm.