9.1 lecture_2b

Lecture 2b:

Dual Simplex Method

Jeff Chak-Fu WONG

Department of Mathematics

Chinese University of Hong Kong

[email protected]

MAT581SS

Mathematics for Logistics

Produced by Jeff Chak-Fu WONG 1

TABLE OF CONTENTS

1. Some Useful Deductions from the Complementary Slackness

Theorem

2. Dual Simplex Method

TABLE OF CONTENTS 2

SOME USEFUL DEDUCTIONS FROM THE C.S. THEOREM

SOME USEFUL DEDUCTIONS FROM THE C.S. THEOREM 3

The complementary slackness conditions as stated in Theorem 4

(see Lecture note 2a, page 81) provide certain useful information

about the optimal solution of the dual (primal), knowing the optimal

solution of the prime (dual).

For this let us consider the first complementary slackness condition,

namely wT (Ax− b) = 0, which can be expanded as

m∑

i=1

wi

(n∑

j=1

aij xj − bi

)= 0. (1)


But in (1), the finite sum of m non-positive quantities equals zero and

hence we get

wi

(n∑

j=1

aij xj − bi

)= 0 (i = 1, 2, · · · , m). (2)

Now for each i, the L.H.S. of (2) is the product of two numbers and

therefore

n∑

j=1

aij xj < bi ⇒ wi = 0, (3)

and

wi > 0⇒

n∑

j=1

aij xj = bi. (4)


In view of the above relations we infer that

• if for the optimal solution X of the primal, the ith constraint holds as strict

inequality,

• then in the optimal solution w of the dual, the ith component namely

wi must be zero.

Equivalently

• if in the optimal solution w of the dual, wi > 0,

• then at the optimal point of the primal the ith primal constraint holds as

an equation.

Therefore (3) - (4) are also called complementary slackness conditions.

Equation (3) Equation (4)

n∑j=1

aij xj < bi ⇒ wi = 0 wi > 0 ⇒n∑

j=1

aij xj = bi


In view of the above relations we infer that

• if for the optimal solution X of the primal, the ith constraint holds as strict

inequality,

• then in the optimal solution w of the dual, the ith component namely

wi must be zero.

Equivalently

• if in the optimal solution w of the dual, wi > 0,

• then at the optimal point of the primal the ith primal constraint holds as

an equation.

Therefore (3) - (4) are also called complementary slackness conditions.


n∑j=1

aij xj < bi ⇒ wi = 0 wi > 0 ⇒n∑

j=1

aij xj = bi


ECONOMIC INTERPRETATION OF THE C.S. CONDITIONS

ECONOMIC INTERPRETATION OF THE C.S. CONDITIONS 8

Problem (13) Problem (14)

max CT X min bT w

subject to subject to

AX ≤ b AT w ≥ C

with X ≥ 0 with w ≥ 0

• – If the primal problem (13) has certain meaningful economic

interpretation

– then it is expected that the dual problem (14) and related

duality results will also have some meaningful economic

interpretations.

• In fact this is true and for details we may refer to an appropriate

book on mathematical economics.


• However, as an illustration we do present here an economic

meaning of the complementary slackness conditions (3) and

(4).


n∑j=1

aij xj < bi ⇒ wi = 0 wi > 0 ⇒n∑

j=1

aij xj = bi



max CT X min bT w


AX ≤ b AT w ≥ C


In the primal problem (13), let us interpret the parameters b, C and A as

follows

bi = units of the ith raw material available (i = 1, 2, · · · , m)

cj = per unit profit for the jth product (j = 1, 2, · · · , n)

aij = units of ith raw material used in producing one unit of the jth product

(i = 1, 2, · · · , m, j = 1, 2, · · · , n).

• Thus we are given m different raw materials which can be used to

produce n different products.

• These raw materials are available in limited supply and the entire output

can be sold in the market.

• Our aim is to obtain the units of each product to be produced so that total


profit is maximum.

• The mathematical model of the above problem just leads to problem

(13).


max CT X min bT w


AX ≤ b AT w ≥ C




max CT X min bT w


AX ≤ b AT w ≥ C


Let us now look at the dual problem (14) in the context of the economic

interpretation of problem (13) as described above.

• Note that

– the dimension of cj is dollars per unit of the jth product and

– the dimension of aij is units of the ith raw material per unit of the jth

product.

• Therefore the constraintsm∑

i=1

aijwi ≥ cj

imply that the dimension of wi must be per unit of the ith raw materialbecause in the given inequality the dimensions of both sides must be

same.


m∑

i=1

aijwi ≥ cj

Thus to each raw material i, there corresponds a dual variable wi which

gives the valuation (or price) of one unit of the ith raw material.

In economics wi are called the shadow prices (or accounting prices, or

fictitious prices or imputed prices) and are different from the actual

prevailing prices as explained below.


Let us consider the situation where it is decided that the raw materials on

hand should be insured against fire, theft etc.

This insurance is intended to protect the total income of the products after

they are sold in the market.

The problem is to find that insurance scheme which is large enough to provide thefull compensation and at the same time minimizes the total insurance cost.

It can be seen that the mathematical model of this later problem is precisely

the dual problem (14).


max CT X min bT w


AX ≤ b AT w ≥ C



The dual variables wi are called imputed values or shadow prices for the

various raw materials.

As mentioned earlier these wi are not the actual per unit costs of raw

materials but rather what we really perceive about them.

If we now look at the first complementary slackness condition

n∑

j=1

aij xj < bi ⇒ wi = 0,

then we observe that

• if in our profit maximization problem, at the optimal production level,

the ith raw material is not being used fully (i.e. < bi)

• then that raw material should be a cheap (the way we perceive) raw

material because in the optimal solution of the dual, wi = 0.


On the other hand, when looking at the second complementary slackness

condition, we have:

wi > 0 ⇒n∑

j=1

aij xj = bi.

• If the ith raw material is costly (wi > 0)

• then in the optimal solution of the primal, the ith raw material will be

consumed fully, i.e.n∑

j=1

aij xj = bi.


ALTERNATE ALGORITHMS FOR SOLVING LPP’S

ALTERNATE ALGORITHMS FOR SOLVING LPP’S 18

The complementary slackness theorem also opens up possibilities of

developing new algorithms for solving LPP’s.

This is because it says that

• if we achieve

1. the primal feasibility,

2. the dual feasibility and

3. the complementary slackness conditions

• then we have optimality for both primal and dual.

As there are three things to be achieved at the end, we may think

of algorithms which

• maintain any two of these throughout and

• stop when the third is also achieved.

ALTERNATE ALGORITHMS FOR SOLVING LPP’S 19

THE DUAL SIMPLEX METHOD

THE DUAL SIMPLEX METHOD 20

Here we present a stepwise description of the dual simplex

algorithm which is due to Carlton E. Lemke .

Our aim here is that the dual simplex method is essentially equivalent tosolving the dual of the given problem by the usual simplex method.


Let the given LPP be

max CTX

subject to

AX = b (5)

X ≥ 0, (6)

where C ∈ Rn,b ∈ R

m, A = [aij ]m×n with rank A = m.

In (5), we do not insist that b ≥ 0, because the starting point of the

dual simplex algorithm is a basic solution XB = B−1b for which all

(zj − cj) ≥ 0 but some xBimay be negative, i.e. the solution XB is

dual feasible but it may not be primal feasible.


The following steps are:

Step 1 Start with a primal basic solution XB = B−1b (for some basis matrix

B) for which all (zj − cj) ≥ 0.

Step 2

• Check if all xBi≥ 0.

• If the answer is ’yes’ then we stop as XB is optimal to the given LPP.

• If some xBi< 0 then we go to Step 3.

Step 3

• Find the most negative value of xBi, i.e.

xBr= min

i{xBi

∣∣ xBi< 0}.

• Then xBrbecomes non-basic and column br of the variable xBr

leaves

the basis.

• The row corresponding to the variable xBrbecomes the pivot row.


Step 4

• Find(zk − ck)

yrk

= maxj

{(zj − cj)

yrj

∣∣∣∣ yrj < 0

}

then the variable xk becomes a basic variable and the corresponding

column ak enters the basis.

• The column corresponding to (zk − ck) becomes the pivot column.

Step 5 Get the new dual simplex tableau by using the pivoting which is

exactly same as in the usual simplex method, and then go to Step 2.


Here we make the following observations

(i)

• In the dual simplex method we

– first find the pivot row, and then

– find the pivot column,

• In the simplex method we

– first find the pivot column, and then

– find the pivot row.

(ii) The pivot element in the dual simplex method is always negative,

whereas in the simplex method it is always positive.

(iii)

• The maximum ratio criteria in the dual simplex method is used to

make sure that all (zj − cj) remain non-negative in the next iteration.

• The minimum ratio criteria in the simplex method is used to

guarantee that all xBiremain non-negative in the next iteration.


(iv)

• It is possible that in the pivot row there may not be any yrj < 0.

• In that case it can be shown that the given LPP is infeasible.

• This situation is very similar to the usual simplex method where we know

that

– if in the pivot column there is no yij > 0,

– then the given LPP has unbounded solution.

• As ’unboundedness’ and ’infeasibility’ are dual concepts, it is natural

that the dual simplex method should check infeasibility because we

know that the simplex method checks unboundedness of the given LPP.


(v)

• Steps 2 and 3 of the dual simplex method seem to be very natural but

Step 4 probably needs some justification.

• Given that for the current dual simplex tableau all (zj − cj) ≥ 0.

• Once the pivot row has been identified the value of r gets fixed.

• Then the basic aim of Step 4 is to guarantee that all (zj − cj) remain

non-negative for the next tableau as well.

• But

(zj − cj) = (zj − cj) −yrj

yrk

(zk − ck)

and therefore (zj − cj) ≥ 0 means that for all j,

(zj − cj) −yrj

yrk

(zk − ck) ≥ 0. (7)

• Now in (7), (zj − cj) ≥ 0, (zk − ck) ≥ 0, yrk < 0 and therefore (7) trivially

holds for all j for which yrj ≥ 0.


c1 c2 · · · cj ck · · · cn

CB XB x1 x2 · · · xj xk · · · xn

cB1xB1

b1 y1j y1k

cB2xB2

b2 y2j y2k

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

cBrxBr

br < 0 yrj yrk < 0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

cBmxBm

bm ymj ymk

z(XB) (z1 − c1) (z2 − c2) · · · (zj − cj ) (zk − ck) · · ·


c1 c2 · · · cj ck · · · cn

CB XB x1 x2 · · · xj xk · · · xn

cB1xB1

b1 −

br

yrk

y1k y1j −

yrj

yrk

y1j 0

cB2xB2

b2 −

br

yrk

y2k y2j −

yrj

yrk

y2j 0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

ck xBr

br

yrk

yrj

yrk

1

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

cBmxBm

bm −

br

yrk

ymk ymj −

yrj

yrk

ymj 0

z(XB) (z1 − c1) (z2 − c2) · · · (zj − cj ) 0 · · ·


zj = cB1

(y1j −

yrj

yrk

y1j

)+ cB2

(y2j −

yrj

yrk

y2j

)

+ · · · + ck

yrj

yrk

+ · · · + cBm

(ymj −

yrj

yrk

ymj

)

=(cB1y1j + cB2y2j + · · · + cBm

ymj

)− cBr

yrj + ck

yrj

yrk

−yrj

yrk

(cB1y1k + cB2y2k + · · · + cBm

ymk − cBryrk

)

= zj − cBryrj + ck

yrj

yrk

−yrj

yrk

(zk − cBryrk)

= zj −yrj

yrk

(zk − ck) .

That is

zj − cj = zj − cj −yrj

yrk

(zk − ck)

j = 1, 2, · · · , n, and , j 6= k.


zj − cj = zj − cj −yrj

yrk

(zk − ck)

j = 1, 2, · · · , n, and , j 6= k.

• Therefore we have to bother about those j for which yrj < 0.

• In that case (7) giveszj − cj

yrj

≤zk − ck

yrk

,

i.e.zk − ck

yrk

= maxj

{zj − cj

yrj

∣∣∣∣ yrj < 0

},

which is nothing but the maximum ratio criteria as given in Step 4.


Example 1 Use the dual simplex method to solve the following LPP

max −2x1 − x2

subject to

2x1 − x2 − x3 − x4 = 3

x1 − x2 + x3 − x5 = 2

x1, x2, x3, x4, x5 ≥ 0.


Now taking the variables x4 and x5 as basic variables we obtain

XB =

x4

x5

=

−1 0

0 −1

3

2

=

−3

−2

y1 =

−1 0

0 −1

2

1

=

−2

−1

y2 =

−1 0

0 −1

−1

−1

=

1

1

y3 =

−1 0

0 −1

−1

1

=

1

−1

y4 =

−1 0

0 −1

−1

0

=

1

0

y5 =

−1 0

0 −1

0

−1

=

0

1


cB =

0

0

, z(XB) = 0

(z1 − c1) = 0− (−2) = 2

(z2 − c2) = 0− (−1) = 1

(z3 − c3) = 0− 0 = 0

(z4 − c4) = 0− 0 = 0

(z5 − c5) = 0− 0 = 0.

As for the basic solution XB , all (zj − cj) ≥ 0, we have got the right

situation to use the dual simplex method.


The initial dual simplex tableau is

XB y1 y2 y3 y4 y5

← x4 = −3 -2 1 1 1 0

x5 = −2 -1 1 -1 0 1

0 2 1 0 0 0

↑


FIRST ITERATION

FIRST ITERATION 37

Step 2 As not all xBi≥ 0, the current solution is not optimal.

Step 3 xBr = min{xBi| xBi

< 0} = min{−3,−2}, which corresponds

to x4 and therefore x4 becomes nonbasic variable.

Step 4 As in the pivot row only one yrj < 0, we get the first column

as the pivot column and x1 becomes a basic variable.

XB y1 y2 y3 y4 y5

← x4 = −3 -2 1 1 1 0 − 1

2× R1

x5 = −2 -1 1 -1 0 1

0 2 1 0 0 0

↑

2/-2 – – – –

FIRST ITERATION 38

XB y1 y2 y3 y4 y5

← x4 = −3/2 1 -1/2 -1/2 -1/2 0

x5 = −2 -1 1 -1 0 1 R2 + R1

0 2 1 0 0 0 R3 − 2× R1

↑

Step 5 The pivoting element is ykr = −2 and after pivoting we get

the next tableau as

XB y1 y2 y3 y4 y5

x1 = 3/2 1 -1/2 -1/2 -1/2 0

← x5 = −1/2 0 1/2 -3/2 -1/2 1

-3 0 2 1 1 0

↑

– – 1/(-3/2) 1/(-1/2) –

FIRST ITERATION 39

XB y1 y2 y3 y4 y5

x1 = 3/2 1 -1/2 -1/2 -1/2 0

← x5 = −1/2 0 1/2 -3/2 -1/2 1

-3 0 2 1 1 0

↑

FIRST ITERATION 40

XB y1 y2 y3 y4 y5

x1 = 3/2 1 -1/2 -1/2 -1/2 0

← x5 = −1/2 0 1/2 -3/2 -1/2 1

-3 0 2 1 1 0

↑

FIRST ITERATION 41

SECOND ITERATION

SECOND ITERATION 42

• The current solution is still not optimal as x5 = −1/2 < 0.

• So we identify the pivot row and conclude that x5 becomes a

nonbasic variable.

To identify the pivot column we use the maximum ratio criteria,

i.e.

max

{1

−3/2,

1

−1/2

}= −2/3,

which corresponds to the value (z3 − c3) and therefore x3

becomes a basic variable.

• We also note that the pivot element is −3/2.

XB y1 y2 y3 y4 y5

x1 = 3/2 1 -1/2 -1/2 -1/2 0

← x5 = −1/2 0 1/2 -3/2 -1/2 1 − 2

3× R2

-3 0 2 1 1 0

↑

SECOND ITERATION 43

XB y1 y2 y3 y4 y5

x1 = 3/2 1 -1/2 -1/2 -1/2 0 R1 + 1

2× R2

x3 = −1/3 0 -1/3 1 1/3 -2/3

-3 0 2 1 1 0 R3 −R2

• After pivoting, we get the following tableau

XB y1 y2 y3 y4 y5

x1 = 5/3 1 -2/3 0 -2/3 -1/3

x3 = 1/3 0 -1/3 1 1/3 -2/3

-10/3 0 5/3 0 2/3 2/3

• As all xBi≥ 0, we obtain the optimal solution as

(x∗

1 = 5/3, x∗

2 = 0, x∗

3 = 1/3) and the optimal value as (−10/3).

SECOND ITERATION 44

Example 2 Use the dual simplex method to check that the following

LPP is infeasible

max −x1

subject to

x1 − x2 ≥ 3

−x1 + x2 ≥ 4

x1, x2 ≥ 0.

Solution The above problem changes to:

max −x1

subject to

−x1 + x2 ≤ −3

x1 − x2 ≤ −4

x1 ≥ 0, x2 ≥ 0

SECOND ITERATION 45

Taking x3 and x4 as the surplus variables:

max −x1

subject to

−x1 + x2 + x3 = −3

x1 − x2 + x4 = −4

x1, x2, x3, x4 ≥ 0

we get the following dual simplex tableaus

XB y1 y2 y3 y4

x3 = −3 -1 1 1 0

← x4 = −4 1 -1 0 1

0 1 0 0 0

↑

SECOND ITERATION 46

Taking x3 and x4 as the surplus variables:

max −x1

subject to

−x1 + x2 + x3 = −3

x1 − x2 + x4 = −4

x1, x2, x3, x4 ≥ 0

we get the following dual simplex tableaus

XB y1 y2 y3 y4

x3 = −3 -1 1 1 0

← x4 = −4 1 -1 0 1

0 1 0 0 0

↑

SECOND ITERATION 47

XB y1 y2 y3 y4

x3 = −3 -1 1 1 0

← x4 = −4 1 -1 0 1 −1×R2

0 1 0 0 0

↑

XB y1 y2 y3 y4

x3 = −3 -1 1 1 0 R1 + (−1)×R2

x2 = 4 -1 1 0 -1

0 1 0 0 0

XB y1 y2 y3 y4

x3 = −7 0 0 1 1

x2 = 4 -1 1 0 -1

0 1 1 0 0

SECOND ITERATION 48

XB y1 y2 y3 y4

x3 = −7 0 0 1 1

x2 = 4 -1 1 0 -1

0 1 1 0 0

Now x3 < 0 but in that row, no yrj < 0, hence the given LPP is

infeasible. Figure 1 verifies the same thing graphically.

2

1

-x + x = 421

x - x = 321

x

x

Figure 1:

SECOND ITERATION 49

9.1 lecture_2b

Documents