sensitivity analysis and duality in lp · duality formulation of the primal-dual problems duality...

Sensitivity Analysis and Duality in LP

Xiaoxi Li

EMS & IAS, Wuhan University

Oct. 13th, 2016 (week vi)

Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 1 / 35

Organization of this lecture

Contents:

Sensitivity Analysis

Shadow price and reduced costChange in the RHS constants (resource’s capacity)Change in objective coefficients (unit revenue)

Duality

Formulation of the Primal-Dual problemsDuality properties: weak, strong, and the complementary slackness


Sensitivity Analysis

Question after solving a LP problem:

How does the optimal solution / value change in response tomodification of LP model data?In what range of the LP model data is an optimal solution robust?


Motivation for these questions:

allowance for certain error in modelling phase;management decisions in response to market or technologyadjustment: i.e. whether to

purchase/rent additional resource(s) if available;open a new production line if possible;modify the production plan facing changing in demand (unitrevenue).


An example under consideration:

Maximize z = 5x1 +4.5x2 +6x3

s.t.

6x1 + 5x2 + 8x3 ≤ 60 1

10x1 +20x2 +10x3 ≤ 150 2x1 ≤ 8 3

x1, x2, x3 ≥ 0 +

(0.1)

After applying the simplex method, we obtain the final simplex tableau:

Eq. Basic variables z x1 x2 x3 s1 s2 s3 RHS(0) z 1 0 0 4

71114

135 0 51 3

7(1) x2 0 0 1 −2

7 −17

335 0 4 2

7(2) s3 0 0 0 −11

7 −27

114 1 11

7(3) x1 0 1 0 11

727 - 1

14 0 6 37

Basic variables: {x2,s3,x1}; Nonbasic variables: {x3,s1,s2}.


Shadow price

Question: how would the optimal value change if we increase one unitof Resource (1), (2) or (3) ?

At the optimal solution,s1 and s2 are nonbasic variables, equal to zero: both Constraint (1)and Constraint (2) bind and both resources are exhausted.s3 is basic variable, equal to 11/7: Resource (3), which is the limitproduction for Activity 1, has 11/7 units un-used.

Intuitively, increasing or decreasing the capacity of Resource (3) by alittle bit will have no effect on the optimal solution/value; whileincreasing or decreasing the capacity of Resource (1) or Resource (2)even by a little bit will change the optimal solution/value.

The Shadow price for a resource/constraint is the changing rate of theoptimal value in the resource capacity when the modification is small.


Computation of Shadow price

A nice property : there is no need to re-do the simplex method so as toobtain the shadow prices; rather, they can be red directly from the final(optimal) simplex tableau.

Claim. The shadow price for a resource/functional constraint is theassociated slack variable’s coefficient in Eq (0) of the final (optimal)tableau.

for a resource un-exhausted at optimum (basic variable), it is zero;

for a resource exhausted at optimum (nonbasic variable), it is positive.

For the example. y1 =1114 , y2 =

135 , y3 = 0.


Shadow price

Claim. The shadow price for a resource/constraint is the associatedslack variable’s coefficient in Eq (0) of the final (optimal) tableau.

Proof.Consider an increase of small ∆ in Resource (i)’s capacity, i.e. theLP is the same as before except that bi is replaced by bi +∆.w.l.o.g, we assume no tie during the application of the simplexmethod to the original LP problem.For |∆| small, the same operations of the simplex method appliesfor the perturbed problem also (why?).The obtained final (optimal) simplex tableau has the same entriesas the the unperturbed one except for the RHS vector, which willbe the sum of the unperturbed RHS vector andan "additional vector corresponding the coefficients for ∆".The "additional vector corresponding the coefficients for ∆" is thesame as the column for si.


To illustrate, consider in the example that b1 = 60 is replaced by 60+∆.

Eq. Basic variables z x1 x2 x3 s1 s2 s3 RHS ∆

(0) z 1 -5 -4.5 -6 0 0 0 0 + 0(1) s1 0 6 5 8 1 0 0 60 + 1(2) s2 0 10 20 10 0 1 0 150 + 0(3) s3 0 1 0 0 0 0 1 8 + 0

Eq. Basic variables z x1 x2 x3 s1 s2 s3 RHS ∆

(0) z 1 0 0 47

1114

135 0 51 3

7 + 1114

(1) x2 0 0 1 −27 −1

7335 0 4 2

7 + −17

(2) s3 0 0 0 −117 −2

7114 1 11

7 + −27

(3) x1 0 1 0 117

27 - 1

14 0 6 37 + 2

7

The initial and the final (optimal) simplex tableaux.

s1 and ∆ have the same column vector along the computation.

The final vector defines the changing rates of optimal value/solution.

The same operations work for s3 but the changing rates for optimalvalue/solution are all zero (check).


Interpretations of the shadow price:

Marginal worth to the LP problem of each resource.For an example, if some additional capacity of Resource (i) isavailable for purchasing with a unit price ui. Then it will not beprofitable if this price is beyond its shadow price, i.e. ui > yi.


Reduced cost

Above we have defined and analysed the shadow price for eachfunctional constraint. The shadow price for a nonnegativeconstraint has an analogous meaning.It is termed the "reduced cost" due to the fact: it is connected by"≥’ rather than "≤", so an increase on the RHS entry (which iszero) means a restriction on xj, thus incurring a cost.

Claim. The reduced cost for an activity/nonnegative constraint is thenegative of the associated decision variable’s coefficient in Eq (0) ofthe final (optimal) tableau.

for an activity of positive production at optimum (basic), it is zero;

for a resource of zero production at optimum (nonbasic), it is positive.

Proof. The similar computation as that for the shadow price, except for anattention paid on the opposite sign.



71114

135 0 51 3

7(1) x2 0 0 1 −2

7 −17

335 0 4 2

7(2) s3 0 0 0 −11

7 −27

114 1 11

7(3) x1 0 1 0 11

727 - 1

14 0 6 37

For the example. c1 = 0, c2 = 0, c3 =−47 .

Illustrations.

At optimum, x3 = 0 thus Activity (3) is not produced. By forcing aproduction of ∆ > 0 unit of Activity (3), a loss of 4

7 in the optimal value isinduced.

At optimum, x1 = 6 37 > 0 and x2 = 4 2

7 > 0 thus both Activity (1) and Activity(2) are produced. By forcing a production of (small) ∆ > 0 unit of eitherActivity (1) or Activity (2), a loss of zero in the optimal value is induced.


A different way of computing Activity (j)’s reduced cost cj involving theshadow prices {yi} and its unit revenue cj; (pricing out an activity )

reduced cost (net unit revenue) = unit revenue – "marginal cost".

cj = cj−∑i aijyi.

Interpretations of the reduced cost.

The reduced cost measures the unit worth (cost if it is negative) ofopening a new production line.Consider a LP without Activity (3): it has the same optimal solutionstructure (basis, shadow prices...) as when Activity (3) is allowed.

A negative reduced cost for Activity (3) implies that there is no needto consider open this production line;On the other hand, if cj is large enough so cj = cj−∑i aijyi > 0, thenit should consider open this line.


Change in the RHS constants: the optimum range

The shadow price/reduced cost measures the changing rate of theoptimal value in the RHS constant when the change is small.When the change of the RHS constant could be significant, weconsider the optimum range within which theoptimal basis remains the same.


For |∆| small, the obtained final simplex tableau after adding ∆ unit ofResource (1) is:


71114

135 0 51 3

7 +1114 ∆

(1) x2 0 0 1 −27 −1

73

35 0 4 27 −

17 ∆

(2) s3 0 0 0 −117 −2

71

14 1 117 −

27 ∆

(3) x1 0 1 0 117

27 - 1

14 0 6 37 +2

7 ∆

To have the above tableau still optimal thus the optimal basis {x2,s3,x1}remains the same, we should have:

x2 = 4 27−

17 ∆ ≥ 0⇐⇒ ∆≤ 30.

s3 =117 −

27 ∆ ≥ 0⇐⇒ ∆≤ 11

2 .

x1 = 6 37+

27 ∆ ≥ 0⇐⇒ ∆≥−45

2 .

Thus the optimum range is: −452 ≤ ∆≤ 11

2 and 752 ≤ bnew

1 ≤ 1312 .

Question: compute the optimum range for Resource (2) & (3), Activity 1, 2, 3.


Change in the objective coefficients: the optimumrange

Generally, change in the objective coefficients is less sensitive tothe optimal solution/value than the change in the RHS constants.Consider no "shadow price" here, but only the "optimum range".The optimum range for Activity j is the range of ∆ such that theoptimal solution remains the same after cj is replaced by cj +∆.Consider a change in Activity j’s coefficient from cj to cj +∆.In the perturbed LP problem, applying the same simplex-methodoperations as unperturbed will give a "final" simplex tableaualmost the same as before, except for xj’s coefficient in Eq. (0).


To illustrate, we take the example and distinguish two types of activitiesat optimum of the unperturbed problem.

x3 is a nonbasic variable.

The final tableau after operations (assure yourself that it is like this).


7−∆1114

135 0 51 3

7(1) x2 0 0 1 −2

7 −17

335 0 4 2

7(2) s3 0 0 0 −11

7 −27

114 1 11

7(3) x1 0 1 0 11

727 - 1

14 0 6 37

To keep the same optimal solution unchanged, we should have:

47−∆≥ 0⇐⇒ ∆≤ 4

7, thus cnew

1 ≤ 6+47= 6

47.

Question. Show that Activity 3’s optimum range is such that the new reducedcost for it remains nonnegative.


x1 or x2 is a basic variable.

The final tableau after operations (assure yourself that it is like this).

Eq. Basic variables z x1 x2 x3 s1 s2 s3 RHS(0) z 1 −∆ 0 4

71114

135 0 51 3

7(1) x2 0 0 1 −2

7 −17

335 0 4 2

7(2) s3 0 0 0 −11

7 −27

114 1 11

7(3) x1 0 1 0 11

727 - 1

14 0 6 37

The above tableau is not optimal for ∆ 6= 0 (since x1 should be a basicvariable!), a further operation "Eq.(0)’=Eq.(0)+3 Eq.(3)" is needed.

Eq. Basic z x1 x2 x3 s1 s2 s3 RHS(0) z 1 0 0 4

7+117 ∆

1114+

27 ∆

135−

114 ∆ 0 51 3

7+6 37 ∆

(1) x2 0 0 1 −27 −1

73

35 0 4 27

(2) s3 0 0 0 −117 −2

71

14 1 117

(3) x1 0 1 0 117

27 - 1

14 0 6 37


Now to have the optimal solution remain the same, we should have:47+

117 ∆ ≥ 0 ⇐⇒ ∆≥− 4

11 .1114+

27 ∆ ≥ 0 ⇐⇒ ∆≥−11

4 .135−

114 ∆ ≥ 0 ⇐⇒ ∆≤+2

5 .

Thus− 4

11≤ ∆≤ 2

5,

and4

711≤ cnew

1 = c1 +∆≤ 525.

Ex. Compute the optimum range for Activity 2.


Duality in LP

optimal dual variable yi = primal shadow price of Constraint (i);

optimal primal variable xj = dual shadow price of Constraint (j).Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 20 / 35

The shadow price and the duality

The shadow prices yi = optimal multipliers of constraints, which

measure the marginal worth of an additional unit of resource;can be used for pricing out an activity : Activity i’s net marginalrevenue (reduced cost) is computed as cj = cj−∑j aijyj.can be seen as the internal pricing mechanism within the firm forthe allocation of scared resources to competing activities.

Duality is a unifying theory that help develops the relationship betweena given LP problem (the primal) and another one (the dual) stated interms of variables with a shadow-price interpretation. This theory

helps further understanding of the optimal multipliers in terms ofshadow-price interpretation;

helps take the advantage of the computational efficiency of the dualproblem (over the primal one).


Back to this example we have studied before:

Maximize z = 5x1 +4.5x2 +6x3

s.t.

6x1 + 5x2 + 8x3 ≤ 60 1

10x1 +20x2 +10x3 ≤ 150 2x1 ≤ 8 3

x1, x2, x3 ≥ 0 +

After applying the simplex method, we obtain the final simplex tableau:

Eq. Basic z x1 x2 x3 s1 s2 s3 RHS(0) z 1 0 0 4

71114

135 0 51 3

7(1) x2 0 0 1 −2

7 −17

335 0 4 2

7(2) s3 0 0 0 −11

7 −27

114 1 11

7(3) x1 0 1 0 11

727 - 1

14 0 6 37

Basic variables: {x2,s3,x1}; Nonbasic variables: {x3,s1,s2}.


Observations/properties of the shadow prices yi.

Value and interpretation.Positive shadow price Zero shadow pricey1 =

1114 , y2 =

135 y3 = 0

s1 and s2 nonbasic s3 basicConstraint (1) & (2) bind Constraint (3) does not bindResource (1) & (2) exhausted Resource (3) has unused part

Pricing out the activities (cj = cj−∑i aijyi ≤ 0).

Zero reduced cost Negative reduced costc1 = c2 = 0 c3 =− 4

7x1, x2 basic x3 nonbasicActivity (1) & (2) produced Activity (3) not produced(1),(2): unit revenue = marginal cost (3): unit revenue < marginal cost

Opportunity cost for consuming firm’s resources (v = ∑i biyi).

v = 60× 1114 +150× 1

35 +8×0 = 3607 = 51 3

7v = z∗ = ∑j cjx∗j the optimal value of the (primal) LP problem.


Imagine that the firm does not own these capacities of resourcesbut has to rent them for the production (=⇒ minimize the rents).Then a dual LP problem, characterized by the above summarizedproperties of shadow prices, can be formulated as follows.

Minimize v = 60y1 +150y2 +8y3

s.t.

6y1 + 10y2 + y3 ≥ 5 (1)5y1 + 20y2 ≥ 4.5 (2)8y1 + 10y2 ≥ 6 (3)y1, y2, y3 ≥ 0 (+)


Formulation of the primal-dual problem


Properties of LP duality

We consider the (P)-(D) formulation in standard form and write themin compact forms.

Let c,x ∈ Rn, A ∈ Rm×n, b,y ∈ Rm.

(P) Maxx z = cTx

s.t. Ax≤ b, x≥ 0.

(D) Miny v = bTy

s.t. ATy≥ c, y≥ 0.

Verify. The dual of the dual problem (D) is the primal problem (P).


Weak Duality Property

Theorem (Weak Duality Property)Let x ∈ Rn be a feasible solution to (P) and let y ∈ Rm be a feasiblesolution to (D). Then

cT x≤ yTAx≤ bT y.

Moreover, if cT x = bT y, then x solves (P) and y solves (D).

CorollaryIf (P) is unbounded, then (D) is infeasible; if (D) is unbounded, then(P) is infeasible.

Proof.in (P): Ax≤ b, y≥ 0 =⇒ yTAx≤ bT y;in (D): AT y≥ c, x≥ 0 =⇒ xTAT y≥ cT x.


Strong Duality Property

Theorem (Strong Duality Property)If either (P) or (D) has a finite optimal solution, then so does theother, and the optimal values are equal.

Proof.Assume that (P) has a finite optimal solution x∗ and let z∗ = cTx∗ be itsoptimal value. Let y∗ ∈ Rm be the associated shadow price vector for (P)’sfunctional constraints. Properties of the shadow prices:

reduced costs are nonpositive: c = c−ATy∗ ≤ 0;

shadow prices are nonnegative: y∗ ≥ 0;

resources’ opportunity cost is equal to value of (P): bTy∗ = z∗.

The Weak Duality Property implies that bTy∗ = z∗ = cTx∗ ≤ bT y for any feasibley to (D). Now we have shown that y∗ is feasible, so it is optimal to (D).The other direction is the same since the dual of the dual is the primal.


The Complementary Slackness Conditions

CorollaryLet (x,y) be a pair of feasible solutions to (P)-(D). Then (x,y) solves(P)-(D) iff. cTx = yTAx = bTy (]).

Proof. Weak Duality Property ("if") + Strong Duality Property ("only if").

Theorem (Complementary Slackness Conditions)Let (x,y) be a pair of feasible solutions to (P)-(D). Then (x,y) solves(P)-(D) iff.

either xj = 0 or ∑i aijyi = cj for j = 1, ...,n;either yj = 0 or ∑j aijxi = bi for i = 1, ...,m.

Proof.(]): xT(ATy− c) = ∑j xj

(∑i aijyi− ci

)= 0;

feasibility: xj ≥ 0 and ∑i aijyi− ci ≥ 0 for j = 1, ...,n.


The Complementary Slackness Conditions

The following corollary can be used to check whether a given feasiblesolution solves the LP problem or not.

Corollary (Complementary Slackness Conditions)Let x ∈ Rn be a feasible solution to (P). Then x solves (P) iff. thereexists a vector y ∈ Rm feasible for D) such that the following holds:

(i)for each i = 1, ...,m, if ∑j aijxj < bi, then yi = 0, and(ii) for each i = 1, ...,n, if xj > 0 then ∑i aijyi = cj.

Proof.The "only if " part is implied by the Theorem (ComplementarySlackness Conditions).To see the the "if " part, observe in the Theorem (Weak DualityProperty) the inequality "cTx≤ yTAx≤ bTy".


The Complementary Slackness Conditions: anexample

We use the Corollary (Complementary Slackness Conditions) to checka particular point x = (4,3) is optimal to the LP problem:

Maximize z = 3x1 +5x2

s.t.

x1 ≤ 4 1

2x2 ≤ 12 23x1 +2x2 ≤ 18 3x1, x2 ≥ 0 +

(0.2)

From item (i) in Corollary (Complementary Slackness Conditions):Constraint 1 : 4 = 4 =⇒ no condition on y1

Constraint 2 : 2×3 = 6 < 12 =⇒ y2 = 0

Constraint 3 : 3×4+2×3 = 18 =⇒ no condition on y3


The Complementary Slackness Conditions: anexample

From item (ii) in Corollary (Complementary Slackness Conditions):{Activity 1 : x1 = 4 > 0 =⇒ y1 +0y2 +3y3 = 3Activity 2 : x2 = 3 > 0 =⇒ 0y1 +2y2 +2y3 = 5

Putting the above three equations of (y1,y2,y3) together:Constraint 2 : y2 = 0Activity 1 : y1 +3y3 = 3Activity 2 : 2y2 +2y3 = 5

We solved that (y1,y2,y3) = (−92 ,0,

52), which is not feasible for dual

problem of LP (0.2) since y1 =−92 < 0. This implies that (4,3) is not

optimal for the primal LP (0.2).


sensitivity analysis and duality in lp · duality formulation of the primal-dual problems duality...

Documents