column generation - cslog.uni-bremen.de filehands-on tutorial on optimization f. eberle, r. hoeksma,...

Hands-on Tutorial on OptimizationF. Eberle, R. Hoeksma, and N. Megow

September 28, 2018

Column Generation

Cutting Stock

Problem: Cutting StockGiven: Length W ∈ N

n orders withdemand di ∈ N and alength wi ∈ N with li ≤ L

Task: Minimize the number of rolls of length Wwhile satisfying the demand

Interactive: A First LP for Cutting Stock

Definition (Cutting Pattern)A cutting pattern is a vector x ∈ Nn

0, where xi denotes the number ofshort rolls of length wi cut out of a long roll.

K :=∑n

i=1 di : upper bound on the number of long rollsBinary variables y (k) indicating whether roll k is neededCutting pattern x (k) for roll k

Formulate an ILP for the cutting stock problem!




K :=∑n

i=1 di : upper bound on the number of long rolls

Binary variables y (k) indicating whether roll k is neededCutting pattern x (k) for roll k





K :=∑n

i=1 di : upper bound on the number of long rollsBinary variables y (k) indicating whether roll k is needed

Cutting pattern x (k) for roll k





K :=∑n

i=1 di : upper bound on the number of long rollsBinary variables y (k) indicating whether roll k is neededCutting pattern x (k) for roll k



minK∑

k=1y (k)

s.t.K∑

k=1x (k)

i ≥ di for all i = 1, . . . , n (demand)

K∑k=1

wix (k)i ≤ y (k)W for all k = 1, . . . , K (valid pattern)

y (k) ∈ {0, 1} for all k = 1, . . . , Kx (k) ∈ Zn for all k = 1, . . . , K

Problems with this formulation?

Modeling: Enumerate all possibleStrength of LP relaxation: integrality gap ≥ 2

Symmetry:I Cutting patternsI Distribution of the unused large rolls

Observation: Problems in cutting plane and branch & boundapproaches.

⇒ Try a different ILP formulation.

Interactive: A Second LP for Cutting Stock

Q := {x ∈ Nn0 :

∑ni=1 wixi ≤W }

Huge!Enumerate Q = {x (1), x (2), . . . , x (|Q|)}.Variable u(j) denotes how often cutting pattern x (j) ∈ Q is used.



Q := {x ∈ Nn0 :

∑ni=1 wixi ≤W } Huge!

Enumerate Q = {x (1), x (2), . . . , x (|Q|)}.Variable u(j) denotes how often cutting pattern x (j) ∈ Q is used.



min|Q|∑j=1

u(j)

s.t.|Q|∑j=1

x (j)i u(j) ≥ di for all i = 1, . . . , n (demand)

u(j) ∈ N0 for all j = 1, . . . , |Q|

Benefits: Integrality gap is significantly reduced because the cuttingpatterns themselves cannot be fractional anymore.

Problems: Generating Q is insane.

⇒ Recall polyhedral description of LPs!

Idea of Column Generation

Observations:1. In any vertex u, |Q| inequalities have to be tight.

2. There are only n nontrivial inequalities.

⇒ Only very few variables u(j) can ever be positive.⇒ Many have to be of the type u(j) = 0.

Guess the correct choice of the non-zero variables,i.e., restrict to a smaller subset Q′ ⊂ Q.

How do we check if our guess Q′ is already a solution over Q?How do we enlarge Q′ if it is suboptimal?Duality theory


Observations:1. In any vertex u, |Q| inequalities have to be tight.2. There are only n nontrivial inequalities.








How do we check if our guess Q′ is already a solution over Q?How do we enlarge Q′ if it is suboptimal?

Duality theory

The Cutting Stock LP and its Dual

min|Q|∑j=1

u(j)

∣∣∣∣ max maxn∑

i=1dizi

s.t.|Q|∑j=1

x (j)i u(j) ≥ di ∀ i ∈ [n]

∣∣∣∣ zi s.t.n∑

i=1x (j)

i zi ≤ 1 ∀ j ∈ [|Q|]

u(j) ≥ 0 ∀ j ∈ [|Q|]

∣∣∣∣ ≤ zi ≥ 0 ∀ i ∈ [n]

Restrict to Q′.


min|Q|∑j=1

u(j)∣∣∣∣ max

maxn∑

i=1dizi

s.t.|Q|∑j=1

x (j)i u(j) ≥ di ∀ i ∈ [n]

∣∣∣∣ zi

s.t.n∑

i=1x (j)

i zi ≤ 1 ∀ j ∈ [|Q|]

u(j) ≥ 0 ∀ j ∈ [|Q|]∣∣∣∣ ≤

zi ≥ 0 ∀ i ∈ [n]

Restrict to Q′.


min|Q|∑j=1

u(j)∣∣∣∣ max max

n∑i=1

dizi

s.t.|Q|∑j=1

x (j)i u(j) ≥ di ∀ i ∈ [n]

∣∣∣∣ zi

s.t.n∑

i=1x (j)

i zi ≤ 1 ∀ j ∈ [|Q|]

u(j) ≥ 0 ∀ j ∈ [|Q|]∣∣∣∣ ≤

zi ≥ 0 ∀ i ∈ [n]

Restrict to Q′.


min|Q|∑j=1


n∑i=1

dizi

s.t.|Q|∑j=1

x (j)i u(j) ≥ di ∀ i ∈ [n]

∣∣∣∣ zi s.t.n∑

i=1x (j)

i zi ≤ 1 ∀ j ∈ [|Q|]

u(j) ≥ 0 ∀ j ∈ [|Q|]∣∣∣∣ ≤

zi ≥ 0 ∀ i ∈ [n]

Restrict to Q′.


min|Q′|∑j=1


n∑i=1

dizi

s.t.|Q′|∑j=1

x (j)i u(j) ≥ di ∀ i ∈ [n]

∣∣∣∣ zi s.t.n∑

i=1x (j)

i zi ≤ 1 ∀ j ∈ [|Q′|]

u(j) ≥ 0 ∀ j ∈ [|Q′|]∣∣∣∣ ≤ zi ≥ 0 ∀ i ∈ [n]

Restrict to Q′.

Recap: Duality Theory

I Basic primal solution corresponds to a basic dual solutionI They can be derived by complementary slacknessI Q′ ⊂ Q corresponds to a restriction of the dual constraints.I A primal/dual pair is optimal iff

1. their objective values agree (in this case always true) and2. both solutions are feasible.

Idea: To check if a solution over Q′ is optimal over Q as well, we willcalculate the corresponding dual solution for the restricted LP. Then,we check all dual constraints for feasibility.If all constraints are feasible, we found an optimal solution.Else, we add the violated constraint j for some cutting pattern x (j) tothe dual which corresponds to adding the variable u(j) to the primal.




Idea: To check if a solution over Q′ is optimal over Q as well, we willcalculate the corresponding dual solution for the restricted LP. Then,we check all dual constraints for feasibility.

If all constraints are feasible, we found an optimal solution.Else, we add the violated constraint j for some cutting pattern x (j) tothe dual which corresponds to adding the variable u(j) to the primal.




Idea: To check if a solution over Q′ is optimal over Q as well, we willcalculate the corresponding dual solution for the restricted LP. Then,we check all dual constraints for feasibility.If all constraints are feasible, we found an optimal solution.Else, we add the violated constraint j for some cutting pattern x (j) tothe dual which corresponds to adding the variable u(j) to the primal.

Finding a violated constraint

Let z∗ be a dual solution.Finding j ∈ Q such that

n∑i=1

x (j)i z∗i − 1 > 0.

is equivalent to finding

m∗ := maxj∈Q

n∑i=1

x (j)i z∗i

and checking whether m∗ > 1.

The Cutting Stock Pricing Problem

Let z∗ be a dual solution.

maxn∑

i=1z∗i xi

s.t.n∑

i=1wixi ≤W

x ∈ N0

Observation: No need to completely enumerate Q.The Pricing problem should be efficiently solvable.

If the optimal objective is greater than 0, we have found a cuttingpattern that violates a dual inequality.

Column Generation: General Idea

1. Start with an LP and its dualmax cT xs.t. Ax ≤ b

x ≥ 0(P)

min bT ys.t. AT y ≥ c

y ≥ 0(D)

Restrict to J ′ ⊂ [n] to obtain the reduced master problem

max (c ′)T x ′

s.t. A′x ′ ≤ bx ′ ≥ 0

(P’)min bT ys.t. (A′)T y ≥ c ′

y ≥ 0(D’)

2. Solve (P’) and (D’) to get an optimal primal/dual pair (x ′, y).

Column Generation: General Idea

3. Find a violated inequality of (D’) by solving the pricing problem

maxj

cj − a(j)y =: z

s.t. a(j) is the j-th column of A(Π)

4. If z > 0, add xj to (P’) and go to 2.Otherwise, set x ′j = 0 if j /∈ J ′ and output (x ′, y).

An Example: Parameters

Parameters:

W = 70 w1 = 17 d1 = 6w2 = 15 d2 = 2w3 = 63 d3 = 1

Q′ = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}

An Example: Step 1

Reduced set Q′ = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}

min u1 + u2 + u3s.t. u1 ≥ 6

u2 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

z ≥ 0

Optimal solution u = (6, 2, 1) with dual z = (1, 1, 1).Pricing problem: max x1 + x2 + x3 − 1

s.t. 17x1 + 15x2 + 63x3 ≤ 70x ∈ N0

Possible solution x = (4, 0, 0)

An Example: Step 1

Reduced set Q′ = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}min u1 + u2 + u3s.t. u1 ≥ 6

u2 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

z ≥ 0


s.t. 17x1 + 15x2 + 63x3 ≤ 70x ∈ N0


An Example: Step 1


u2 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

z ≥ 0

Optimal solution u = (6, 2, 1) with dual z = (1, 1, 1).

Pricing problem: max x1 + x2 + x3 − 1s.t. 17x1 + 15x2 + 63x3 ≤ 70

x ∈ N0


An Example: Step 1


u2 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

z ≥ 0


s.t. 17x1 + 15x2 + 63x3 ≤ 70x ∈ N0


An Example: Step 2

Reduced set Q′ = {(1, 0, 0), (0, 1, 0), (0, 0, 1), (4, 0, 0)}

min u1 + u2 + u3 + u4s.t. u1 + 4u4 ≥ 6

u2 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

4z1 ≤ 1z ≥ 0

Optimal solution u = (0, 2, 1, 64) with dual z = (1

4 , 1, 1).Pricing problem: max 1

4x1 + x2 + x3 − 1s.t. 17x1 + 15x2 + 63x3 ≤ 70

x ∈ N0


An Example: Step 2

Reduced set Q′ = {(1, 0, 0), (0, 1, 0), (0, 0, 1), (4, 0, 0)}

min u1 + u2 + u3 + u4s.t. u1 + 4u4 ≥ 6

u2 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

4z1 ≤ 1z ≥ 0


4 , 1, 1).

Pricing problem: max 14x1 + x2 + x3 − 1

s.t. 17x1 + 15x2 + 63x3 ≤ 70x ∈ N0


An Example: Step 2

Reduced set Q′ = {(1, 0, 0), (0, 1, 0), (0, 0, 1), (4, 0, 0)}

min u1 + u2 + u3 + u4s.t. u1 + 4u4 ≥ 6

u2 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

4z1 ≤ 1z ≥ 0


4 , 1, 1).Pricing problem: max 1

4x1 + x2 + x3 − 1s.t. 17x1 + 15x2 + 63x3 ≤ 70

x ∈ N0


An Example: Step 3Reduced set Q′ = {(1, 0, 0), (0, 1, 0), (0, 0, 1), (4, 0, 0), (0, 4, 0)}

min u1 + u2 + u3 + u4 + u5s.t. u1 + 4u4 ≥ 6

u2 + 4u5 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

4z1 ≤ 14z2 ≤ 1

z ≥ 0

Optimal solution u = (0, 0, 1, 64 , 1

2) with dual z = (14 , 1

4 , 1).Pricing problem: max 1

4x1 + 14x2 + x3 − 1

s.t. 17x1 + 15x2 + 63x3 ≤ 70x ∈ N0

End


min u1 + u2 + u3 + u4 + u5s.t. u1 + 4u4 ≥ 6

u2 + 4u5 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

4z1 ≤ 14z2 ≤ 1

z ≥ 0


2) with dual z = (14 , 1

4 , 1).

Pricing problem: max 14x1 + 1

4x2 + x3 − 1s.t. 17x1 + 15x2 + 63x3 ≤ 70

x ∈ N0

End


min u1 + u2 + u3 + u4 + u5s.t. u1 + 4u4 ≥ 6

u2 + 4u5 ≥ 2u3 ≥ 1

u ≥ 0

max 6z1 + 2z2 + z3s.t. z1 ≤ 1

z2 ≤ 1z3 ≤ 1

4z1 ≤ 14z2 ≤ 1

z ≥ 0


2) with dual z = (14 , 1

4 , 1).Pricing problem: max 1

4x1 + 14x2 + x3 − 1

s.t. 17x1 + 15x2 + 63x3 ≤ 70x ∈ N0

End

column generation - cslog.uni-bremen.de filehands-on tutorial on optimization f. eberle, r. hoeksma,...

Documents