column generation
DESCRIPTION
Column Generation. By Soumitra Pal Under the guidance of Prof. A. G. Ranade. Agenda. Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions. 10. 5. 3. Cutting Stock Problem. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/1.jpg)
Column Generation
By
Soumitra Pal
Under the guidance of
Prof. A. G. Ranade
![Page 2: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/2.jpg)
Agenda
• Introduction• Basics of Simplex algorithm• Formulations for the CSP• (Delayed) Column Generation• Branch-and-price• Flow formulation of CSP & solution• Conclusions
![Page 3: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/3.jpg)
Cutting Stock Problem
• Given larger raw paper rolls
• Get final rolls of smaller widths
10
5 3
![Page 4: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/4.jpg)
Cutting Stock Problem (2)
• Raw width (W) = 10• No of finals given below
i Width (wi) Quantity (bi)
1 3 92 5 793 6 904 9 27
• Minimize total no of raws reqd. to be cut
![Page 5: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/5.jpg)
Agenda
• Introduction• Basics of Simplex algorithm• Formulations for the CSP• (Delayed) Column Generation• Branch-and-price• Flow formulation of CSP & solution• Conclusions
![Page 6: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/6.jpg)
5x1 - 8x2 ≥ -80 -5x1 - 4x2 ≥ -100-5x1 - 2x2 ≥ -80-5x1 + 2x2 ≥ -50
Min -x1 -x2Objective/
Cost fn
Constraints
![Page 7: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/7.jpg)
(10,0)
(13,5)
(12,10)
(8,15)
(0,10)
5x1 - 8x2 ≥ -80 -5x1 - 4x2 ≥ -100-5x1 - 2x2 ≥ -80-5x1 + 2x2 ≥ -50
![Page 8: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/8.jpg)
-x1-x2 = -10 -x1-x2 = -20 -x1-x2 = -30
Cost decreases in this direction
![Page 9: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/9.jpg)
(10,0)
(13,5)
(12,10)
(8,15)
(0,10)
Simplex method
![Page 10: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/10.jpg)
contraints of no. variablesof no.
0
:s.t.Min
1211
22222121
11212111
2211
ml
xbxaxaxa
bxaxaxabxaxaxa
xcxcxc
i
mlmlmm
ln
ln
ln
![Page 11: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/11.jpg)
0
:s.t.000Min
1211
222222121
111212111
212211
i
mnlmlmm
lln
lln
nllln
xbxxaxaxa
bxxaxaxabxxaxaxa
xxxxcxcxc
![Page 12: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/12.jpg)
0:s.t.
Min
xbAx
cx
![Page 13: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/13.jpg)
(10,0,130,50,30,0)
(13,5,110,10,0,0)
(12,10,60,0,0,10)
(8,15,0,0,10,40)
(0,10,0, 60,60,70)
(0,0,80, 100,80,50)
Basic & non-basic 5x1 - 8x2 ≥ -80 -5x1 - 4x2 ≥ -100-5x1 - 2x2 ≥ -80-5x1 + 2x2 ≥ -50
![Page 14: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/14.jpg)
'1'
'11'
1'1'
'
''
11
1
1
equation, previousin it Putting
variablebasic-non one increase corner,next out find To0
0 ,0
0min
NBB
NB
NB
N
B
B
BBB
B
B
BB
BNB
NxBxxNxBbBxbBNxBx
xx
x
bBx
NBI
bBcxccxbBxbBx
xbx
NBAx
xcccx
![Page 15: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/15.jpg)
NBccc
xNBcccxcx
xNBccxcxcxccxxcNxBxcxcxccx
BN
NBN
NBNBBNNBB
NNNBBNNBB
1
'1'
'1'''
''1'''
cost reduced called isbracket in termThe
)(costin Change
)()(
cost New
• If all elements of it are non-negative, it is already optimal• Otherwise choose the non-basic variable corresponding to
most negative value to enter the basic
![Page 16: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/16.jpg)
• How to find out outgoing basic variable?
ofcomponent kth
)( ofcomponent kth of Min value
0. becomes ofcomponent )k(say one until increased becan It .
variablebasic new with mutilpies that ofcolumn thebe Let
1
'th
1
1'1'
dxbB
xx
NBd
bBNxBx
B
B
i
NB
![Page 17: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/17.jpg)
Few steps of simplex algorithm
• Compute reduced cost• If all >= 0, optimal stop.• Otherwise choose the most negative
component• Corresponding variable enters basis• Compute the ratios for finding out leaving basis• Update basic variables and B for next step
NBcc BN1
'1'NBB NxBxx
![Page 18: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/18.jpg)
Smart way of doing
variableleavingabout info gives
gives, ofcolumn theis wherecost reduced and gives
gives
dx
dNaaBdyNcycyB
xbBx
B
NB
BB
![Page 19: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/19.jpg)
yd
Start
![Page 20: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/20.jpg)
yd
Reduced cost vector
![Page 21: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/21.jpg)
yd
Selection of non-basic variable
![Page 22: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/22.jpg)
yd
y vector
![Page 23: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/23.jpg)
yd
Selection of leaving basic variable
![Page 24: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/24.jpg)
Preparation for next step
yd
![Page 25: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/25.jpg)
yd
Entering variable in 2nd iteration
![Page 26: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/26.jpg)
yd
Leaving variable in 2nd iteration
![Page 27: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/27.jpg)
yd
Preparation for 3rd iteration
![Page 28: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/28.jpg)
yd
Final iteration
![Page 29: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/29.jpg)
Done so far
• Basics of (revised) Simplex algorithm– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
![Page 30: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/30.jpg)
Agenda
• Introduction• Basics of Simplex algorithm• Formulations for the CSP• (Delayed) Column Generation• Branch-and-price• Flow formulation of CSP & solution• Conclusions
![Page 31: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/31.jpg)
raw k fromcut widthi of finals of no.
otherwise 0used, is raw k if 1
widthafor index iraw afor index kwidthsdifferent of no.mraws available of no.K
,,1,,1integer and 0,,11or 0
,,1
,,1 :s.t.
Min
th
th
th
1
1
1
ik
k
ik
k
k
m
iiki
i
K
kik
K
kk
x
y
miKkxKky
KkWyxw
mibx
y
First step to solution - Formulation
Kantorovich formulation
![Page 32: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/32.jpg)
Kantorovich formulation example
1 2 3 4
1 ,4
3 ,1 ,10 ,1 ,4
21
132111
4231
k
kk
k xx
xxxyyyyK
![Page 33: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/33.jpg)
LP relaxation• Integer programming is hard• Convert it to a linear program
raw k fromcut widthi of finals of no.
otherwise 0used, is raw k if 1
widthafor index iraw afor index kwidthsdifferent of no.mraws available of no.K
,,1,,1integer and 0,,11or 0
,,1
,,1 :s.t.
Min
th
th
th
1
1
1
ik
k
ik
k
k
m
iiki
i
K
kik
K
kk
x
y
miKkxKky
KkWyxw
mibx
y
0 ≤ yk ≤ 1
![Page 34: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/34.jpg)
LP relaxation (2)
• LP relaxation is poor• For our example, optimal LP objective is 120.5• The optimal integer objective solution value is
157• Gap is 36.5• It can be as bad as ½ of the integer solution
• Can we get a better LP relaxation?
![Page 35: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/35.jpg)
jpattern fromcut widthi of finals of no.
jpattern in cut raws of no.patterns possible all ofset J
pattern cutting ajwidthsdifferent of no.m
widthafor index i
integer and 0
,,1 :s.t.
Min
th
ij
j
j
iJj
jij
Jjj
a
x
Jjx
mibxa
x
Another Formulation
Gilmore-GomoryFormulation
![Page 36: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/36.jpg)
2790799
000000001000000110000121000321100100
:9:6:5:3
9
8
7
6
5
4
3
2
1
4
3
2
1
xxxxxxxxx
wwww
Example formulation
3*1 + 5*0 + 6*1 + 9*0 = 9 ≤ 10
![Page 37: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/37.jpg)
LP relaxation of Gilmore-Gomory
• LP relaxation is better• For our example, 156.7• Integer objective solution, 157• Gap is 0.3• Conjecture: Gap is less than 2 for practical
cutting stock problems
![Page 38: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/38.jpg)
Issues
• The number of columns are exponential• Optimal value is still not integer
• Gilmore-Gomory proposed an ingenuous way of working with less columns
• Branch-and-price to solve the other problem
![Page 39: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/39.jpg)
Done so far
• Basics of (revised) Simplex algorithm– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
• Formulation of the CSP– LP relaxation– Bounds
![Page 40: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/40.jpg)
Agenda
• Introduction• Basics of Simplex algorithm• Formulations for the CSP• (Delayed) Column Generation• Branch-and-price• Flow formulation of CSP & solution• Conclusions
![Page 41: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/41.jpg)
Column Generation
• In pricing step of simplex algorithm one basic variable leaves and one non-basic variable enters the basis
• This decision is made from the reduced cost vector
• The component with most negative value determines the entering variable
yNcNBcc NBN or 1
![Page 42: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/42.jpg)
Column Generation (2)• In terms of columns, we need to find
• That is equivalent to finding a such that
• For cutting stock problem, cj=1, hence it is equivalent to
• With implicit constraint
• Pricing sub-problem
} ofcolumn a is |{min argj
Nayac jjj
} ofcolumn a is |)(min{ Aayaac
}1|max{ yaya
Wwa
![Page 43: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/43.jpg)
Column Generation (3)
AnyNew Columns?
STOP(LP Optimal)
SolveRestricted Master Problem
(RMP)
SolvePricing Problem
Update RMP withNew Columns
No
Yes
![Page 44: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/44.jpg)
2790799
000000001000000110000121000321100100
:9:6:5:3
9
8
7
6
5
4
3
2
1
4
3
2
1
xxxxxxxxx
wwww
Example formulation
![Page 45: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/45.jpg)
T
BB
a
aaaa
aaaa
yyB
xBC
0101findcan weg,programmin dynamicor n enumeratio Using
109653constraintknapsack Given
1.1.121
31 maximize toneed we
112/13/11111
2790
5.393
1/271/902/79
3/9
,
1000010000200003
,
2790799
4321
4321
![Page 46: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/46.jpg)
2781
5.399
271.990
5.399
27.905.39
,
1000010100200001
replaced is Bin column first ,9
*90*9*1/90*31/3 /
01031
3dt
t
xB
t
dx
daBd
B
TT
B
T
![Page 47: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/47.jpg)
5.15627815.399 Solution
optimalalready is Hence1 subproblem ofsolution get not can We
109653costraintknapsack Given
1.1.121.0 maximize toneed we
112/101111
2781
5.399
,
1000010100200001
4321
4321
B
B
x
aaaa
aaaa
yyB
xB
![Page 48: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/48.jpg)
Done so far• Basics of (revised) Simplex algorithm
– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
• Formulation of the CSP– LP relaxation– Bounds
• Delayed column generation– Restricted master problem– Sub-problem to generate column– Repeated till optimal
![Page 49: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/49.jpg)
Agenda
• Introduction• Basics of Simplex algorithm• Formulations for the CSP• (Delayed) Column Generation• Branch-and-price• Flow formulation of CSP & solution• Conclusions
![Page 50: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/50.jpg)
Integer solution
• A formulation of the problem which gives ‘good’ LP relaxation
• Solved the LP relaxation using column generation
• But, the LP relaxation is not the ultimate goal, we need integer solution
• In the example x2 is 39.5• One way is to rounding
![Page 51: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/51.jpg)
• What to do if there are multiple fractional variables?
• The method commonly followed is branch -and-bound technique of solving integer programs
• Basic fundae– create multiple sub-problems with extra
constraints– solve the sub-problems using column generation– take the best of solutions to the sub problems
![Page 52: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/52.jpg)
• Key is the rules of dividing into sub problems (this is called branching strategy)
• Things to look into– Branching rule does not destroy column
generation sub problem property– Efficiency of the process
• Ingenuity of the formulation and branching strategy
• Use of running bounds to prune (fathomed)• This technique is called branch-and-price• We see another formulation next
![Page 53: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/53.jpg)
Done so far• Basics of (revised) Simplex algorithm
– Formulation– Corners, basic variables, non-basic variables– How to move from corner to corners– Optimal value
• Formulation of the CSP– LP relaxation– Bounds
• Delayed column generation– Restricted master problem– Sub-problem to generate column– Repeated till optimal
• Branch-and-price– Basic fundae
![Page 54: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/54.jpg)
Agenda
• Introduction• Basics of Simplex algorithm• Formulations for the CSP• (Delayed) Column Generation• Branch-and-price• Flow formulation of CSP & solution• Conclusions
![Page 55: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/55.jpg)
Flow formulation
• Bin packing problem• Similar to the cutting stock problem
bin items
![Page 56: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/56.jpg)
w2 w2 w2 w2
w1 w1 w1
loss loss loss loss loss
0 1 2 3 4 5
w2 w2
0 1 2 3 4 5
loss
Example• Bin capacity W = 5• Item sizes – w1=3 and w2=2
![Page 57: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/57.jpg)
Formulation equations• Problem is equivalent to finding minimum
flow between nodes 0 & W
0integer 0integer
,,2,1
,
1,,2,1,00,
:s.t.
Min
)()(
zx
mdbxWjzWi
jzxx
z
ij
Awkkdwkk
Ajkjk
Aijij
d
d
![Page 58: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/58.jpg)
• Issue of symmetry in the graph• 3 rules to reduce symmetry
1. Arcs are specified according to decreasing width
2. A bin can never start with a loss3. Number of consecutive arcs corresponding
to a single item size must not exceed number of orders
![Page 59: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/59.jpg)
w2 w2 w2 w2
w1w1 w1
loss loss loss loss loss
01 2 3 4
5
Invalid as per rule 2
Invalid as per rule 1
w2 w2 w2
w1
loss loss loss0 1 2 3 4 5
![Page 60: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/60.jpg)
Branch-and-price
• The sub-problem finds an longest unit flow path where cost of each arc is given by y vector
• Branching heuristics– Xij >= ceil(xij)– Xij <= floor(xij)
![Page 61: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/61.jpg)
Few other points
• Loss constraints from the LP relaxation value is equated with the sum of loss variables
• With the above constraint it can be shown that the solution will be always integral
• The algorithm uses this fact by incrementally adding the lower bound
• This is done finite no. of times• No of new constraints are added is finite• Algorithm runs in pseudo-polynomial time
![Page 62: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/62.jpg)
Conclusions & Future work
• Column generation is a success story in large scale integer programming
• The flexibility to work with a few columns make real life problems tractable
• Ingenuous formulations and branching heuristics are used to improve efficiency
• Need to explore more applications with different heuristics
![Page 63: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/63.jpg)
Acknowledgements
• Valerio de Carvalho for borrowing some of his explanations exactly
• http://www-fp.mcs.anl.gov/otc/Guide/CaseStudies/simplex/applet/SimplexTool.html for the java applet
• AMPL & CPLEX tools• My guide
![Page 64: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/64.jpg)
![Page 65: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/65.jpg)
Thank you!
![Page 66: Column Generation](https://reader035.vdocument.in/reader035/viewer/2022081507/5681675e550346895ddc2ef2/html5/thumbnails/66.jpg)