optimization bounds from binary decision diagramspublic.tepper.cmu.edu/jnh/bddboundscp2014.pdf ·...
TRANSCRIPT
![Page 1: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/1.jpg)
Optimization Bounds from
Binary Decision Diagrams
J. N. Hooker
Joint work with
David Bergman, André Ciré, Willem-Jan van Hoeve
Carnegie Mellon University
Journal Presentation Track
CP 2014, Lyon, France
![Page 2: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/2.jpg)
Based on…
D. Bergman, A.A. Ciré, W.-J. van Hoeve, J.N. Hooker,
Optimization bounds from binary decision diagrams,
INFORMS Journal on Computing 26 (2014) 253–268.
![Page 3: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/3.jpg)
Binary Decision Diagrams
• BDDs historically used for circuit design and verification.
– Lee 1959, Akers 1978, Bryant 1986.
![Page 4: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/4.jpg)
Binary Decision Diagrams
• BDDs historically used for circuit design and verification.
– Lee 1959, Akers 1978, Bryant 1986.
• Compact graphical representation of boolean function.
– Can also represent feasible set of problem with binary
variables.
– Slight generalization (MDDs) represents finite domain
variables.
![Page 5: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/5.jpg)
Binary Decision Diagrams
• BDDs historically used for circuit design and verification.
– Lee 1959, Akers 1978, Bryant 1986.
• Compact graphical representation of boolean function.
– Can also represent feasible set of problem with binary
variables.
– Slight generalization (MDDs) represents finite domain
variables.
• Reduced BDD is result of superimposing isomorphic
subtrees in a search tree.
– Unique reduced BDD for given variable ordering.
![Page 6: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/6.jpg)
A little history…
• First CP workshop was in 1993 (first conference 1995)
– I was there! My paper competed with BDDs.
• Logic circuit verification by Benders decomposition
The Newport Papers: Proceedings of PPCP 1993
![Page 7: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/7.jpg)
A little history…
• First CP workshop was in 1993 (first conference 1995)
– I was there! My paper competed with BDDs.
• Logic circuit verification by Benders decomposition
The Newport Papers: Proceedings of PPCP 1993
• First International Conference on AI and OR (1995)
– Organizers: M. Ginsberg, JNH
– Several CP papers
![Page 8: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/8.jpg)
A little history…
• First CP workshop was in 1993 (first conference 1995)
– I was there! My paper competed with BDDs.
• Logic circuit verification by Benders decomposition
The Newport Papers: Proceedings of PPCP 1993
• First International Conference on AI and OR (1995)
– Organizers: M. Ginsberg, JNH
– Several CP papers
• First CPAIOR workshop (1999)
– Organizers: F. Focacci, A. Lodi, M. Milano, D. Vigo
![Page 9: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/9.jpg)
Binary Decision Diagrams
• Goal: derive bound on optimal value from BDD.
– Good bounds are essential to optimization.
![Page 10: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/10.jpg)
Binary Decision Diagrams
• Goal: derive bound on optimal value from BDD.
– Good bounds are essential to optimization.
• BDD can grow exponentially with problem size.
– So we use a smaller, relaxed BDD that represents superset of
feasible set.
• Andersen, Hadžić, Hooker, Tiedemann 2007.
– Introduced as alternative to domain store in CP.
• For alldiff systems, reduced search tree from
>1 million nodes to 1 node.
• Subsequent papers by…
– Bergman, Ciré, Hadžić, Hoda, van Hoeve, Hooker, Kell, O’Sullivan,
Sabharwal, Samulowitz, Saraswat, Tiedemann, Yunes
![Page 11: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/11.jpg)
Binary Decision Diagrams
• Goal: derive bound on optimal value from BDD.
– Good bounds are essential to optimization.
• BDD can grow exponentially with problem size.
– So we use a smaller, relaxed BDD that represents superset of
feasible set.
• Andersen, Hadžić, Hooker, Tiedemann 2007.
– Introduced as alternative to domain store in CP.
• For alldiff systems, reduced search tree from
>1 million nodes to 1 node.
• Subsequent papers by…
– Bergman, Ciré, Hadžić, Hoda, van Hoeve, Hooker, Kell, O’Sullivan,
Sabharwal, Samulowitz, Saraswat, Tiedemann, Yunes
• We focus on stable set problem on a graph…
![Page 12: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/12.jpg)
1
2 3
4
6 5
Stable Set Problem
Let each vertex have weight wi
Select set of nonadjacent vertices to maximize i i
i
w x
![Page 13: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/13.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Exact BDD for
stable set
problem
“zero-suppressed”
BDD
![Page 14: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/14.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
x1 = 1
x2 = 0
Exact BDD for
stable set
problem
“zero-suppressed”
BDD
![Page 15: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/15.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Paths from top
to bottom
correspond to
the 11 feasible
solutions
![Page 16: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/16.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Paths from top
to bottom
correspond to
the 11 feasible
solutions
![Page 17: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/17.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Paths from top
to bottom
correspond to
the 11 feasible
solutions
![Page 18: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/18.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Paths from top
to bottom
correspond to
the 11 feasible
solutions
![Page 19: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/19.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Paths from top
to bottom
correspond to
the 11 feasible
solutions
…and so
forth
![Page 20: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/20.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
For objective
function,
associate
weights with
arcs
w2
w3
w4 w4
w5
w5
w6
0
0
0
0
0 0
0 0
0
w1
w3
![Page 21: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/21.jpg)
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
For objective
function,
associate
weights with
arcs
w1
w2
w3
w3 w4 w4
w5
w5
w6
0
0
0
0
0 0
0 0
0
Optimal solution
is longest path
![Page 22: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/22.jpg)
{123456}
{23456}
{35} {3456}
{456} {4}
{56} {5}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build BDD,
associate state
with each node
as in dynamic
programming
Control
![Page 23: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/23.jpg)
{123456} x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build BDD,
associate state
with each node
Control
![Page 24: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/24.jpg)
{123456}
{23456}
{35}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build BDD,
associate state
with each node
Control
![Page 25: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/25.jpg)
{123456}
{23456}
{35} {3456}
{4}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build BDD,
associate state
with each node
Control
![Page 26: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/26.jpg)
{123456}
{23456}
{35} {3456}
{456} {4}
{5}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Merge nodes
that correspond
to the same
state
Control
![Page 27: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/27.jpg)
{123456}
{23456}
{35} {3456}
{456} {4}
{56} {5}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Merge nodes
that correspond
to the same
state
Control
![Page 28: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/28.jpg)
{123456}
{23456}
{35} {3456}
{456} {4}
{56} {5}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Merge nodes
that correspond
to the same
state
Control
![Page 29: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/29.jpg)
{123456}
{23456}
{35} {3456}
{456} {4}
{56} {5}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Width = 2
Merge nodes
that correspond
to the same
state
Control
![Page 30: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/30.jpg)
{123456}
{23456}
{35} {3456}
{456} {4}
{56} {5}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Width = 2
BDD after
node merger is
not
necessarily
reduced
Control
![Page 31: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/31.jpg)
Objective Function
• In general, objective function can be any function of the
current state.
– Linear or nonlinear, convex or nonconvex
– There is a unique reduced BDD with canonical edge costs.
![Page 32: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/32.jpg)
{123456} x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 33: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/33.jpg)
{123456}
{23456}
{35}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 34: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/34.jpg)
{123456}
{23456}
{35} {3456}
{4}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 35: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/35.jpg)
{123456}
{23456}
{3456}
{4}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 36: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/36.jpg)
{123456}
{23456}
{3456}
{456} {4}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 37: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/37.jpg)
{123456}
{23456}
{3456}
{456}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 38: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/38.jpg)
{123456}
{23456}
{3456}
{456}
{56}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 39: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/39.jpg)
{123456}
{23456}
{3456}
{456}
{56}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 40: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/40.jpg)
{123456}
{23456}
{3456}
{456}
{56}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Width = 1
To build relaxed
BDD, merge
some additional
nodes as we go
along
![Page 41: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/41.jpg)
{123456}
{23456}
{3456}
{456}
{56}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Represents 18
solutions,
including 11
feasible
solutions
Width = 1
![Page 42: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/42.jpg)
{123456}
{23456}
{3456}
{456}
{56}
{6}
x1
x2
x3
x4
x5
x6
1
2 3
4
6 5
Longest path
gives bound
of 3 on optimal
value of 2
Width = 1
![Page 43: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/43.jpg)
• Variable ordering is important.
– Just as in branching methods.
– Some orderings provide tighter bounds.
– Recursive model can dictate natural variable ordering.
Variable Ordering
![Page 44: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/44.jpg)
• Variable ordering is important.
– Just as in branching methods.
– Some orderings provide tighter bounds.
– Recursive model can dictate natural variable ordering.
• For stable set problem, width of exact BDD is bounded
by Fibonacci numbers.
– For an ordering induced by maximal path decomposition of the
graph.
Variable Ordering
![Page 45: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/45.jpg)
• Variable ordering is important.
– Just as in branching methods.
– Some orderings provide tighter bounds.
– Recursive model can dictate natural variable ordering.
• For stable set problem, width of exact BDD is bounded
by Fibonacci numbers.
– For an ordering induced by maximal path decomposition of the
graph.
• We used a dynamic ordering heuristic.
– Next variable is the one appearing in the smallest number of
states on the current level.
• Better than maximal path ordering.
Variable Ordering
![Page 46: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/46.jpg)
• Which nodes to merge when building relaxation?
• A quality-of-solution heuristic seems by far the best.
– Define a state variable that indicates solution quality.
– Merge nodes associated with poor solution quality
– In this case, solution quality = longest path into node.
– We lose information in areas not likely to be part of the solution.
Merging Heuristic
![Page 47: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/47.jpg)
• Wider BDDs
yield tighter
bounds.
– But take
longer to
build.
Width of BDD
![Page 48: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/48.jpg)
• Random and benchmark instances
• Compare with LP bound at root node in CPLEX
– Use clique cover IP formulation
• Requires precomputing clique cover.
Comparison with LP Bound
max
1, all cliques in clique cover
0,1 , all
k
i i
i
i k
i C
i
w y
y C
y i
![Page 49: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/49.jpg)
• CPLEX settings
– Turn off presolve
• It makes CPLEX bound worse or at most 1 better.
• BDD bounds don’t use presolve.
– Use full cutting plane resources (50 years of development)
– Use interior point (barrier) LP solver
• Faster than simplex on these instances.
Comparison with LP Bound
![Page 50: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/50.jpg)
Random instances
LP + cuts
BDD 1000
BDD 10,000
3.7
9.8
7.8
10.6 13.6 15.0
15.3
9.4
4.6
1.1
0.9 0.8
0.7
0.5
0.2
0.07 0.02 0.01
15.0 13.8
11.8
7.8 3.9
0.5 0.07 0.02 0.01
Bound /
Optim
al valu
e
Density
Avg solution time, sec
![Page 51: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/51.jpg)
Random instances
BDD 10,000
3.7 1.1
0.5 0.07 Density
BD
D 1
000 B
ound /
Optim
al valu
e
LP+cuts Bound / Optimal value
![Page 52: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/52.jpg)
Random instances
BD
D 1
0,0
00 B
ound /
Optim
al valu
e
LP+cuts Bound / Optimal value
![Page 53: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/53.jpg)
DIMACS instances
LP + cuts
BDD 1000
BDD 10,000
6.9
50.2
149
45.1
4.2
6.5
3.5
6.5
2.9
0.01
55.4
34.3
43.6
10.4
0.01
Bound /
Optim
al valu
e
Density
![Page 54: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/54.jpg)
DIMACS instances
BD
D 1
000 B
ound /
Optim
al valu
e
LP+cuts Bound / Optimal value
![Page 55: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/55.jpg)
DIMACS instances
BD
D 1
0,0
00 B
ound /
Optim
al valu
e
LP+cuts Bound / Optimal value
![Page 56: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/56.jpg)
• Fast incremental calculation of bound.
– Modify relaxed BDD after variable is fixed in branching tree
(fast).
– Recompute longest path (fast).
• However, may be better to rebuild relaxation from
scratch.
– Research issue.
Incrementality
![Page 57: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/57.jpg)
• Restricted BDDs as primal heuristic
– All paths are feasible.
– Longest path is a good feasible solution
– Reduce width by intelligently removing nodes.
• Apply to stable set problem
– DIMACS instances
– Compare with lower bound obtained by CPLEX primal heuristics
at rote node.
– Details and updates in Bergman, Ciré, van Hoeve, Yunes,
J. of Heuristics 2014.
Restricted BDDs
![Page 58: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/58.jpg)
1
2
4
3
5 6
1
2
3
4
5
6
Restricted BDD
Width = 3
Longest path is
feasible solution
and gives upper bound
of 2 on optimal value of 3
![Page 59: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/59.jpg)
DIMACS instances
![Page 60: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/60.jpg)
• Assist existing solvers
– MIP solvers.
• Bound at root node.
• Can generate BDD bounds at nodes below root node, due to
speed of BDD processing.
– CP solvers.
• Bound at root node.
How to Use BDD Bounds
![Page 61: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/61.jpg)
• Part of general BDD-based solver
– Branch in the relaxed BDD.
– Combine with BDD-based propagation
• BDD plays role of LP relaxation, cutting planes
– BDD-based primal heuristic
– Dynamic programming style formulation
• Specify state variable for BDDs
• Linearity, convexity irrelevant
– Details in talk by David Bergman tomorrow (ACP Thesis Award).
How to Use BDD Bounds
![Page 62: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/62.jpg)
• Novel branching scheme
– Branch on nodes of relaxed BDD.
– …rather than on variables.
Branching in Relaxed BDD
![Page 63: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/63.jpg)
1
2
4
3
5 6
1
2
3
4
5
6
Relaxed BDD
Exact down
to here
![Page 64: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/64.jpg)
1
2
4
3
5 6
1
2
3
4
5
6
Relaxed BDD
Branch on nodes
in this layer
![Page 65: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/65.jpg)
1
2
4
3
5 6
1
2
3
4
5
6
First branch
Relaxed BDD
![Page 66: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/66.jpg)
1
2
4
3
5 6
1
2
3
4
5
6
Second branch
Relaxed BDD
![Page 67: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/67.jpg)
1
2
4
3
5 6
1
2
3
4
5
6
Third branch
Relaxed BDD
Continue recursively
![Page 68: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/68.jpg)
• BDD-based optimization parallelizes very efficiently
– Near-linear speedups.
– Details in Bergman, Ciré, Sabharwal, Samulowitz, Saraswat,
van Hoeve, CPAIOR 2014.
Parallelization
![Page 69: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/69.jpg)
• Flexibility of dynamic programming
– Without having to search an exponential state space.
• Potential for “big data” optimization
– No need to “load” a huge model, as in integer programming.
– Recursive model is essentially constant size.
Recursive Modeling
![Page 70: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/70.jpg)
• Example: simple one-machine sequencing problem.
– At least 6 integer programming models.
– Very simple DP state
• set of jobs not yet sequenced
• finish time of last job processed.
Recursive Modeling
![Page 71: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/71.jpg)
• Example: simple one-machine sequencing problem.
– At least 6 integer programming models.
– Very simple DP state
• set of jobs not yet sequenced
• finish time of last job processed.
• Formulate constraints that are hard to write in IP.
– For example, job cannot be processed until certain components
are available from earlier jobs.
– Maintenance schedule, conditional time windows, etc.
– Easily written in terms of current state.
Recursive Modeling
![Page 72: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/72.jpg)
• Example: simple one-machine sequencing problem.
– At least 6 integer programming models.
– Very simple DP state
• set of jobs not yet sequenced
• finish time of last job processed.
• Formulate constraints that are hard to write in IP.
– For example, job cannot be processed until certain components
are available from earlier jobs.
– Maintenance schedule, conditional time windows, etc.
– Easily written in terms of current state.
• Nonstandard objective functions.
– Job processing cost depends on which components available
from earlier jobs.
– Can be table lookup function.
Recursive Modeling
![Page 73: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/73.jpg)
• Min bandwidth and linear arrangement problems.
– Sequence electronic components.
– Some pairs are connected by a wire.
– Minimize longest wire, or sum of wire lengths.
Recursive Modeling
1 2
3
4 5
1 2 3 4 5
Bandwidth = 3
sum = 9
![Page 74: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/74.jpg)
• Min bandwidth and linear arrangement problems.
– Sequence electronic components.
– Some pairs are connected by a wire.
– Minimize longest wire, or sum of wire lengths.
• No usable IP model
• Recursive model
– Control = length of wire connecting pair of components
– State = (for each component) remaining positions it can be
assigned
Recursive Modeling
1 2
3
4 5
1 3 4 5 2
Bandwidth = 3
sum = 9
![Page 75: Optimization Bounds from Binary Decision Diagramspublic.tepper.cmu.edu/jnh/BDDboundsCP2014.pdf · Optimization Bounds from Binary Decision Diagrams J. N. Hooker Joint work with David](https://reader035.vdocument.in/reader035/viewer/2022070523/605e7de3cdd7ff3c754bdf66/html5/thumbnails/75.jpg)
• Continuous global optimization.
– Recursive models.
– Massive discretization of continuous variables.
– Node merger heuristics control discretization density.
– No need for convexification and Lipschitz bounds.
– Use “big data” approach of applying simple model to huge
dataset (of discrete values).
Future Work