Large neighbourhood Benders' search

Stephen J. Maher

Lancaster University Management School,s.maher3@lancaster.ac.uk

8th March 2018

1 / 23

Structured mixed integer programmingBasic idea: Minimise a linear objective function over a set of solutionssatisfying a structured set of linear constraints.

min c>x + d>y ,

subject to Ax ≥ b,

Bx + Dy ≥ g ,

x ∈ Zp1+ × Rn1−p1

+ ,

y ∈ Zp2+ × Rn2−p2

+ .

2 / 23

Solving structured mixed integer programs

I State-of-the-art general purpose solversI CPLEX, Gurobi, Xpress, SCIP, . . .

SCIP PrimalHeuristic

actconsdiving

alns

bound

cliquecoefdiving

completesol

crossover

dins

distributiondiving

dualval

feaspump

fixandinfer

fracdiving

gins

guideddiving

indicator

intdiving

intshifting

linesearchdiving

localbranching

locks

lpface

mpecmultistart

mutation

nlpdiving

objpscostdiving

octane

ofins

oneopt

proximity

pscostdiving

randomrounding

rens

reoptsols

repair

rins

rootsoldiving

rounding

shift&prop

shifting

simplerounding

subnlp

sync

trivial

trivialnegation

trysol

twoopt

undercover

vbounds

veclendiving zero

objective

zirounding

Event

Expr.Interpr.

CppAD

Propagator

dualfix

genvbounds

nlobbt

obbt

orbitalfixing

probing

pseudoobj

redcostroot

redcost

sync

vbounds

· · ·

Reader

bnd

ccg cip

cnf

diff

fix

fzngms

lp

mps

mst

opbosil

pbm

pip

ppm

rlp sol

wbo

zpl

Pricer

NLP

filtersqp

ipopt

worhp

LP

cpx

grb

msk

none

qsoxprs

clp

spx1

spx2

Relax

ConstraintHandler

abspower

and

bivariate

bounddisjunction

cardinality

components

conjunc-tion

countsols

cumulativedisjunc-

tion

indicator

integral

knapsack

linear

linking

logicor nonlinear

orbisack

orbitope

or

pseudoboolean

quadratic

setppc

soc

sos1 sos2

superindicator

symresack

varbound

xor

Conflict

Branch

allfullstrong

cloud

distribution

fullstrong

inference

leastinf

mostinf

multiaggr

nodereopt

pscost

randomrelpscost Node

selector

bfsbreadthfirst

dfs

estimate

hybridestim

restartdfs

uct

Tree

Presolver

boundshift

convertint tobin

domcol

dualagg

dualcomp

dualinfer

gateextract

implfree

implics

inttobinary

qpkktref

redvubsparsify

stuffing

symbreak

symmetry

trivialtworowbnd

Implications

Separator

aggregation

cgmip

clique

closecuts

convexproj

disjunctive

eccuts

gauge

gomoryimpliedbounds

intobj

mcfoddcycle

rapidlearning

strongcg

zerohalf

Cutpool

Dialog

default

3 / 23

Solving structured mixed integer programs

I State-of-the-art general purpose solversI CPLEX, Gurobi, Xpress, SCIP, . . .

I Decomposition techniquesI Column generation, Lagrangian relaxation/decomposition, Benders'

decomposition, . . .

SCIP PrimalHeuristic

actconsdiving

alns

bound

cliquecoefdiving

completesol

crossover

dins

distributiondiving

dualval

feaspump

fixandinfer

fracdiving

gins

guideddiving

indicator

intdiving

intshifting

linesearchdiving

localbranching

locks

lpface

mpecmultistart

mutation

nlpdiving

objpscostdiving

octane

ofins

oneopt

proximity

pscostdiving

randomrounding

rens

reoptsols

repair

rins

rootsoldiving

rounding

shift&prop

shifting

simplerounding

subnlp

sync

trivial

trivialnegation

trysol

twoopt

undercover

vbounds

veclendiving zero

objective

zirounding

Event

Expr.Interpr.

CppAD

Propagator

dualfix

genvbounds

nlobbt

obbt

orbitalfixing

probing

pseudoobj

redcostroot

redcost

sync

vbounds

· · ·

Reader

bnd

ccg cip

cnf

diff

fix

fzngms

lp

mps

mst

opbosil

pbm

pip

ppm

rlp sol

wbo

zpl

Pricer

NLP

filtersqp

ipopt

worhp

LP

cpx

grb

msk

none

qsoxprs

clp

spx1

spx2

Relax

ConstraintHandler

abspower

and

bivariate

bounddisjunction

cardinality

components

conjunc-tion

countsols

cumulativedisjunc-

tion

indicator

integral

knapsack

linear

linking

logicor nonlinear

orbisack

orbitope

or

pseudoboolean

quadratic

setppc

soc

sos1 sos2

superindicator

symresack

varbound

xor

Conflict

Branch

allfullstrong

cloud

distribution

fullstrong

inference

leastinf

mostinf

multiaggr

nodereopt

pscost

randomrelpscost Node

selector

bfsbreadthfirst

dfs

estimate

hybridestim

restartdfs

uct

Tree

Presolver

boundshift

convertint tobin

domcol

dualagg

dualcomp

dualinfer

gateextract

implfree

implics

inttobinary

qpkktref

redvubsparsify

stuffing

symbreak

symmetry

trivialtworowbnd

Implications

Separator

aggregation

cgmip

clique

closecuts

convexproj

disjunctive

eccuts

gauge

gomoryimpliedbounds

intobj

mcfoddcycle

rapidlearning

strongcg

zerohalf

Cutpool

Dialog

default

3 / 23

Solving large scale optimisation problemsAim: Embed Benders' decomposition within a state-of-the-art solver toprovide e�ective tools to employ decomposition to solve large-scaleoptimisation problems.

4 / 23

Solving large scale optimisation problemsAim: Embed Benders' decomposition within a state-of-the-art solver toprovide e�ective tools to employ decomposition to solve large-scaleoptimisation problems.

I Develop a general Benders' decomposition framework

I Harness the capabilities of state-of-the-art MIP solvers when usingdecomposition techniques

I Integration of Benders' decomposition and large neighbourhoodsearch heuristics.

4 / 23

Benders' decompositionOriginal problem

min c>x + d>y ,

subject to Ax ≥ b,

Bx + Dy ≥ g ,

x ∈ Zp1+ × Rn1−p1

+ ,

y ∈ Rn2+ .

5 / 23

Benders' decomposition

min c>x + f (x),

subject to Ax ≥ b,

x ∈ Zp1+ × Rn1−p1

+ .

where

f (x) = miny∈Rn2

+

{d>y |Bx + Dy ≥ g}

5 / 23

Benders' decompositionMaster problem

min c>x + ϕ,

subject to Ax ≥ b,

ϕ ≥ u>ω (g − Bx) ∀ω ∈ O,0 ≥ u>ω (g − Bx) ∀ω ∈ F ,ϕ ∈ R+,

x ∈ Zp1+ × Rn1−p1

+ .

Subproblem

z(x) = min d>y ,

subject to Dy ≥ g − Bx ,

y ∈ Rn2−p2+ .

6 / 23

Benders' decompositionMaster problem

min c>x + ϕ,

subject to Ax ≥ b,

ϕ ≥ u>ω (g − Bx) ∀ω ∈ O,0 ≥ u>ω (g − Bx) ∀ω ∈ F ,{no-good/integer cuts},

ϕ ∈ R+,

x ∈ Zp1+ × Rn1−p1

+ .

Subproblem

z(x) = min d>y ,

subject to Dy ≥ g − Bx ,

y ∈ Zp2+ × Rn2−p2

+ .

6 / 23

Standard Benders' implementation

Start

Solve master problem

Solve subproblems

z(x) > ϕ

Stop

No

Yes - add cut

I Easy to understand and simple to implement.

I Not always e�ective, large overhead in repeatedly solving masterproblem.

7 / 23

Branch-and-check

I Modern solvers pass through a number of di�erent stages duringnode processing.

I Some of these stages can be used to generate Benders' cuts.

I By interrupting node processing, Benders' cuts are generated duringthe tree search.

Solving process

Start Init Presolving

Stop

Node selection

Processing

Branching

Conflict analysis

Primal heuristics

LP inf.

LP feas.IP inf.

IP feas.

Domain propagation

Solve LP

Pricing

Cuts

Enforce constraints

8 / 23

Branch-and-check

I Modern solvers pass through a number of di�erent stages duringnode processing.

I Some of these stages can be used to generate Benders' cuts.

I By interrupting node processing, Benders' cuts are generated duringthe tree search.

Cut generation - Standard Benders'

Start Init Presolving

Stop

Node selection

Processing

Branching

Conflict analysis

Primal heuristics

LP inf.

LP feas.IP inf.

IP feas.

Domain propagation

Solve LP

Pricing

Cuts

Enforce constraints

8 / 23

Branch-and-check

I Modern solvers pass through a number of di�erent stages duringnode processing.

I Some of these stages can be used to generate Benders' cuts.

I By interrupting node processing, Benders' cuts are generated duringthe tree search.

Cut generation - Branch-and-check

Start Init Presolving

Stop

Node selection

Processing

Branching

Conflict analysis

Primal heuristics

LP inf.

LP feas.IP inf.

IP feas.

Domain propagation

Solve LP

Pricing

Cuts

Enforce constraints

8 / 23

Large neighbourhood search heuristics

I Concept: Identify improved primal solutions by solving an auxiliaryproblem that is a restriction of the original problem.

I The auxiliary problem is typically formed by �xing variables or theaddition of constraints.

I The restricted auxiliary problem is expected to be easier to solvethan the original problem.

I State-of-the-art solvers employ many variants of largeneighbourhood search heuristics

I Crossover, DINS, Local branching, proximity search, RENS, . . .

I Very e�ective in �nding solutions to di�cult optimisation problems.

9 / 23

Large Neighbourhood Benders' Search

I Concept: Using large neighbourhood search to improve theconvergence of the Benders' decomposition algorithm.

Large neighbourhoodsearch

Restriction oforiginal problem

Quickly generateprimal solutions

- Restriction of master problem- Same subproblems- All generated cuts valid for original

- Primal solution satisfy subproblem constraints- Higher quality solutions

With Benders' Decomposition

I Builds upon successful trust region approaches.

10 / 23

Benders' decomposition everywhere

I Local branching (Rei et al. (2009)) and proximity search (Boland etal. (2015)) have demonstrated potential to enhance BD using LNSheuristics.

I Modern MIP solvers are endowed with vast array of LNS heuristics.

I LNS heuristics solve sub-MIP instances�can be solved by Benders'decomposition.

Aim: Employ Benders' decomposition in all available LNSheuristics.

11 / 23

Benders' decomposition everywhere

I Local branching (Rei et al. (2009)) and proximity search (Boland etal. (2015)) have demonstrated potential to enhance BD using LNSheuristics.

I Modern MIP solvers are endowed with vast array of LNS heuristics.

I LNS heuristics solve sub-MIP instances�can be solved by Benders'decomposition.

Aim: Employ Benders' decomposition in all available LNSheuristics.

11 / 23

Generic Benders' Decomposition

SCIP

Benders' decompositionconstraint handler

Benders' decompositionCore

Optimality cuts

Feasibility cuts

Integer cuts

Default Benders'decomposition plugin

Flexibilty for customBD implementation

SMPS file reader

12 / 23

Using the Benders' decomposition framework

I First option:I Provide an instance in SMPS (Stochastic MPS) format to SCIP.I The SMPS format consists of three components: core model, time

�le for stages and stochastic information.

I To use the Benders' decomposition algorithm, the following settingmust be used

reading/sto/usebenders = TRUE.

13 / 23

Using the Benders' decomposition framework

I Second option:I SCIPcreateBendersDefault(master SCIP, array of

subproblem SCIPs, number of subproblems)

I Master SCIP and Subproblem SCIP instances must be created bythe user.

I Variables common between the master problem and subproblemsmust have the same name.

I All variable mappings between master and subproblems aregenerated automatically.

14 / 23

Using the Benders' decomposition framework

I Third option:I Implement a custom Benders' decomposition plugin: benders_xyz.c

and benders_xyz.h

I Required callbacks:I Mapping between the master and subproblem variables,I Method to create each subproblem.

I Various optional callbacks:I Pre subproblem solving, a solving method for the subproblem, post

solving, freeing subproblem, and usual SCIP callbacks (init, initsol,exit, exitsol, copy, ...)

15 / 23

Using Benders' decomposition in LNS

Advantages:

I Solutions are guaranteed to be optimal w.r.t the subproblems.

I Generated cuts can be transferred to the original problem.

I Flexible w.r.t SCIP development.

I Simple changes of parameters can provide aggressive use of LNS forBenders' decomposition.

16 / 23

Test problems

Stochastic Capacitated Facility Location Problem

I CAP instances from OR-Library with 25 or 50 facilities.

I Instances with 250 and 500 scenarios.

I 48 instances.

Stochastic Network Interdiction Problem

I The same set of instances as used by Bodour et al. (2017).

I Graph with 738 nodes and 2586 arcs, 320 possible sensor locations.

I 456 scenarios.

Stochastic Multiple Knapsack Problem

I Instances collected from SIPLIB.

I First stage: 240 binary variables, 50 knapsack constraints.

I Second stage: 120 binary variables, 5 knapsack constraints.

I 30 instances, each with 20 scenarios.

17 / 23

Experiments

Benders - Standard implementation of branch-and-check.

I Benders' cuts are generated from the LP, Relaxation, Pseudo andfeasible solutions

LNS Check - BD with large neighbourhood Benders' search.

I Employing Benders' decomposition within each LNS heuristic

Transfer cuts - Extension of large neighbourhood Benders' search.

I All cuts generated during the large neighbourhood Benders' searchare transferred to the original problem

18 / 23

Primal Integral (Berthold (2013))

Primal Gap

γ(Z p, Z p) =

0, if |Z p(t)| = |Z p| = 0,

1, if Z p(t)× Z p < 0,

|Z p(t)− Z p|max{|Z p(t)|, |Z p|}

, otherwise.

Integral

P(T ) =

∫ T

t=0

γ(Z p, Z p)dt

19 / 23

SCFLP

Performance pro�le of the primal integral

250 Scenarios

1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5Ratio to best setting

0.0

0.2

0.4

0.6

0.8

1.0

Fract

ion o

f in

stance

s

Benders

LNS check

Transfer cuts

500 Scenarios

1.0 1.5 2.0 2.5 3.0 3.5Ratio to best setting

0.0

0.2

0.4

0.6

0.8

1.0

Fract

ion o

f in

stance

sBenders

LNS check

Transfer cuts

20 / 23

SNIP

Performance pro�le of the primal integral

Class 3

1 2 3 4 5 6 7Ratio to best setting

0.0

0.2

0.4

0.6

0.8

1.0

Fract

ion o

f in

stance

s

Benders

LNS check

Transfer cuts

Class 4

1.0 1.5 2.0 2.5Ratio to best setting

0.0

0.2

0.4

0.6

0.8

1.0

Fract

ion o

f in

stance

sBenders

LNS check

Transfer cuts

21 / 23

SMKP

Performance pro�le of the primal integral

1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7Ratio to best setting

0.0

0.2

0.4

0.6

0.8

1.0

Fract

ion o

f in

stance

s

Benders

LNS check

Transfer cuts

22 / 23

Key points

I SCIP has been extended with a generic implementation of Benders'decomposition.

I Benders' decomposition has been implemented as abranch-and-check algorithm.

I Functionality is available to use Benders' decomposition within LNSheuristics.

I Integrating Benders' decomposition and LNS heuristics cansigni�cantly enhance the primal bound improvement.

23 / 23