miping the probabilistic integer programming problem

MIPing the Probabilistic Integer MIPing the Probabilistic Integer Programming ProblemProgramming Problem

Anureet SaxenaACO PhD Student,

Tepper School of Business,

Carnegie Mellon University.

(Joint Work with Vineet Goyal and Miguel Lejuene)

Why Probabilistic Programming?Why Probabilistic Programming?

Set of Customers

Demand Constraints

Capacity Constraints

Transportation Cost

Set of Facilities

Fixed Cost

Why Probabilistic Programming?Why Probabilistic Programming?

Set of CustomersSet of Customers

Demand ConstraintsDemand Constraints

Capacity ConstraintsCapacity Constraints

Transportation CostTransportation Cost

Set of FacilitiesSet of Facilities

Fixed CostFixed Cost

Uncertain Future• Population Shift• Evolution of Market Trends• Ford opens a manufacturing unit• Google closes its R&D center

Why Probabilistic Programming?Why Probabilistic Programming?

A random 0/1 vector which incorporates the uncertain future into the optimization model

Why Probabilistic Programming?Why Probabilistic Programming?

Probabilistic Constraint

Reliability Level

Probabilistic MIP ModelProbabilistic MIP Model

Deterministic Probabilistic

Random 0/1 Vector(Joint Distribution)

Reliability Level

Why Probabilistic Programming?Why Probabilistic Programming?

• Facility Location– Strategic Planning– Population shift– Evolution of market trends– Demographic Changes

• Contingency Service– Minimum Reliability Principle

• Production Design and Manufacturing– Uncertain Demand– Lot Sizing and Inventory Problems

Must Read!

Strategic facility locationby Owen and Daskin

A Simple AlgorithmA Simple AlgorithmRandom 0/1 Vector(Joint Distribution)

Reliability Level

1. Enumerate all possible 0/1 realizations of .

2. For each 0/1 realization whose cdf is greater than or equal to p, solve the deterministic problem

Prekopa, Beraldi, Ruszczynski ApproachPrekopa, Beraldi, Ruszczynski Approach

Prekopa, Beraldi, Ruszczynski ApproachPrekopa, Beraldi, Ruszczynski Approach

000

100 010 001

011101110

111

Prekopa, Beraldi, Ruszczynski ApproachPrekopa, Beraldi, Ruszczynski Approach

p-efficient frontier

2-Phase Algorithm2-Phase Algorithm

Enumeration of p-efficient points

Solving a Deterministic Problem for each p-efficient point

2-Phase Algorithm2-Phase Algorithm

Enumeration of p-efficient points

Solving a Deterministic Problem for each p-efficient point

Independent

Beraldi & Ruszczynski ApproachBeraldi & Ruszczynski Approach

scp41

scp42

Explosive GrowthIn computation

time

2-Phase Algorithm2-Phase Algorithm

Enumeration of p-efficient points

Solving a Deterministic Problem for each p-efficient point

Pitfall

Our ApproachOur Approach

Enumeration of p-efficient points

Solving a Deterministic Problem for each p-efficient point

Integrate the 2-phases

Our ApproachOur Approach

Enumeration of p-efficient points

Solving a Deterministic Problem for each p-efficient point

Integrate the 2-phases

Independent

Our ModelOur Model

Log of cumulative probability of block t

Non-Linear MIPing

Our ModelOur Model

Log of cumulative probability of block t

Our ModelOur Model

Log of cumulative probability of block t

Beraldi & Ruszczynski Approach: Beraldi & Ruszczynski Approach: ComparisonComparison

scp41

scp42

All instances solved in less than 1sec by

CPLEX 9.0. CPLEX enumerated less than

50 nodes solving most instances at the root

node

Key ObservationsKey Observations

• Models any arbitrary distribution• Exponential number of constraints for each block • Linear in the input size for generic distribution• Encodes the enumeration phase as a Mixed Integer

Program• Allows us to exploit state-of-art MIP solvers to perform

intelligent enumeration.

Key ObservationsKey Observations

• Models any Models any arbitraryarbitrary distribution distribution• Exponential number of constraints for each block Exponential number of constraints for each block • Linear in the input size for generic distributionLinear in the input size for generic distribution• Encodes the enumeration phase as a Mixed Integer Encodes the enumeration phase as a Mixed Integer

ProgramProgram• Allows us to exploit state-of-art MIP solvers to perform Allows us to exploit state-of-art MIP solvers to perform

intelligent enumeration.intelligent enumeration.

Research Question

The model has an exponential number of constraints for each block. Is there a way to reduce the number of constraints?

The Answer is YesThe Answer is Yes

p-Inefficient Frontierp-Inefficient Frontier

Refined FormulationRefined Formulation

Add t constraints only for lattice

points above the frontier

Set-Covering Constraint for maximally p-

inefficient points

Refined FormulationRefined FormulationBlock Size10

A Tough Instance - A Tough Instance - p31p31

• SSCFLP instance from the Holmberg test-bed• 30 facilities and 150 customers• Deterministic instance can be solved in 80 sec.• Probabilistic instance has 15 blocks of size 10 each• CPLEX was unable to solve the probabilistic instance

within 2 hours!!

A Tough Instance - A Tough Instance - p31p31

A Tough Instance - A Tough Instance - p31p31

Research Question

Why is this instance so difficult to solve?

AnswerAnswer

Big-M Constraints

Polarity CutsPolarity Cuts

Big-M Constraints model P

Facets of P can strengthen the model

Polarity CutsPolarity Cuts

• We know all the extreme points and extreme rays of P

• Compact description of polar

• Facets of P can be found by solving the linear program derived from the polar

• The linear program has lot more rows than columns – dual simplex algorithm.

A Tough Instance - A Tough Instance - p31p31

Tough Instance Solved

• % Gap closed at Root Node 67.84%• Time Spent in Strengthening 0.83 sec• Time Spent in Solving Separation LP 0.30 sec• Time Taken by CPLEX 9.0 after Strengthening 51.65 sec • No. of Branch-and-Bound enumerated by CPLEX 9.0 2300• Total time taken to solve the instance to optimality 53.04 sec

Computational ResultsComputational Results

• Implementation– COIN-OR Modules– CPLEX 9.0

• Selection Criterion– ORLIB & Holmberg Instances– Instances which can be solved in 1hr

• Computational Power– P4 Processor– 2GB RAM

• Library of Instances – PCPLIB

Test BedTest BedProblem Set Number of Instances # Rows # Columns

OrLib Set Covering 60 50-500 500-5000OrLib Warehouse Location (Cap) 37 66-100 816-2550OrLib p-Median (Cap) 20 101-201 2550-10100Holmberg Facility Location (Cap) 70 60-230 510-6030

• 2 Distributions – as in BR [2002]• 4 Reliability levels – 0.80, 0.85, 0.90, 0.95• 2 Block Sizes – 5, 10 • Total Number of Instances per Deterministic Instance = 16

Computational ResultsComputational Results

Deterministic Problem

Number of Probabilistic Instances

Number of Unsolved Instances

% Relative Gap (Unsolved Instances)

Set Covering 1440 37 11.69CWLP 888 0 -Cap k-Median 480 0 -SSCFLP 1680 22 0.45

Computational ResultsComputational Results

Deterministic Problem

Solution Time (sec)

Number of Branch-and-Bound Nodes

Set Covering 160.81 7440CWLP 0.31 30Cap k-Median 43.79 1464SSCFLP 31.27 2248

Impact of Polarity CutsImpact of Polarity Cuts

Deterministic Problem

% Duality Gap Closed

% Time Spent

Set Covering 23.74 0.22

CWLP 11.44 9.43

Cap k-Median 0.00 0.21

SSCFLP 18.45 0.29

Polarity Cuts' Strengthening

Value of InformationValue of Information

Deterministic Problem

Value of Information (%)

Set Covering 5.75CWLP 15.05Cap k-Median 9.54SSCFLP 4.60

Value of InformationValue of Information

Deterministic Problem Value of Information

(%)Set Covering 5.75CWLP 15.05Cap k-Median 9.54SSCFLP 4.60

Empirical Observation

Probabilistic versions of simple and moderately difficult mixed integer programs can themselves be formulated as MIPs which can be solved in reasonable amount of time.

Structured DistributionsStructured Distributions

Research Question

Is it possible to exploit structure of distributions to design models which are polynomial in the input size?

Stationary DistributionsStationary Distributions

Definition

A distribution function F is said to be stationary if F(z) depends only on the number of ones in z.

Principle of Indistinguishability.

Stationary DistributionsStationary Distributions

000

100 010 001

011101110

111

Stationary DistributionsStationary Distributions

Can be converted to a MIP with linear number of additional variables and constraints!!

Stationary DistributionsStationary Distributions

A model with linear number of variables and constraints!!

Stationary DistributionsStationary Distributions

Deterministic Problem

Number of Probabilistic

Instances

Number of Unsolved Instances

% Relative Gap (Unsolved

Instances)

Solution Time (sec)

Value of Information

Set Covering 1920 127 21.34 112.98 10.42

CWLP 1184 0 - 0.09 25.15

Cap k-Median 640 0 - 2.90 15.25

SSCFLP 2240 17 0.45 9.36 8.51

• 8 Block Sizes: 5, 10, 20, 50, m/4, m/3, m/2, m• 4 Threshold Probabilities: 0.80, 0.85, 0.90, 0.95

Number of Instances per deterministic instance= 32

Stationary DistributionsStationary Distributions

Research Question

What is that unique property of stationary distributions which allowed us to design a linear sized model?

Disjunctive Shattering Property

The lattice of a stationary distribution can be partitioned into polynomial number of pieces each of which has a polynomial sized description.

Stationary DistributionsStationary Distributions

000

100 010 001

011101110

111

Summary

BR Algorithm MIP Model

p-Inefficiency

Polarity Cuts

ComputationalResults

Stationary Distributions

Super LinearSpeedup

Refinement

Strengthening

Our ContributionOur Contribution

Thank you for your attention