chapter 3: discrete optimization integer...

Chapter 3: Discrete Optimization – IntegerProgramming

Edoardo Amaldi

DEIB – Politecnico di [email protected]

Website: http://home.deib.polimi.it/amaldi/OPT-16-17.shtml

Academic year 2016-17

Edoardo Amaldi (PoliMI) Optimization Academic year 2016-17 1 / 26

http://home.deib.polimi.it/amaldi/OPT-16-17.shtml

3.1 Integer Programming models

A huge variety of practical decision-making problems arising in science, engineering andmanagement can be formulated/approximated as linear optimization problems where(some of) the variables must take integer/discrete values.

Generic discrete optimization problem:

minx∈X

c(x)

where X is a discrete set and c : X → R is the objective function.

Example: X ⊆ { all subsets of a given finite set }.

A natural and systematic way to study/investigate such problems is to express them asinteger (programming) optimization problems.


Definitions:

A Mixed Integer Linear Programming (MILP) problem is an optimization problem like

min c t1x + c t2y

s.t. A1x + A2y ≥ b

x ≥ 0 integer, y ≥ 0

with matrices A1 ∈ Zm×n1 and A2 ∈ Zm×n2 , and vectors c1 ∈ Zn1 , c2 ∈ Zn2 and b ∈ Zm.

If all the variables are restricted to be integer, we have an Integer Linear Programming(ILP) problem:

min c tx

s.t. Ax ≥ b (1)

x ≥ 0 integer.

If in (1) all the variables xi ∈ {0, 1}, we have a Binary Linear Programming (0-1-ILP)problem.

Without loss of generality: only inequalities and all coefficients are integer.


Observation: The integrality restriction on xi is a nonlinear constraint since it can beformulated as

sin(πxi ) = 0.

Proposition: 0-1-ILP is NP-hard, and ILP/MILP are at least as difficult.

Theory: No algorithm can find, for every instance of 0-1-ILP (ILP/MILP), an optimalsolution in polynomial time in the instance size, unless P=NP.

Practice: Many medium-size (M)ILPs are extremely challenging!

Examples of feasible regions of an ILP and a MILP:

(Mixed) Integer Linear Programming is a powerful and versatile modeling framework forexpressing and tackling discrete optimization problems.


Modeling techniques and examples

Binary variables allow to model a choice between two (several) alternatives or theassociation between two (several) entities.

Example 1: Binary Knapsack problem

Given

n objects

profit pi and weight ai for each object i , with 1 ≤ i ≤ n

knapsack maximum total weight b (capacity)

decide which objects to select so as to maximize total profit while respecting thecapacity constraint.

ILP formulation

Variables: xi = 1 if the i-th object is selected and xi = 0 otherwise, 1 ≤ i ≤ n

max∑n

i=1 pixi∑ni=1 aixi ≤ b

xi ∈ {0, 1} ∀i


A number of direct applications (projects, investments,...) or indirect applications (assubproblem).

Proposition: Binary Knapsack is NP-hard.

Variants with additional constraints

- at most one object among a given subset of objects

- if i-th object selected then also j-th one

- multiple resource constraints (e.g., on volume, cost,...)

- ...


Example 2: Assignment problem

Given

n projects (jobs) and n persons (machines)

cost cij for assigning project i to person j , ∀ i , j ∈ {1, . . . , n}decide which project to assign to each person so as to minimize the total cost whilecompleting all the projects.

Assumption: every person can perform any project, and each person (project) must beassigned to a single project (person).

Number of feasible solutions: n!

ILP formulation

Variables: xij = 1 if i-th project is assigned to j-th person and xij = 0 otherwise,

with 1 ≤ i , j ≤ nmin

∑ni=1

∑nj=1 cijxij

s.t.∑n

i=1 xij = 1 ∀j∑nj=1 xij = 1 ∀i

xij ∈ {0, 1} ∀i , ∀j

Variants with: bipartite graph representing competences, different number of projectsand persons, resource constraints,...


Example 3: Set Covering/Packing/Partitioning problems

Given

finite groundset M = {1, 2, . . . ,m} with 1 ≤ i ≤ m,

collection {M1, . . . ,Mn} of n subsets of M indexed by N = {1, . . . , n} (namelyMj ⊆ M for j ∈ N),

a cost cj for each subset Mj with j ∈ N,

we say that a subset of indices F ⊆ N defines

a cover of M if ∪j∈FMj = M (i.e., each groundset element i is covered at leastonce),

a packing of M if Mj1 ∩Mj2 = ∅ for all j1, j2 ∈ F with j1 6= j2 (i.e., each groundsetelement i is covered at most once),

a partition of M if it is both a cover and a packing of M (i.e., each groundsetelement i is covered exactly once).

The total cost/weight of a subset indexed by F ⊆ N is defined as∑

j∈F cj .

Examples:


Set Covering problem:

Given a finite groundset M = {1, 2, . . . ,m}, a collection {M1, . . . ,Mn} of n subsets of Mindexed by N = {1, . . . , n}, and a cost cj of Mj for each j ∈ N, find a cover of M ofminimum total cost.

ILP formulation

Variables: xj = 1 if subset Mj is selected and xj = 0 otherwise, with j ∈ N

min∑n

j=1 cjxj

s.t.∑

j :i∈Mjxj ≥ 1 ∀i (2)

xj ∈ {0, 1} ∀j

where the inequalities (2) are the so-called covering constraints.

Proposition: Set Covering is NP-hard.


Matrix notation:

min

{n∑

j=1

cjxj : Ax ≥ 1, x ∈ {0, 1}n}

where A = [aij ] with aij = 1 if i ∈ Mj and aij = 0 otherwise, and 1 = (1, 1, . . . , 1)t

Example: Emergency service location (ambulances or fire stations)

M = { areas to be covered }N = { candidate sites }Mj = { areas reachable in at most τ = 10 minutes from candidate site j }

Decide where to locate ambulances so as to minimize the total cost, while guaranteeingthat the next call is served in at most τ minutes.


Set Packing problem:

max

{n∑

j=1

cjxj : Ax ≤ 1, x ∈ {0, 1}n}

where the parameters cj represent ”profits”

Example: Combinatorial auctions

Determine the winner of each item so as to maximize the total revenue (seeintroduction):

max∑

S⊆M b(S)xS

s.t.∑

S⊆M : i∈S xS ≤ 1 ∀i ∈ M

xS ∈ {0, 1} ∀S ⊂ M,

Proposition: Set Packing is NP-hard.


Set Partitioning problem:

min or max

{n∑

j=1

cjxj : Ax = 1, x ∈ {0, 1}n}

where the weights cj may represent ”costs” or ”profits”

Example: Airline crew scheduling

Consider a predefined planning horizon.

M = { flight legs } where a flight leg consists of a single takeoff-landing phase to becarried out within a predefined time window.

Mj = { feasible subsets of flight legs } where a subset of flight legs is feasible if it can becarried out by a same crew while respecting different constraints (e.g., compatible flights,rest periods, total flight time,...).

Assign the crews to the flight legs so as to minimize total cost.

Other example: distribution planning (assign customers to routes)

Proposition: Set Partitioning is NP-hard.


Example 4: Asymmetric Traveling Salesman Problem (ATSP)

Given

a complete directed graph G = (V ,A) with n = |V | nodes

a cost cij ∈ R for each arc (i , j) ∈ A (in case cij =∞)

determine a Hamiltonian circuit (tour), i.e., a circuit that visits exactly once each nodeand comes back to the starting node, of minimum total cost.

Example:

Since G is complete, number of Hamiltonian circuits : (n − 1)!

Proposition: ATSP is NP-hard.

Variety of applications: logistics, microchip manufacturing, scheduling, (DNA)sequencing,...


Also symmetric TSP version with undirected graph G .

Website devoted to TSP: http://www.math.uwaterloo.ca/tsp/

Many variants with

- time windows (earliest and latest arrival time)

- precedence constraints

- capacity constraint

- several vehicles (”Vehicle Routing Problem” – VRP)

- ...


An ILP formulation

Variables: xij = 1 if arc (i , j) is included in the Hamiltonian circuit and xij = 0 otherwise,with (i , j) ∈ A

min∑

(i,j)∈A cijxij

s.t.∑

j∈V :j 6=i xij = 1 ∀i (3)∑i∈V :i 6=j xij = 1 ∀j (4)∑

(i,j)∈δ+(S) xij ≥ 1 ∀ S ⊂ V ,S 6= ∅ (5)

xij ∈ {0, 1} ∀(i , j) ∈ A

where

equations (3) and (4) are the assignment constraints,

δ+(S) = {(i , j) ∈ A : i ∈ S , j ∈ V \ S},

and constraints (5) are the so-called cut-set inequalities.

Observation: The number of constraints (5) grows exponentially with the number ofnodes n.


Alternative ILP formulation

Substitute the cut-set inequalities∑(i,j)∈δ+(S)

xij ≥ 1 ∀ S ⊂ V ,S 6= ∅

with the so-called subtour elimination inequalities:∑(i,j)∈E(S)

xij ≤ |S | − 1 ∀ S ⊆ V , 2 ≤ |S | ≤ n − 1 (6)

where E(S) = {(i , j) ∈ A : i ∈ S , j ∈ S} for S ⊆ V .

Example:

The number of constraints (6) is still exponential w.r.t. the number of nodes. n.


MILP models

1) Pairs of continuous and binary variables allow to model objective functions with fixedcosts.

Example 5: Uncapacitated Facility Location (UFL)

Given

M = {1, 2, . . . ,m} set of clients

N = {1, 2, . . . , n} set of candidate sites where a depot can be located

fixed cost fj for opening a depot in candidate site j , ∀j ∈ N

cij transportation cost if the whole demand of client i is served from depot j ,∀i ∈ M and ∀j ∈ N

decide where to locate the depots and how to serve the clients so as to minimize thetotal (transportation and fixed) costs while satisfying all demands.

Example:

Proposition: UFL is NP-hard.


MILP formulation

Variables:

xij = fraction of demand of client i served by depot j , with 1 ≤ i ≤ m and 1 ≤ j ≤ n

yj = 1 if depot j is opened and yj = 0 otherwise, with 1 ≤ j ≤ n

min∑

i∈M∑

j∈N cijxij +∑

j∈N fjyj

s.t.∑

j∈N xij = 1 ∀i ∈ M∑i∈M xij ≤ myj ∀j ∈ N (7)

yj ∈ {0, 1} ∀j ∈ N

0 ≤ xij ≤ 1 ∀i ∈ M, j ∈ N

where the n constraints (7) link the corresponding variables xij and yj .

Capacitated FL variant: If di is the demand of client i and kj the capacity of depot j ,capacity constraints: ∑

i∈M

dixij ≤ kjyj ∀j ∈ N

N.B.: The solution with, for all i and a given j , xij = 0 and yj = 1 is feasible for MILP,but it cannot be optimal (minimization with fj > 0).


Example 6: Uncapacitated Lot-Sizing (ULS)

Plan the production of a single type of product for the next n periods.

Assumption: the stock is empty at the beginning and it must be empty at the end.

Given

ft fixed cost for producing during period t

pt unit production cost in period t

ht unit storage cost in period t

dt demand in period t

determine a production plan for the next n periods that minimizes the total (productionand storage) costs, while satisfying the demand in each period.

MILP formulation

Variables:

xt = amount produced in period t, with 1 ≤ t ≤ n

st = amount in stock at the end of period t, with 0 ≤ t ≤ n

yt = 1 if production occurs in period t and yt = 0 otherwise, with 1 ≤ t ≤ n


MILP formulation

Variables:

xt = amount produced in period t, with 1 ≤ t ≤ n

st = amount in stock at the end of period t, with 0 ≤ t ≤ n

yt = 1 if production occurs in period t and yt = 0 otherwise, with 1 ≤ t ≤ n

min∑n

t=1 ptxt +∑n

t=1 htst +∑n

t=1 ftyt

s.t. st = st−1 + xt − dt ∀txt ≤ Myt ∀t

s0 = 0, sn = 0 ∀tst , xt ≥ 0 ∀tyt ∈ {0, 1} ∀t

where M > 0 is large enough (upper bound on the maximum amount produced duringany period). For instance: xt ≤ (

∑nt=1 dt + sn − s0)yt

N.B.: Since st =∑t

i=1 xi + s0 −∑t

i=1 di , it is possible to delete the storage variables st

How can we account for a minimum lot sizes?


2) Binary variables also allow to impose disjunctive constraints such as:

either a1x ≤ b1 or a2x ≤ b2

with x ∈ R and 0 ≤ x ≤ u, where u is the upper bound vector.

Illustration of feasible region:

Introduce a binary variable yi for each original constraint aix ≤ bi , with 1 ≤ i ≤ 2, andconsider the following constraints:

aix − bi ≤ M(1− yi ) for i = 1, 2

y1 + y2 = 1

yi ∈ {0, 1} for i = 1, 2

0 ≤ x ≤ u,

where M ≥ max1≤i≤2{aix − bi : 0 ≤ x ≤ u}.

Clearly, if y1 = 1 then x satisfies a1x ≤ b1 while a2x ≤ b2 is inactive, and conversely ify2 = 1.


Example 8: Scheduling problem

Given

m machines and n products

for each product j , the deadline dj and the time pjk needed to process product j onmachine k, for ≤ k ≤ m

determine an optimal schedule so as to minimize the time needed to complete allproducts, while satisfying all deadlines.

Assumptions:

products cannot be processed simultaneously on the same machine

whenever started, the execution of a product on a machine cannot be interrupted(non-preemptive scheduling).

For simplicity’s sake, also assume: each product must be processed on all the machinesaccording to the order of the machine indices.


MILP formulation

Variables:

tjk = time when product j is started on machine k, with 1 ≤ j ≤ n and 1 ≤ k ≤ m

t time when all the products are completed

yijk = 1 if product i is processed before product j on machine k, and yijk = 0otherwise, with 1 ≤ i , j ≤ n and 1 ≤ k ≤ m

min t

s.t. tjm + pjm ≤ t ∀jtjm + pjm ≤ dj ∀j

tjk + pjk ≤ tj,k+1 ∀j , ∀k ∈ {1, . . . ,m − 1}tik + pik ≤ tjk + M(1− yijk) ∀k, ∀i ,∀j with i < j (8)

tjk + pjk ≤ tik + Myijk ∀k, ∀i ,∀j with i < j (9)

t ≥ 0, tjk ≥ 0, yijk ∈ {0, 1} ∀i ,∀j , ∀k,

where M is a large enough parameter (e.g., M =∑n

j=1 dj).

Constraints (8) and (9) ensure that no pair i , j of products are processed simultaneouslyon the same machine (either i preceds j or j preceds i).

Extension: each product must be processed on subset of machines in arbitrary order.


3) An appropriate combination of continuous and binary variables allows to modelarbitrary (nonconvex) piecewise linear cost functions.

Example 7: Minimization of piecewise linear cost functions

Consider an arbitrary (not necessarily convex) piecewise linear function f (x), withf : [x1, xk ]→ R.

Suppose that x1 < x2 < . . . < xk and that f (x) is specified by the points (x i , f (x i )), fori = 1, . . . , k.

Example of minx∈[x1,xk ] f (x):

Any x ∈ [x1, xk ] and the corresponding value f (x) can be expressed as

x =k∑

i=1

λixi and f (x) =

k∑i=1

λi f (x i ) withk∑

i=1

λi = 1 and λ1, . . . , λk ≥ 0,

Clearly, the choice of the coefficients λi is not unique.


It becomes unique if we require that at most two consecutive λi can be nonzero.

Any x ∈ [x i , x i+1] is then represented as

x = λixi + λi+1x

i+1 with λi + λi+1 = 1 and λi ≥ 0, λi+1 ≥ 0.

By defining

yi = 1 if x i ≤ x ≤ x i+1 and yi = 0 otherwise, for i = 1, . . . , k − 1

the problem minx∈[x1,xk ] f (x) can be formulated as follows:

min∑k

i=1 λi f (x i )

s.t.∑k

i=1 λi = 1∑k−1

i=1 yi = 1

λ1 ≤ y1, λk ≤ yk−1

λi ≤ yi−1 + yi i = 2, . . . , k − 1

λi ≥ 0, yi ∈ {0, 1} i = 1, . . . , k

N.B.: If yj = 1 then λi = 0 for all i , 1 ≤ i ≤ n, different from j or j + 1.


Linearization of products of variables

- Linearizing the product of two (several) binary variables:

The product z = y1 · y2, with yi ∈ {0, 1} for i = 1, 2 and z ∈ {0, 1}, can be replaced by

z ≤ y1

z ≤ y2

z ≥ y1 + y2 − 1

Readily extendable to the product of three or more binary variables.

- Linearizing the product of a binary variable and bounded continuous variable:

The product z = x · y , with x ∈ [0, u], y ∈ {0, 1} and z ∈ [0, u], can be replaced by

0 ≤ z ≤ uy

z ≤ x

z ≥ x − (1− y)u

As we shall see, the product of two continuous variables cannot be linearized exactly.


chapter 3: discrete optimization integer...

Documents