lecture 7: lagrangian relaxation and duality...
TRANSCRIPT
Lecture 7: Lagrangian Relaxation and Duality Theory
(3 units)
Outline
I Lagrangian dual for linear IP
I Lagrangian dual for general IP
I Dual Search
I Lagrangian decomposition
1 / 23
Joseph Louis Lagrange
Joseph Louis Lagrange (1736-1813)
Lagrange was one of the creators of the calculus of variations,deriving the Euler-Lagrange equations for extrema of functionals.He also extended the method to take into account possibleconstraints, arriving at the method of Lagrange multipliers.
2 / 23
Lagrangian Dual for Linear Integer Programming
I IP with “nice” and complicated constraints:
max cT x
s.t. Ax ≤ b (nice constraints)Dx ≤ d (complicated constraintsx ∈ Zn
+.
Observation: Dropping Dx ≤ d results in an easy problem.
3 / 23
I More general form:
(IP) max cT x
s.t. Dx ≤ d
x ∈ X ,
where Dx ≤ d is m hard constraints.I Largrangian relaxation:
(LP(u)) z(u) = max cT x + uT (d − Dx)
s.t. x ∈ X .
Property: Problem (LP(u)) is a relaxation of (IP) for allu ≥ 0.
I Lagrnagian dual of (IP):
(LD) minu≥0
z(u).
For equality constraints: Dx = d . The Lagrangain dual is
(LD) minu∈Rm
z(u).
4 / 23
Example 1
I Uncapacitated facility location problem:The uncapacitatedfacility location problem (UFLP) involves locating an undeter-mined number of facilities to minimize the sum of the(annualized) fixed setup costs and the variable costs of servingthe market demand from these facilities.
I Formulation:
(IP) max∑
i∈M
∑
j∈N
cijxij −∑
j∈N
fjyj
s.t.∑
j∈N
xij = 1, i ∈ M
xij − yj ≤ 0, i ∈ M, j ∈ N
x ∈ R|M|×‖N|, y ∈ {0, 1}|N|.
Locations: N = {1, . . . , n}; Clients: M = {1, . . . ,m}, cij :profit
5 / 23
I Lagrangian relaxation (IP(u)):
z(u) = max∑
i∈M
∑
j∈N
(cij − ui )xij −∑
j∈N
fjyj +∑
i∈M
ui
s.t. xij − yj ≤ 0, i ∈ M, j ∈ N
x ∈ R|M|×‖N|, y ∈ {0, 1}|N|.I Decomposition: z(u) = zj(u) +
∑i∈M ui , where
zj(u) = max∑
i∈M
(cij − ui )xij − fjyj
s.t. xij − yj ≤ 0, i ∈ M
xij ≥ 0, i ∈ M, yj ∈ {0, 1}.
It is a subproblem for location j . We have
zj(u) = max{0,∑
i∈M
max(cij − ui , 0)− fj}.
6 / 23
Example 2
I Generalized assignment problem (GAP). The generalizedassignment problem is an excellent example for illustratingLagrangian relaxation because it is rich with readily apparentstructure.
I Formulation:
Z = minm∑
i=1
n∑
j=1
cijxij (1)
s.t.m∑
i=1
xij = 1, j = 1, . . . , n (2)
n∑
j=1
aijxij ≤ bi , i = 1, . . . ,m (3)
xij ∈ {0, 1}, for all i , j . (4)
7 / 23
I Relaxations 1:
ZD1(u) = minm∑
i=1
n∑
j=1
cijxij +n∑
j=1
uj(m∑
i=1
xij − 1)
s.t. (3), (4)
= minm∑
i=1
n∑
j=1
(cij + uj)xij −n∑
j=1
uj
s.t. (3), (4).
This problem reduces to m 0-1 knapsack problems and canthus be solved in time proportional to n
∑mi=1 bi .
8 / 23
I Relaxation 2
ZD2(v) = minm∑
i=1
n∑
j=1
cijxij +m∑
i=1
vi (n∑
j=1
aijxij − bi )
s.t. (2), (4)
= minn∑
j=1
m∑
i=1
(cij + viaij)xij −m∑
i=1
vibi
s.t. (2), (4).
Since constraints (2) are generalized upper bound (GUB)constraints, it is 0-1 GUB problem. Such a problem is easilysolved in time proportional to nm by determiningmini (cij + viaij) for each j and setting the associated xij = 1.Remaining xij are set to zero.
I Classical paper: Marshall L. Fisher, The Lagrangian RelaxationMethod for Solving Integer Programming Problems, Manage.Sci., Vol. 50, No. 12 Supplement, December 2004, pp.1861–1871. (Original, MS, Vol. 27(1), 1-18, 1981).
9 / 23
How Good is Lagrangian Dual?
I Consider linear program:
min{cT x | Ax ≤ b, Cx ≤ d , x ∈ X},where X is a finite integer set, e.g., X = {0, 1}n.
I Let {x1, . . . , xN} be the finite integer points in{x ∈ X | Cx ≤ d}.
I Dual problem:
v(D) = maxλ≥0
d(λ)
= maxλ≥0
minx∈X
[cT x + λT (Ax − d)]
= maxλ≥0
mini=1,...,N
[cT x i + λT (Ax i − b)]
= max η
s.t. cT x i + λT (Ax i − b) ≤ η, i = 1, . . . ,N,
λ ≥ 0, η ∈ R.
10 / 23
I The latter problem is a linear problem. Taking its dual gives
v(D) = minN∑
i=1
µi (cT x i )
s.t.N∑
i=1
µi (Ax i − d) ≤ 0,
N∑
i=1
µi = 1, µi ≥ 0.
Setting x =∑N
i=1 µixi , we imply that (D) is equivalent to the
following LP:
(PR) min cT x
s.t. Ax ≤ b,
x ∈ Co({x ∈ X | Cx ≤ d}).
11 / 23
I Theorem:
v(D) = v(PR) = min{cT x | Ax ≤ b, x ∈ Co({x ∈ X | Cx ≤ d})}.
I v(D) ≥ v(LP) = min{cT x | Ax ≤ b, Cx ≤ d}).I If Co({x ∈ X | Cx ≤ b}) = {x ∈ X | Cx ≤ b}, then
v(D) = v(LP). Lagrangian bound is equal to the LPrelaxation. Example: {x | Zn
+ | l ≤ x ≤ u}.
12 / 23
Figure: Lagrangian bound for linear integer programming.
13 / 23
Key issues in Lagrangian dual
I How to solve the Lagrangian relaxation?
I How to solve the dual problem?
I When Lagrangian relaxation useful?
14 / 23
Solving Relaxation Problem
I Linear integer program;
I Many combinatorial optimization problems
I separable integer programming (knapsack problem)
I 0-1 unconstrained quadratic problem
I 0-1 quadratic knapsack program
15 / 23
Dual Search Methods
I Problem: How to solve the dual problem (D)?
(D) maxλ≥0
d(λ).
I Three Basic Methods:I Subgradient methodI Outer approximation methodI Bundle method
16 / 23
Subgradient methodI Subgradient method
λk+1 = P+(λk + skξk/‖ξk‖),where sk > 0 is the stepsize.
I Stepsize rules for sk :I Rule 1 for stepsize (constant):
sk = ε,
where ε > 0 is a constant.I Rule 2 for stepsize:
+∞∑
k=1
s2k < +∞ and
+∞∑
k=1
sk = +∞.
I Rule 3 for stepsize:
sk → 0, k → +∞, and+∞∑
k=1
sk = +∞.
17 / 23
I A sophisticated stepsize rule:
sk = ρwk − d(λk)
‖ξk‖ , 0 < ρ < 2,
where wk is an approximation of the optimal value v(D),wk ≥ d(λk) and ξk 6= 0.
18 / 23
Outer approximation methodI Rewrite dual problem (D) as
max µ
s.t. µ ≤ f (x) + λT (g(x)− b),∀ x ∈ X ,
λ ≥ 0.
I Approximate the LP by:
max µ
s.t. µ ≤ f (x j) + λT (g(x j)− b), ∀x j ∈ T k ,
λ ≥ 0.
I Finite convergenceI Singly constrained case of (P):
(Ps) min f (x)
s.t. g(x) ≤ b
x ∈ X .
19 / 23
0 0.5 1 1.5 2 2.5 3 3.5 4−4
−2
0
2
4
6
8
d(λ)
λ1 =5/3 λ2=2
l1
l2
l3
λ
y
Figure: Dual function.
20 / 23
Lagrangian decomposition
I Purpose:I Induce decomposition of the problem into independent
subproblems;I Capture different structural characteristics of the problem;I Obtain stronger bounds than by standard Lagrangean
relaxation schemes
I How?I Identify the parts of the problem that should be split;I Replace variables in each part by copies or substitute new
expressions;I Dualize the copy or substitution expression.
21 / 23
I Convex program with nice and hard constraints:
(CP) min f (x) + g(x)
s.t. x ∈ X (nice constraints)x ∈ Y (hard constraints)
I Lagrangian decomposition via copying constraints
(CP) min f (x) + g(z)
s.t. x ∈ X (nice constraints)x = z , (link constraints)z ∈ Y (hard constraints)
22 / 23
I Associating a multiplier to each link constraint:
(CP) min f (x) + g(z)
s.t. x ∈ X
λ ⇒ x = z ,
z ∈ Y
I Decomposition:
d(λ) = min f (x) + g(z) + λT (x − z)
s.t. x ∈ X
z ∈ Y
= min f (x) + λT x + min g(z)− λT z
s.t. x ∈ X , s.t. z ∈ Y
= d2(λ) + d2(λ) ⇒ two easier subproblems!
23 / 23