LECTURE 5: CONJUGATE DUALITY 1. Primal problem and its conjugate dual 2. Duality theory and optimality conditions 3. Relation to other type of dual problems 4. Linear conic programming problems

Motivation of conjugate duality Min f(x) (over R) = Max g(y) (over R) and h(y) = - g(y) • f(x) + h(y) = f(x) – g(y) can be viewed as “duality gap” • Would like to have (i) weak duality 0 (ii) strong duality 0 = f(x*) + h(y*)

Where is the duality information • Recall the fundamental result of Fenchel’s conjugate inequality

• Need a structure such that in general

and at optimal solutions 0 = <x*, y*> = f(x*) + h(y*)

Concept of dual cone • Let X be a cone in

• Define its dual cone • Properties: (i) Y is a cone. (ii) Y is a convex set. (iii) Y is a closed set.

Observations

Conjugate (Geometric) duality

Dual side information • Conjugate dual function

• Dual cone

Properties: 1. Y is a cone in 2. Y is closed and convex 3. both are closed and convex.

Conjugate (Geometric) dual problem

Observations

Conjugate duality theory

Conjugate duality theory

Proof

Conjugate duality theory

Example – Standard form LP

Conjugate dual problem

Dual LP

Example – Karmarkar form LP

Example – Karmarkar form LP

Example – Karmarkar form LP • Conjugate dual problem becomes

which is an unconstrained convex programming problem.

Illustration

Example – Posynomial programming • Nonconvex programming problem

• Transformation

Posynomial programming • Primal problem: Conjugate dual problem:

Dual Posynomial Programming

h(y) , supx∈En[< x , y > −f (x)] < +∞

Let g(x) =∑n

j=1 xjyj − log∑n

j=1 cjexj

∂g∂xj

= 0 ⇒ yj =cje

x∗j∑nj=1 cje

x∗j

⇒ yj > 0 and∑n

j=1 yj = 1.⇒ Ω is closed.⇒ Ω = y ∈ En|yj > 0 and

∑nj=1 yj = 1.

log(yj) = log(cjex∗

j )− log∑n

j=1 cjex∗

j

= log(cj) + x∗j − log∑n

j=1 cjex∗

j

⇒ log(yjcj

) + log∑n

j=1 cjex∗

j = x∗j

h(y) = supx∈En [< x , y > −f (x)] = g(x∗)=

∑nj=1 x∗j yj − log

∑nj=1 cje

x∗j

=∑n

j=1 yj [log(yjcj

) + log∑n

j=1 cjex∗

j ]− log∑n

j=1 cjex∗

j

=∑n

j=1 yj log(yjcj

) +∑n

j=1 yj log∑n

j=1 cjex∗

j − log∑n

j=1 cjex∗

j

=∑n

j=1 yj log(yjcj

) + (∑n

j=1 yj)log∑n

j=1 cjex∗

j − log∑n

j=1 cjex∗

j

=∑n

j=1 yj log(yjcj

)

Conjugate dual problem

Degree of difficulties • When degree of difficulty = 0, we have a system of linear

equations:

• When degree of difficulty = k, we have

Duality gap • Definition:

• Observation:

Extremality conditions • Definition:

• Corollary:

Proof of Corollary

Necessary and sufficient conditions • Corollary

• Observation

When will the duality gap vanish?

Nonlinear complementarity problem

Lagrangian Function and Saddle Point

I Let f (x) : S⋂

X and h(y) : Ω⋂

Y be a known conjugatepair with X being closed and convex and f ∈ C′(S).

I The lagrangian function is defined as

L(x , y) , f (x)− < x , y > .

I A point pair (x , y) ∈ S × Y is called a saddle point ofL(x , y) if

L(x , y) > L(x , y) > L(x , y), ∀x ∈ S, y ∈ Y ,

or

infx∈S

L(x , y) = L(x , y) = supy∈Y

L(x , y).

Saddle Point and Optimality

Theorem: (saddle point⇒ Optimality)If (x , y) ∈ S × Y is a saddle point of L(x , y), then x ∈ S ∗ andy ∈ T ∗.Proof: By definition,

infx∈S

L(x , y) = L(x , y) = supy∈Y

L(x , y)

1L(x , y) = infx∈S L(x , y)

= infx∈S[f (x)− < x , y >]= − supx∈S[< x , y > −f (x)]

Hence y ∈ Ω and L(x , y) = −h(y), (now, y ∈ Y⋂

Ω = T )

2L(x , y) = supy∈Y L(x , y)

= supy∈Y [f (x)− < x , y >]

= f (x) + supy∈Y [− < x , y >]

= f (x)− infy∈Y [< x , y >]

Since Y is a cone, x ∈ dual(Y ) = X and infy∈Y [< x , y >] = 0.Hence L(x , y) = f (x). (Now, x ∈ S

⋂X = S )

3 Putting 1 and 2 together, we have

f (x) = L(x , y) = −h(y).

Hence f (x) + h(y) = 0, and x ∈ S ∗, y ∈ T ∗.

Saddle Point and Optimality

Theorem (Convex Optimality with No Gap⇒ Saddle Point)Let f : S

⋂X and h : Ω

⋂Y be a closed convex conjugate dual

pair with no duality gap. Then (x , y) is a saddle point of L(x , y)if and only if x ∈ S ∗ and y ∈ T ∗.Proof: We need only to prove that “If x ∈ S ∗ and y ∈ T ∗, then(x , y) is a saddle point of L(x , y).” When x ∈ S ∗ and y ∈ T ∗,we have < x , y >= 0. Since there is no duality gap,L(x , y) = f (x)− < x , y >= −h(y). Hence,

−h(y)︸ ︷︷ ︸infx∈S L(x ,y)

= L(x , y) = f (x)︸︷︷︸supy∈Y L(x ,y)

Observations

1. If x ∈ S ∗ is known, then y ∈ T ∗ can be found by solving

max L(x , y) = f (x)− < x , y >s. t. y ∈ Y

When Y is linearly structures, then it is a linearly programmingproblem.2. If y ∈ T ∗ is known, then x ∈ S ∗ can be found by solving

min L(x , y) = f (x)− < x , y >s. t. x ∈ S

Linear conic programming problems • Linear Conic Programming (LCoP) A general conic optimization problem is as follows:

( )

where is a closed and convex co

minimize subject to

" " is a linear operator line an

ke "inner product."d

P c xA x b

x∈=

Dual linear conic dual problem

Duality theorems for Linear CP

Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem): (i) If problems (CP) and (CD) are both feasible, then they have optimal solutions. (ii) If one of the two problems has an interior feasible solution with a finite optimal objective value, then the other one is feasible and has the same optimal objective value. (iii) If one of the two problems is unbounded, then the other has no feasible solution. (iv) If (CP) and (CD) both have interior feasible solutions, then they have optimal solutions with zero duality gap.

Examples of conic programs

Example of conic programs

Example of conic programs

Semi-definite Programming (SDP)

Duality theorems for SDP

Quadratic programming problem

Conjugate dual QP

Conjugate dual QP

Conjugate dual QP