lecture 5: conjugate duality · duality theorems for linear cp • theorem 3 (conic duality theorem...

LECTURE 5: CONJUGATE DUALITY 1. Primal problem and its conjugate dual 2. Duality theory and optimality conditions 3. Relation to other type of dual problems 4. Linear conic programming problems

Motivation of conjugate duality Min f(x) (over R) = Max g(y) (over R) and h(y) = - g(y) • f(x) + h(y) = f(x) – g(y) can be viewed as “duality gap” • Would like to have (i) weak duality 0 (ii) strong duality 0 = f(x*) + h(y*)

Where is the duality information • Recall the fundamental result of Fenchel’s conjugate inequality

• Need a structure such that in general

and at optimal solutions 0 = <x*, y*> = f(x*) + h(y*)

Concept of dual cone • Let X be a cone in

• Define its dual cone • Properties: (i) Y is a cone. (ii) Y is a convex set. (iii) Y is a closed set.

Observations

Conjugate (Geometric) duality

Dual side information • Conjugate dual function

• Dual cone

Properties: 1. Y is a cone in 2. Y is closed and convex 3. both are closed and convex.

Conjugate (Geometric) dual problem

Observations

Conjugate duality theory

Conjugate duality theory

Proof

Conjugate duality theory

Example – Standard form LP

Conjugate dual problem

Dual LP

Example – Karmarkar form LP

Example – Karmarkar form LP

Example – Karmarkar form LP • Conjugate dual problem becomes

which is an unconstrained convex programming problem.

Illustration

Example – Posynomial programming • Nonconvex programming problem

• Transformation

Posynomial programming • Primal problem: Conjugate dual problem:

Conjugate dual problem

Degree of difficulties • When degree of difficulty = 0, we have a system of linear

equations:

• When degree of difficulty = k, we have

Duality gap • Definition:

• Observation:

Extremality conditions • Definition:

• Corollary:

Proof of Corollary

Necessary and sufficient conditions • Corollary

• Observation

When will the duality gap vanish?

Nonlinear complementarity problem

Lagrangian function and duality • Definition:

Observations

Observations

Saddle point theorem

Observations

Observations

Proof of saddle point theorem

Linear conic programming problems • Linear Conic Programming (LCoP) A general conic optimization problem is as follows:

( )

where is a closed and convex co

minimize subject to

" " is a linear operator line an

ke "inner product."d

P c xA x b

x∈=

Dual linear conic dual problem

Duality theorems for Linear CP

Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem): (i) If problems (CP) and (CD) are both feasible, then they have optimal solutions. (ii) If one of the two problems has an interior feasible solution with a finite optimal objective value, then the other one is feasible and has the same optimal objective value. (iii) If one of the two problems is unbounded, then the other has no feasible solution. (iv) If (CP) and (CD) both have interior feasible solutions, then they have optimal solutions with zero duality gap.

Examples of conic programs

Example of conic programs

Example of conic programs

Semi-definite Programming (SDP)

Duality theorems for SDP

Quadratic programming problem

Conjugate dual QP

Conjugate dual QP

Conjugate dual QP