slides minimax problems - hkust€¦ · form as it is an lp and we know that lps do not have...

Minimax Problems

Daniel P. Palomar

Hong Kong University of Science and Technolgy (HKUST)

ELEC547 - Convex Optimization

Fall 2009-10, HKUST, Hong Kong

Outline of Lecture

• Introduction

• Matrix games

• Bilinear problems

• Robust LP

• Convex-concave games

• Lagrange game

• Summary

Daniel P. Palomar 1

Introduction

• Games are optimization problems with more than one decision

maker (or player, using the terminology of game theory), often

with conflicting goals.

• The general theory of games is complicated, but some classes of

games are closely related to convex optimization and have a nice

theory.

• Game theory goes back to the mid-1990s (1947 book by von

Neumann and Morgenstein and Nash theorem in 1950).

Daniel P. Palomar 2

Foreword by Nash in 2008 JSAC Special Issue

Daniel P. Palomar 3

Matrix Game

• A matrix game is a two-player zero-sum discrete game that can be

described by a payoff matrix.

• Player 1 chooses his strategy (index k) to minimize the payoff Pkl.

• Player 2 chooses his strategy (index l) to maximize the payoff Pkl.

• Pure strategies: each user is allowed to choose only one strategy.

• Mixed strategies: each user is allowed to randomize over a set of

strategies.

Daniel P. Palomar 4

Pure-Strategy Matrix Game

• Player 1 can expect that, for a given choice k, player 2 will choose

the largest entry in row k. This means that he has to choose the

row with the minimum largest entry:

mink

maxl

Pkl.

• Similarly, player 2 expects that, for a given choice l, player 1 will

choose the smallest entry in column l. He will then choose the

column with the maximum smalles entry:

maxl

mink

Pkl.

• But, which one is the correct formulation? Are they the same?

Daniel P. Palomar 5

Pure-Strategy Matrix Game (II)

• In general, the two formulations lead to different results.

• However, we can show that one is smaller than the other.

Lemma:

maxl

mink

Pkl ≤ mink

maxl

Pkl.

Proof.min

eyf (x, y) ≤ max

exf (x, y) ∀x, y

maxex

miney

f (x, y) ≤ miney

maxex

f (x, y) .

Daniel P. Palomar 6

Pure-Strategy Matrix Game (II)

• But, in general, for pure-strategy matrix games

maxl

mink

Pkl 6= mink

maxl

Pkl.

• This is related to the concept of Nash equilibrium (NE).

• If the game admits a NE, then

maxl

mink

Pkl = mink

maxl

Pkl.

• Nash famous theorem shows that for games with N players, under

some conditions, a NE exists.

Daniel P. Palomar 7

Mixed-Strategy Matrix Game

• Now consider that each player chooses a strategy with some

probability (independent of the other user).

• Suppose that u and v are the mixed strategies for players 1 and 2.

• The expected payoff is∑

i,j

uivjPij = uTPv.

• Reasoning as before, player 1 will choose u as the solution to

minu

maxv

uTPv

s.t. u ≥ 0 , 1Tu = 1

v ≥ 0 , 1Tv = 1.

Daniel P. Palomar 8

Mixed-Strategy Matrix Game (II)

• Since maxv

uTPv = maxi

[PTu

]i, we can rewrite the problem as

minu

maxi

[PTu

]i

s.t. u ≥ 0 , 1Tu = 1

or, in epigraph form,

(P1)

minimizeu,t

t

subject to u ≥ 0 , 1Tu = 1

t1 ≥ PTu.

Daniel P. Palomar 9

Mixed-Strategy Matrix Game (III)

• Similarly, for player 2 we have

(P2)

maximizeu,µ

µ

subject to v ≥ 0 , 1Tv = 1

Pv ≥ µ1.

• Since max-min ≤ min-max,

p⋆2≤ p⋆

1.

• We can interpret the difference p⋆1− p⋆

2as the advantage conferred

on a player by knowing the opponent’s strategy.

Daniel P. Palomar 10

Mixed-Strategy Matrix Game (IV)

• In this case, however, we have a stronger characterization:

Theorem: For feasible mixed-strategy matrix games:

p⋆1

= p⋆2.

Proof. Let’s find the dual of problem P1. The Lagrangian is

L (t,u; λ, σ, µ) = t + λT(PTu− t1

)− σTu− µ

(1Tu− 1

)

= t(1 − λT1

)+ (Pλ − σ − µ1)T

u + µ,


Mixed-Strategy Matrix Game (V)

Proof. (cont’d) the dual function is

g (λ, σ, µ) =

{µ if λT1 = 1 and Pλ = σ + µ1 ≥ µ1

−∞ otherwise,

and the dual problem is

maximizeλ,µ

µ

subject to λ ≥ 0 , 1Tλ = 1Pλ ≥ µ1

which happens to be problem P2. The result follows because problems P1 and P2

are dual of each other and strong duality holds (since the LPs are feasible).


Bilinear Problems

• Consider the following bilinear problem (more general than the

mixed-strategy matrix game):

minx

maxy

xTPy

s.t. Ax ≤ b

Cy ≤ d.

• Again, max-min ≤ min-max, but do we have equality?

• In this case, we cannot proceed as before since maxy

xTPy 6=

maxi

[PTx

]i.


Bilinear Problems (I)

• Sometimes, depending on the problem structure, the inner maxi-

mization can be solved in closed form (like in the application of

worst-case robust beamforming design).

• In general, however, the inner maximization does not have a closed

form as it is an LP and we know that LPs do not have closed-form

solutions (except in particular cases).

• We can resort to duality to deal with the inner maximization

maxy

xTPy

s.t. Cy ≤ d .


Bilinear Problems (II)

Lemma: The dual problem of the inner maximization is

minλ

dTλ

s.t. PTx = cTλ

λ ≥ 0 .

Proof. The Lagrangian is

L (y;λ) =(PTx

)Ty + λT (d− Cy)

=(PTx − CTλ

)Ty + dTλ,

the dual function isg (λ) =

{dTλ if PTx = CTλ

−∞ otherwise,

and the dual problem follows.


Bilinear Problems (III)

• We are now ready to rewrite the minimax problem in the more

convenient form of a minimization problem:

Lemma: The original minimax problem (assuming feasibility of the

inner maximization) can be rewritten as the following LP:

(Minimax-LP)

minimizex,λ

dTλ

subject to PTx = CTλ

λ ≥ 0

Ax ≤ b.

Proof. Since the original problem has a nonempty feasible set, strong duality

holds and the primal value is equal to the dual value.


Bilinear Problems (IV)

• Let’s deal now with the maximin formulation (as opposed to the

minimax):

maxy

minx

xTPy

s.t. Ax ≤ b

Cy ≤ d.

Lemma: The dual problem of the inner minimization is

maxν

−bTν

s.t. Py + ATν = 0

ν ≥ 0 .


Bilinear Problems (V)

Lemma: The original maximin problem (assuming feasibility of the

inner minimization) can be rewritten as the following LP:

(Maximin-LP)

maximizey,ν

−bTν

subject to Py + ATν = 0

ν ≥ 0

Cy ≤ d.

• So far, we have been able to rewrite the maximin and minimax

problems as LPs.


Bilinear Problems (VI)

• At this point, however, we don’t know yet if maximin equals

minimax, which is answered in the next result:

Theorem: For feasible bilinear problems:

max-min = min-max.

Proof. It follows by showing that (Minimax-LP) and (Maximin-LP) are dual ofeach other and strong duality for feasible LPs.

Let’s find the dual of (Minimax-LP). The Lagrangian is

L (x,λ; ν,σ,µ) = dTλ + νT (Ax− b) − σTλ + µT(PTx− CTλ

)

= (d− σ − Cµ)Tλ +

(ATν + Pµ

)Tx− bTν,


Bilinear Problems (VII)

Proof. (cont’d) the dual function is

g (ν, σ, µ) =

{−bTν if d− Cµ = σ ≥ 0 and ATν + Pµ = 0

−∞ otherwise,

and the dual problem is

maximizeν,µ

−bTν

subject to Pµ + ATν = 0

ν ≥ 0

d ≥ Cµ

which is identical to (Maximin-LP).


Robust LP

• An LP is of the form:

minimizex

cTx

subject to aTi x + bi ≤ 0 i = 1, · · · ,m.

• The parameters of the problem c, (ai, bi) for i = 1, · · · ,m are

usually assumed to be perfectly known.

• In real applications, however, there may be uncertainty in these

parameters. This leads to robust optimization.

• We will next consider the case where knowledge of the parameter

ai is imperfect: ai ∈ Ai where the uncertainty set Ai is known.


Robust LP (I)

• The uncertainty set can be modeled in different ways, e.g., as an

ellipsoid (as in the application of robust beamforming) or as a

polyhedron.

• We will model Ai as an affine transformation of a polyhedron:

Ai = {ai = ai + Biui, Diui ≤ di} .

• The robust LP (to imperfect knowledge of ai) is then

minimizex

cTx

subject to supui∈Ui

(ai + Biui)Tx + bi ≤ 0 i = 1, · · · ,m

where Ui = {ui | Diui ≤ di}.


Robust LP (II)

• We can now rewrite the robust formulation as

minimizex

cTx

subject to fi (x) ≤ 0 i = 1, · · · ,m

where fi (x) is the optimal value of

maximizeui

(xTBi

)ui +

(aT

i x + bi

)

subject to Diui ≤ di

whose dual problem is

minimizezi

dTi zi +

(aT

i x + bi

)

subject to DTi zi = BT

i x

zi ≥ 0.


Robust LP (III)

• Now, from strong duality, the dual value equals the maximum value

fi (x). Therefore, we have the following result:

Theorem: The robust LP can be rewritten as the following LP:

minimizex,{zi}

cTx

subject to dTi zi +

(aT

i x + bi

)≤ 0 i = 1, · · · ,m

DTi zi = BT

i x

zi ≥ 0.


Convex-Concave Games

• Consider now a more general game where the payoff function is

f (x,y) with player 1 minimizing over x and player 2 maximizing

over y.

• We already know that

maxy

minx

f (x,y) ≤ minx

maxy

f (x,y) .

• We say that (x⋆,y⋆) is a solution of the game (Nash equilibrium

or saddle point) if

f (x⋆,y) ≤ f (x⋆,y⋆) ≤ f (x,y⋆) ∀x,y.

• In words: at a saddle point, neither player can do better by

unilaterally changing his strategy.


Convex-Concave Games (I)

• Convex-concave problems have been extensively characterized and

we now state two of the nicest results:

Lemma: If f is convex-concave (and some other conditions), then a

saddle-point exists.

Lemma: If a saddle-point exists, then max-min = min-max.

Proof. From the existence of a saddle-point, we have

f (x⋆,y⋆) = minx

f (x,y⋆) ≤ maxy

minx

f (x,y)

as well asf (x⋆,y⋆) = max

yf (x⋆,y) ≥ min

xmax

yf (x⋆,y) ,


Convex-Concave Games (II)

Proof. (cont’d) which implies

maxy

minx

f (x,y) ≥ minx

maxy

f (x⋆,y) .

This combined with the well-known inequality

maxy

minx

f (x,y) ≤ minx

maxy

f (x,y)

shows thatmax

ymin

xf (x,y) = min

xmax

yf (x,y) .


Lagrange Game

• We will now study a very particular game related to Lagrange

duality.

• Consider the Lagrangian of a general optimization problem:

L (x;λ) = f0 (x) +

m∑

i=1

λifi (x) .

• Observe that

supλ≥0

L (x;λ) =

{f0 (x) if fi (x) ≤ 0 i = 1, · · · ,m

+∞ otherwise


Lagrange Game (I)

• Therefore, we can express the optimal value of the primal problem

asp⋆ = inf

xsupλ≥0

L (x;λ) .

• On the other hand, by definition of dual function (as infimum of

the Lagrangian), we have

d⋆ = supλ≥0

g (λ) = supλ≥0

infx

L (x;λ) .

• Thus, from the fact that max-min ≤ min-max, we have

d⋆ ≤ p⋆,

i.e., weak duality holds!


Lagrange Game (II)

• Recall that max-min = min-max for a convex-concave function.

• Now, if the primal problem is convex, then the Lagrangian L (x;λ)

is convex in x and concave (linear in fact) in λ.

• Thus (caution: additional assumptions are needed as the dual

feasible set is not compact),

d⋆ = p⋆,

i.e., strong duality holds.


Summary

• We have explored minimax problems (two-player zero-sum games).

• In practice, minimax formulations appear in robust optimization.

• In some cases, a saddle-point or Nash equilibrium exists and then

max-min = min-max.

• Different ways to deal with minimax problems: 1) use specific

numerical methods for minimax problems, 2) solve in closed-form

the inner maxim., 3) rewrite the inner maxim. as the dual minimiz.,

4) use a technique like the S-lemma to get rid of the inner maxim.

• Game theory considers the more general and complicated case of

N players.


References on Minimax

• R. T. Rockafellar, Convex Analysis , 2nd ed. Princeton, NJ: Princeton Univ. Press, 1970.

• D. P. Bertsekas, A. Nedic, and A. E. Ozdaglar, Convex Analysis and Optimization, Athena

Scientific, Belmont, MA, 2003.

• Daniel P. Palomar, John M. Cioffi, and Miguel A. Lagunas, “Uniform Power Allocation in

MIMO Channels: A Game-Theoretic Approach,” IEEE Trans. on Information Theory, vol.

49, no. 7, July 2003.

• Antonio Pascual-Iserte, Daniel P. Palomar, Ana Perez-Neira, Miguel A. Lagunas, “A Robust

Maximim Approach for MIMO Comm. with Imperfect CSI Based on Convex Optimization,”

IEEE Trans. on Signal Processing, vol. 54, no. 1, Jan. 2006.

• Jiaheng Wang and Daniel P. Palomar, “Worst-Case Robust Transmission in MIMO Channels

with Imperfect CSIT,” IEEE Trans. on Signal Processing , vol. 57, no. 8, pp. 3086-3100,

Aug. 2009.


References on Game Theory

• J. Osborne and A. Rubinstein, A Course in Game Theory , MIT Press, 1994.

• Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, “Competitive Design of Multiuser

MIMO Systems based on Game Theory: A Unified View,” IEEE JSAC: Special Issue on Game

Theory , vol. 25, no. 7, Sept. 2008.

• Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, “Optimal Linear Precoding

Strategies for Wideband Noncooperative Systems Based on Game Theory,” IEEE Trans. on

Signal Processing , vol. 56, no. 3, March 2008.

• Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, “Asynchronous Iterative Water-

Filling for Gaussian Frequency-Selective Interference Channels,” IEEE Trans. on Information

Theory , vol. 54, no. 7, July 2008.

• Gesualdo Scutari, Daniel P. Palomar, and Sergio Barbarossa, “Cognitive MIMO Radio: A

Competitive Optimality Design Based on Subspace Projections,” IEEE Signal Processing

Magazine, Nov. 2008.


slides minimax problems - hkust€¦ · form as it is an lp and we know that lps do not have...

Documents