optimization models and methods with applications in · pdf fileoptimization models and...
TRANSCRIPT
Optimization Models and Methodswith Applications in Finance
Javier Nogaleswww.est.uc3m.es/Nogales
@fjnogales
BCAM & UPV/EHU CoursesBilbao, February, 2013
Nogales - UC3M Optimization & Finance BCAM 2013 1 / 112
Some references
• J. Nocedal and S.J. Wright. Numerical Optimization. Springer-Verlag, 2006
• S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge UniversityPress, 2004
• G. Cornuejols and R. Tutuncu: Optimization Methods in Finance.Cambridge University Press, 2007
• W. T. Ziemba and R. G. Vickson (Ed.): Stochastic optimization models infinance. World Scientific, 2006
• A. Ruszczynski, A. Shapiro (Ed.): Stochastic Programming. Elsevier , 2003
• S. W. Wallace (Ed.) Applications of Stochastic programming. Book DataLimited, UK, 2005
Nogales - UC3M Optimization & Finance BCAM 2013 2 / 112
Structure of the course
1 Introduction and modeling
2 Unconstrained Optimization and Applications
3 Constrained Optimization and Applications
4 Optimization under Uncertainty with Applications
Nogales - UC3M Optimization & Finance BCAM 2013 3 / 112
Introduction
Optimization:An important tool in decision sciences and the analysis of physical systems
Several definitions:
• The discipline of applying advanced analytical methods to help makebetter decisions
• Narrowing your choices to the very best when there are virtuallyinnumerable feasible options and comparing them is difficult
• Making the best possible choice for a vector in Rn from a set ofpossible choices
Nogales - UC3M Optimization & Finance BCAM 2013 4 / 112
Introduction
Some examples:
• Airlines companies need to schedule their crew and planes in order tominimize their costs; investors build portfolios to maximize theirreturns (given a level of risk); industry companies try to maximizetheir efficiency in design and operations of their production planning;etc.
• Several applications in: Economy, Finance, Engineering, Medicine,Government, etc.
Nogales - UC3M Optimization & Finance BCAM 2013 5 / 112
Framework
• Formulating the problem: Modeling
• Solving the problem
• Studying the properties of problems and solutions
• Designing efficient algorithms to compute these solutions
• Applying the algorithms to obtain a solution
• Validating the solution - conducting sensitivity analysis
• Applying these solutions in practice
Nogales - UC3M Optimization & Finance BCAM 2013 6 / 112
Formulating the problem
Three basic elements of an optimization problem (mathematical program):
• Objective to be optimized: profit, time, energy, costs,. . .
• Variables (decisions): timetable for airplanes taking-off, amount ofmoney to invest in each asset, . . .
• Constraints: some decisions are not allowed: airplanes taking-off isconstrained for air security, risk must be controlled,. . .
Nogales - UC3M Optimization & Finance BCAM 2013 7 / 112
Formulating the problem
• General framework:
minimize f (x)
subject to x ∈ X ,
where f is the objective function of n variables and X is a subset ofRn containing the constraints of the variables. This set is calledfeasible region
• The objective can be minimized or maximized. It is equivalent notingthat
max f (x) = −min−f (x)
Nogales - UC3M Optimization & Finance BCAM 2013 8 / 112
Solving the problem
• Both commercial and open source
• Specific for different classes of problems (CPLEX and Gurobi for LP,KNITRO and SNOPT for NLP, SeDuMi and SDPT3 for SDP, . . . )
• Also available in general packages (Solver in Excel, Optimizationtoolbox and CVX in Matlab, . . . )
• Open source software: COIN-OR, http://www.coin-or.org
Nogales - UC3M Optimization & Finance BCAM 2013 9 / 112
Basic Definitions
• If X = Rn, then Unconstrained Optimization
• If f is linear and X is a polyhedron, then Linear Programming.Otherwise, Nonlinear Programming
• If f and X are convex, then Convex Optimization
• If X contains discrete variables, then Discrete Optimization or IntegerProgramming
In this course, focus on Nonlinear Optimization:
X = x : ci (x) = 0, i ∈ E , ci (x) ≥ 0, i ∈ I,
where f and ci are sufficiently differentiable. In this case, we can obtaingood local information in an efficient way
Nogales - UC3M Optimization & Finance BCAM 2013 10 / 112
Global vs Local Solutions
• For the general optimization framework:
minimize f (x)
subject to x ∈ X , (P)
• A point (decision) x∗ is a local solution of (P) if there are no betterpoints in a neighborhood of the solution, that is,
∃ε > 0 t.q. f (x∗) ≤ f (x) ∀x ∈ X s.t. ‖x − x∗‖ < ε
• A point (decision) x∗ is a global solution of (P) if there are no betterpoints in all the feasible region, that is,
f (x∗) ≤ f (x) ∀x ∈ X
Nogales - UC3M Optimization & Finance BCAM 2013 11 / 112
Global vs Local Solutions
• If previous inequalities are strict, solutions are locally strict
• Global optima are in general difficult to identify and to locate. Oneexception: convex optimization (global ≡ local)
• Local Optimization: improved solutions and easy to find themGlobal Optimization: best overall solutions but difficult to find them
Nogales - UC3M Optimization & Finance BCAM 2013 12 / 112
Existence of solutions
• Before trying to find a solution, does a solution exist for a givenproblem?
• We can assure the existence of at least one minimizer if:
• f is a continuous function and X is a compact subset (WeierstrassTheorem)
• In practice, the feasible region will be closed but not necessarilybounded. We need a stronger condition:
• f is a continuous function, X is a closed subset and f is coercive in X ,that is,
f (x)→∞ when ‖x‖ → ∞
Nogales - UC3M Optimization & Finance BCAM 2013 13 / 112
Existence of solutions
• There are difficulties if there exist infinite solutions. Hence, findisolated solutions:
• A local solution x∗ is isolated if there exist a neighborhood of x∗ suchthat there are no other local solutions
• Isolated is stronger than strict. Example:
f (x) = x4(2 + cos(1/x)), f (0) = 0
has an strict local minimum at x = 0 but there exists an infinitynumber of local minimizers near x = 0
Nogales - UC3M Optimization & Finance BCAM 2013 14 / 112
Modeling: Some Examples
• Portfolio Optimization and Asset Allocation
• Developed by Harry Markowitz in the 1950s: formalize thediversification principle in portfolio selection
• 1990 winner of the Nobel Memorial Prize for Economics
Nogales - UC3M Optimization & Finance BCAM 2013 15 / 112
Portfolio Optimization
• An investor has n available assets where she can allocate her money(buy and hold). The variable Ri represents the (random) return ofasset iThe problem is:
maximizex∑i
Rixi
subject to∑
xi = 1,
but, is this problem well-defined?
• Well-defined problem:
maximizex∑i
µixi
subject to∑
xi = 1,
where µi = E (Ri ). But, is its solution reasonable?
Nogales - UC3M Optimization & Finance BCAM 2013 16 / 112
Portfolio Optimization
• An investor has n available assets where she can allocate her money(buy and hold). The variable Ri represents the (random) return ofasset iThe problem is:
maximizex∑i
Rixi
subject to∑
xi = 1,
but, is this problem well-defined?
• Well-defined problem:
maximizex∑i
µixi
subject to∑
xi = 1,
where µi = E (Ri ). But, is its solution reasonable?
Nogales - UC3M Optimization & Finance BCAM 2013 16 / 112
Portfolio Optimization
• Markowitz: Risk measured through the variance of the portfolio return
maximizex µT x − 1
2γ xTΣx
subject to∑
xi = 1,
where Σ = Var(R) and γ is the risk-aversion coefficient
• This model allows to build the so-called efficient frontier (thoseportfolios that, for a given return, have a minimum risk-level)
Nogales - UC3M Optimization & Finance BCAM 2013 17 / 112
Portfolio Optimization
• In the classical model, µ and Σ are estimated through historical data:past returns, Rit
• Nowadays, there exist much better estimations (factor analysis,Black-Litterman, shrinked estimators, etc.)
• If short-selling is not allowed, then x ≥ 0 must be a constraint
• This is a static model: one decision in time (not a sequence ofdecisions)
• It is a quadratic (convex) problem
Nogales - UC3M Optimization & Finance BCAM 2013 18 / 112
Portfolio Optimization
• Other objective functions and constraints are possible
• Based on utilities:
maximizex ER
(U(R1x1 + · · ·+ Rnxn)
)subject to
∑xi = 1,
where U is a given utility function (logarithmic, power utility, etc.).
But, how can we solve it?
• Nonlinear Programming and Optimization under Uncertainty
Nogales - UC3M Optimization & Finance BCAM 2013 19 / 112
Portfolio Optimization
• Based on Risk Management:
minimizex VaRβ(−(R1x1 + · · ·+ Rnxn)
)subject to
∑xi = 1,
where VaRβ is the Value-at-Risk for a given 0 ≤ β ≤ 1
• This problem is nonlinear and nonconvex (local solutions)
• Conservative solutions: now required by regulators
• Other risk measures to be optimized: Conditional Value-at-Risk(CVaR) , Maximum Drawdown, etc.
• But, how can we solve it?
• Nonlinear Programming and Optimization under Uncertainty
Nogales - UC3M Optimization & Finance BCAM 2013 20 / 112
Portfolio Optimization
Extensions (following days):
• Dynamic formulation: decisions made today affect future decisions
• Dynamic risk-measures (volatiities and correlations)
• Transaction costs
• Robust portfolios
• See http://estimationrisk.blogspot.com for real performance
Nogales - UC3M Optimization & Finance BCAM 2013 21 / 112
Structure of the course
1 Introduction and modeling
2 Unconstrained Optimization and Applications
3 Constrained Optimization and Applications
4 Optimization under Uncertainty with Applications
Nogales - UC3M Optimization & Finance BCAM 2013 22 / 112
Introduction
• Once we know how to model a problem: how to solve it?
• Simplest case: unconstrained optimization
minx
f (x)
where x ∈ Rn and f : Rn → R
• Structure:
• Optimality characterization
• Applications and Examples
• Convex Optimization
Nogales - UC3M Optimization & Finance BCAM 2013 23 / 112
Introduction
• Once we know how to model a problem: how to solve it?
• Simplest case: unconstrained optimization
minx
f (x)
where x ∈ Rn and f : Rn → R
• Structure:
• Optimality characterization
• Applications and Examples
• Convex Optimization
Nogales - UC3M Optimization & Finance BCAM 2013 23 / 112
Definition and Assumptions
• The function f must satisfy:
• It has continuous (Lipschitz) second-order derivatives
• It is bounded below
• It goes to infinity when x →∞
• Main interest: find one local solution
• This solution satisfies necessary and sufficient conditions (optimalitycharacterization)
Nogales - UC3M Optimization & Finance BCAM 2013 24 / 112
Definition and Assumptions
• The function f must satisfy:
• It has continuous (Lipschitz) second-order derivatives
• It is bounded below
• It goes to infinity when x →∞
• Main interest: find one local solution
• This solution satisfies necessary and sufficient conditions (optimalitycharacterization)
Nogales - UC3M Optimization & Finance BCAM 2013 24 / 112
Optimality Conditions
• If f is sufficiently differentiable, we can use Taylor’s expansions toapproximate the objective function in a neighborhood:
f (x∗ + ∆x)− f (x∗) = ∇f (x∗)T∆x + O(‖∆x‖2)
f (x∗ + ∆x)− f (x∗) = ∇f (x∗)T∆x +1
2∆xT∇2f (x∗)∆x + O(‖∆x‖3)
• The deviations respect to the optimum:∇f (x∗)T∆x or ∇f (x∗)T∆x + 1
2 ∆xT∇2f (x∗)∆xhelp us to infer the behavior near the solution: necessary conditions
Nogales - UC3M Optimization & Finance BCAM 2013 25 / 112
Optimality Conditions
• First-order necessary conditions:If x∗ is a local minimizer, then it is easy to guess that the slope atthis point must be zero, that is,
∇f (x∗) = 0 (NC1)
A point x∗ satisfying (NC1) is called stationary point
• Second order necessary conditions:If x∗ is a local minimizer, then it is stationary and the curvature atthis point must be nonnegative, that is,
∇2f (x∗) 0 (NC2)
Nogales - UC3M Optimization & Finance BCAM 2013 26 / 112
Optimality Conditions
• If the problem is nonconvex, we need conditions with secondderivatives to guarantee local optimizers, that is, sufficient conditions
• Second-order sufficient conditions:If x∗ is a point satisfying:
∇f (x∗) = 0 (SC1)
∇2f (x∗) 0 (SC2)
then x∗ is a strict local minimizer of f
Nogales - UC3M Optimization & Finance BCAM 2013 27 / 112
Optimality Conditions
• Example:
f (x1, x2) =1
3x3
1 +1
2x2
1 + 2x1x2 +1
2x2
2 − x2 + 9
Necessary conditions: ∇f (x) =
(x2
1 + x1 + 2x2
2x1 + x2 − 1
)= 0
Therefore, there are two stationary points: xa =
(1−1
)y xb =
(2−3
)
Nogales - UC3M Optimization & Finance BCAM 2013 28 / 112
Optimality Conditions
• Sufficient conditions: ∇2f (x) =
(2x1 + 1 2
2 1
)⇒
∇2f (xa) =
(3 22 1
)y ∇2f (xb) =
(5 22 1
)
• Therefore:
∇2f (xb) is positive definite ⇒ xb is a local minimizer
∇2f (xa) is indefinite and xa is neither a local minimizer nor amaximizer
• This function has neither a global minimum nor a maximum, becausef is not bounded as x1 → ±∞
Nogales - UC3M Optimization & Finance BCAM 2013 29 / 112
Optimality Conditions
• Local minimizers not satisfying the sufficient conditions are calledsingular points
• These points are very difficult to deal with: (i) their optimality cannotbe assured through sufficient conditions with low computational time(in the absence of convexity); (ii) near these points, almost all thealgorithms are slow and erratic
Nogales - UC3M Optimization & Finance BCAM 2013 30 / 112
Optimality conditions
• Example:f1(x) = x3, f2(x) = x4, f3(x) = −x4
• All of these functions satisfy ∇f (0) = ∇2f (0) = 0 ⇒x = 0 is a candidate to be a local minimizer for all of these functions,but:
• while f2 has a local minimizer at x = 0,
f1 has a saddle point
f3 has an local maximizer
Nogales - UC3M Optimization & Finance BCAM 2013 31 / 112
Optimality Conditions
• As a summary, for stationary points:
∇2f (x∗) 0⇒ minimizer
∇2f (x∗) ≺ 0⇒ maximizer
∇2f (x∗) indefinite⇒ saddle point
∇2f (x∗) singular⇒ several performances
Nogales - UC3M Optimization & Finance BCAM 2013 32 / 112
Applications: GARCH models
• GARCH model: generalization of the autoregressive conditionalheteroscedastic (ARCH) model
• Econometric term developed in 1982 by Robert F. Engle, 2003 winnerof the Nobel Memorial Prize for Economics
• Describe an approach to estimate dynamic volatility in financialmarkets
• The GARCH model is widely used in Finance: more real-worldcontext than other forms when trying to predict prices and rates offinancial instruments
Nogales - UC3M Optimization & Finance BCAM 2013 33 / 112
GARCH model: daily S&P 500 returns
Dynamic volatility and volatility clusters
Nogales - UC3M Optimization & Finance BCAM 2013 34 / 112
GARCH model: daily S&P 500 correlations
Stock returns are uncorrelated, but dependent
Nogales - UC3M Optimization & Finance BCAM 2013 35 / 112
Classical Estimation
• Model for daily returns:rt = σtεt
where εt is a vector of disturbances and σt represents the volatilities
• Classical model. Assume i.i.d. normal returns: εt ∼ N (0, 1)Then, σt = σ for all t
• The log-likelihood to be maximized over σ is
L = −1
2
T∑t=1
(N log(2π) + log σ2 +r2t
σ2)
Apply the optimality conditions and obtain: σ2 = 1T
∑Tt=1 r
2t
• This is the sample variance
Nogales - UC3M Optimization & Finance BCAM 2013 36 / 112
GARCH(1,1) model
• GARCH(1,1) model: Volatilities are dynamic
• The log-likelihood to be maximized over (α, β) is
L = −1
2
T∑t=1
(N log(2π) + log σ2t +
r2t
σ2t
)
withσ2t = σ2 × (1− α− β) + αr2
t−1 + βσ2t−1
• In this case, the solution is not explicit: nonconvex problem
• Apply optimization algorithms
Nogales - UC3M Optimization & Finance BCAM 2013 37 / 112
Convex Optimization
• When f is convex, then local and global optimizers are equivalent andeasy to identify
• A function f is convex if
f (αx + (1− α)y) ≤ αf (x) + (1− α)f (y), ∀0 ≤ α ≤ 1
• If f is differentiable (f ∈ C 1), then it is convex if and only if
f (y) ≥ f (x) +∇f (x)T (y − x) ∀x , y
• If f ∈ C 2, then it is convex if and only if
∇2f (x) 0 ∀x
Nogales - UC3M Optimization & Finance BCAM 2013 38 / 112
Differentiable Convex Optimization
• If f is differentiable, there are two reasons why convex problems areimportant:
• In this case, there is no difference between a local and a globalminimizer, that is, every local minimizer is a global solution too.Moreover, the set of global minimizers is convex
If f is strictly convex, then the global minimizer is unique
• The necessary condition ∇f (x∗) = 0 becomes a sufficient one
Nogales - UC3M Optimization & Finance BCAM 2013 39 / 112
Nondifferentiable Convex Optimization
• What happens if f is not differentiable (but convex)?
• We say gx is a subgradient of f at x if
f (y) ≥ f (x) + gTx (y − x) ∀y
• The subdifferential of f at x is the set of all the subgradients:
∂f (x) = gx : f (y) ≥ f (x) + gTx (y − x) ∀y
• The set ∂f (x) is a closed convex set (maybe empty) even if f is notconvex
Nogales - UC3M Optimization & Finance BCAM 2013 40 / 112
Convex Optimization
• If f is not differentiable, there are two reasons why convex problemsare important:
• Again, there is no difference between a local and a global minimizer,that is, every local minimizer is a global solution too
• x∗ is a minimizer of f (x) if and only if 0 ∈ ∂f (x∗)
Nogales - UC3M Optimization & Finance BCAM 2013 41 / 112
Examples
• f (x) = ||x ||1
• ∂f (0) = x : −e ≤ x ≤ e = x : ||x ||∞ ≤ 1
• ∂f (−2e) = −e
• In general, ∂f (x) = u : uT x = ||x ||1 and ||u||∞ ≤ 1
Nogales - UC3M Optimization & Finance BCAM 2013 42 / 112
Examples
• f (x) = ||x ||2
• If x 6= 0, then ∂f (x) = 1||x ||2 x
• If x = 0, then ∂f (0) = x : ||x ||2 ≤ 1
Nogales - UC3M Optimization & Finance BCAM 2013 43 / 112
Applications
• Large regression problems. For the model, y = Xβ + u, the popularLASSO estimator is:
minβ||y − Xβ||22 + ρ||β||1
Widely used in Data Mining and Big Data analysis
Nogales - UC3M Optimization & Finance BCAM 2013 44 / 112
Applications
• Transaction costs in Finance. For the portfolio optimization problem:
maxxµT x − γ
2xTΣx − κ||x − x ||1
where κ represents the proportional transaction costs (around 20 bps)and x denotes previous portfolio allocation before rebalancing
Widely used in Portfolio Management
Nogales - UC3M Optimization & Finance BCAM 2013 45 / 112
Applications
• Jensen inequality. If f is convex and X is a random variable, thenf (E (X )) ≤ E (f (X ))
Widely used in Probability and Statistics
Nogales - UC3M Optimization & Finance BCAM 2013 46 / 112
Structure of the course
1 Introduction and modeling
2 Unconstrained Optimization and Applications
3 Constrained Optimization and Applications
4 Optimization under Uncertainty with Applications
Nogales - UC3M Optimization & Finance BCAM 2013 47 / 112
Constrained Optimization
• Now add constraints
• For instance, in the GARCH(1,1) objective function:
L = −1
2
T∑t=1
(N log(2π) + log σ2t +
r2t
σ2t
)
we can add the constraints σ2t = σ2 × (1− α− β) + αr2
t−1 + βσ2t−1,
and α + β < 1 to assure stationarity
• For instance, in the portfolio selection problem:
µT x − 1
2γ xTΣx
we can add the constraints eT x = 1 (initial wealth), x ≥ 0 (noshort-selling), Ax = b diversification by sectors, etc.
Nogales - UC3M Optimization & Finance BCAM 2013 48 / 112
Constrained Optimization
• Now add constraints
• For instance, in the GARCH(1,1) objective function:
L = −1
2
T∑t=1
(N log(2π) + log σ2t +
r2t
σ2t
)
we can add the constraints σ2t = σ2 × (1− α− β) + αr2
t−1 + βσ2t−1,
and α + β < 1 to assure stationarity
• For instance, in the portfolio selection problem:
µT x − 1
2γ xTΣx
we can add the constraints eT x = 1 (initial wealth), x ≥ 0 (noshort-selling), Ax = b diversification by sectors, etc.
Nogales - UC3M Optimization & Finance BCAM 2013 48 / 112
General Framework
• Constrained Nonlinear Optimization Problem
minimize f (x)
subject to cE(x) = 0 (NLP)
cI(x) ≥ 0
where f and ci have continuous (Lipschitz) second-order derivatives
• Goal: compute a local solution satisfying some optimality conditions
Nogales - UC3M Optimization & Finance BCAM 2013 49 / 112
Optimality conditions
• The terms KKT point and KKT conditions (Karush-Kuhn-Tucker) areused to characterize stationary points when there are constraints
• We say that the first-order KKT conditions for problem (NLP) aresatisfied at a point x∗ if there exists a vector λ∗ (Lagrangemultipliers) such that:
∇x f (x∗)−∇xc(x∗)λ∗ = 0 stationarity (KKT1)
cI(x∗) ≥ 0 and cE(x∗) = 0 feasibility (KKT2)
cI(x∗) λ∗I = 0 complementarity (KKT3)
λ∗I ≥ 0 multipliers sign (KKT4)
• A point x∗ satisfying these conditions is called a KKT point
Nogales - UC3M Optimization & Finance BCAM 2013 50 / 112
Constrained Optimization
• Example:
minimizex f (x) = (x1 − 3/2)2 + (x2 − 1/4)2
subject to c(x) =
1− x1 − x2
1− x1 + x2
1 + x1 − x2
1 + x1 + x2
≥ 0.
The first-order KKT conditions are satisfied at point x∗ = (1, 0)
Nogales - UC3M Optimization & Finance BCAM 2013 51 / 112
Constrained Optimization
• We can check it:
(−1−0.5
)−(−1 −1 1 1−1 1 −1 1
)λ∗1λ∗2λ∗3λ∗4
= 0 (KKT1)
1− 1− 01− 1 + 01 + 1− 01 + 1 + 0
≥ 0 (KKT2)
0022
λ∗1λ∗2λ∗3λ∗4
= 0 (KKT3)
λ∗ ≥ 0, (KKT4)
where λ∗ = (3/4, 1/4, 0, 0)T
Nogales - UC3M Optimization & Finance BCAM 2013 52 / 112
Constrained Optimization
• The stationarity condition (KKT1) can be written as
∇xL(x∗, λ∗) = 0,
whereL(x , λ) = f (x)− λT c(x)
is the Lagrangian function
• We need to take care with the multipliers sign: if we are maximizing,they change; if constraints are ≤, they change
• Notation: g(x) = ∇x f (x), J(x) = ∇xc(x)T
• Active constraints: A(x), those satisfied with equality at a point x
Nogales - UC3M Optimization & Finance BCAM 2013 53 / 112
Constraint Qualifications
• To obtain first-order necessary conditions, first ensure that local linearapproximations are good representations
• LICQ (linear independence constraint qualification): This condition issatisfied at a feasible point x if JA(x) (the gradients of the activeconstraints at x) is a full row rank matrix (linearly-independentgradients)
• MFCQ (Mangasarian-Fromovitz constraint qualification): Thiscondition is satisfied at a feasible point x if the gradients of theequality constraints at x are linearly independent (full row rankjacobian of the equality constraints) and if there exists p 6= 0 suchthat JA(x)p > 0 for all i ∈ A(x) ∩ I and JA(x)p = 0 for all i ∈ E
• The MFCQ is a weaker condition than the LICQ
Nogales - UC3M Optimization & Finance BCAM 2013 54 / 112
Constraint Qualifications
• Example
Feasible region:
4− x21 − 4x2
2 ≥ 0
5− (x1 − 2)2 − x22 ≥ 0
x1, x2 ≥ 0
• At x = (0, 1), the matrix JA(x) has not full rank, hence, LICQ doesnot hold.
• But the MFCQ holds because JA(x)p > 0 if p = (1,−1)
Nogales - UC3M Optimization & Finance BCAM 2013 55 / 112
Constraint Qualifications
• Example
Feasible region:
−x21 + x2 ≥ 0
3x21 − x2 ≥ 0
• At x = (0, 0), the matrix JA(x) has not full rank, hence, the LICQdoes not hold
• In this case, the MFCQ does not hold because JA(x)p > 0 wouldimply p2 > 0 and −p2 > 0. Note there does not exist a linear pathwith strict interior to reach the point (although there exist interiornonlinear paths)
Nogales - UC3M Optimization & Finance BCAM 2013 56 / 112
Constraint Qualifications
• LICQ is the most demanding, but it is very easy to check (check therank of a matrix). It implies that the multiplier vector is unique
• MFCQ is satisfied for a very large class of problems, but it is moredifficult to check (solve a linear program)
Given a point x , if the solution of
minimizep,θ − θsubject to JA(x)p − θe ≥ 0,
0 ≤ θ ≤ 1,
is obtained at θ∗ = 1, then the MFCQ is satisfied at x
With MFCQ there may be an infinite number of multiplier vectors.But these multiplier vectors are bounded
Nogales - UC3M Optimization & Finance BCAM 2013 57 / 112
First-order necessary conditions
• If a constraint qualification holds, then the KKT conditions arenecessary for a local minimizer x∗ of the constrained problem
g(x∗)− J(x∗)Tλ∗ = 0 stationarity (KKT1)
cI(x∗) ≥ 0 and cE(x∗) = 0 feasibility (KKT2)
cI(x∗) λ∗I = 0 complementarity (KKT3)
λ∗I ≥ 0 multipliers sign (KKT4)
Nogales - UC3M Optimization & Finance BCAM 2013 58 / 112
Second-order necessary conditions
• The Lagrangian hessian is denoted by
H(x , λ) = ∇2xxL(x , λ) = ∇2
xx f (x)−∑i
λi∇2xxci (x)
• When a local solution x∗ satisfies the LICQ, then it is a KKT pointand
pTH(x∗, λ∗)p ≥ 0 ∀p tal que J∗Ap = 0 (SONC)
• Condition (SONC) is equivalent to
ZTH(x∗, λ∗)Z 0,
where Z is matrix whose columns for a basis for the null space of J∗A:
JA(x∗)T =(Q1 Q2
)(R0
)= QR,
Q1 dimension n ×m, Z = Q2 dimension n × (n −m), R dimensionm ×m
Nogales - UC3M Optimization & Finance BCAM 2013 59 / 112
Second-order necessary conditions
• Example
minimizex f (x) = x21 − 2x1 + x2
2 − x23 + 4x3
subject to x1 − x2 + 2x3 = 0.
The point x∗ = (2.5,−1.5,−1)T satisfies the KKT conditions withλ∗ = 3
H(x∗, λ∗) =
2 0 00 2 00 0 −2
, Z =
1 −21 00 1
, ZTH(x∗, λ∗)Z =
(4 −4−4 6
) 0.
Hence, conditions (SONC) are satisfied at x∗ = (2.5,−1.5,−1)T
Nogales - UC3M Optimization & Finance BCAM 2013 60 / 112
Second-order sufficient conditions
• Given a KKT point x∗, it satisfies the strict complementaritycondition (SCS) if there exists a Lagrange multiplier λ∗ such thatλ∗i > 0 for all i ∈ A ∩ IThat is, all active constraints have multipliers different from zero
• A point x∗ is an isolated solution to (NLP) if: (i) x∗ is a KKT point,(ii) the LICQ and the SCS are satisfied at x∗ and (iii) for the uniqueλ∗ ∈Mλ(x∗) and for all p 6= 0 such that g∗Tp = 0, J∗Ep = 0 andJ∗I∩Ap ≥ 0, there exists ω > 0 such that pTH(x∗, λ)p ≥ ω‖p‖2
Condition (iii) is equivalent to
ZTH(x∗, λ∗)Z 0
Nogales - UC3M Optimization & Finance BCAM 2013 61 / 112
Mean-variance portfolios
• Markowitz: Investor who cares only about mean and varianceof static portfolio returns should hold the following portfolio:
maxx
µT x− γ
2xTΣx
s.t. xT e = 1,
where µ = E (R), Σ = Var(R) and γ ≡ risk aversion parameter
• From KKT conditions, the solution is:
x =1
γΣ−1µ− λγ
γΣ−1e,
where λγ = µT Σ−1e−γeT Σ−1e
• H(x∗, λ∗) = −γΣ which is negative definite (we are maximizing)
Nogales - UC3M Optimization & Finance BCAM 2013 62 / 112
Convex Optimization
• Consider the (not necessarily convex) problem
minimize f (x)
subject to cE(x) = 0 (Primal)
cI(x) ≥ 0
• Let L(x , λ) = f (x)− λT c(x) be the Lagrangian function
• The dual function is g(λ) = infx L(x , λ)
• The dual problem is
maximize g(λ)
subject to λI ≥ 0 (Dual)
Nogales - UC3M Optimization & Finance BCAM 2013 63 / 112
Convex Optimization
• Let f ∗ be the optimal value function of (Primal)
• If λI ≥ 0, then g(λ) ≤ f ∗
This is the weak duality (lower bound for f ∗)
• Hence, the dual problem provides the best bound g∗ ≤ f ∗ always
• Problem (Dual) is convex even if problem (Primal) is not
• If g∗ = f ∗, then strong duality
Nogales - UC3M Optimization & Finance BCAM 2013 64 / 112
Convex Optimization
• Slater condition (sufficient condition for strong duality):If (Primal) is convex and there exists x s.t. cI(x) > 0, then g∗ = f ∗
(duality gap is 0)
• If the problem (Primal) is convex, that is
f convex, linear equality constraints, concave inequality constraints
then
• There is no difference between a local and a global minimizer, that is,every local minimizer is a global solution too
• Under the Slater condition, x∗ is a minimizer if and only if the KKTconditions are satisfied
(replace gradients in KKT by subdifferentials)
Nogales - UC3M Optimization & Finance BCAM 2013 65 / 112
Structure of the course
1 Introduction and modeling
2 Unconstrained Optimization and Applications
3 Constrained Optimization and Applications
4 Optimization under Uncertainty with Applications
Nogales - UC3M Optimization & Finance BCAM 2013 66 / 112
Optimization under Uncertainty with Applications
• Up to now, we made the implicit assumption that the data of theproblem are all known
• Often, the problem parameters will only be known in the future, orcannot be known exactly before the problem is solved
• Two fundamentally different approaches that address optimizationunder uncertainty:
• Stochastic programming: data uncertainty is random and can beexplained by some probability distribution (average view)
• Robust optimization: data uncertainty is deterministic, but unknown.Solution behaves well under all possible realizations of the data(worst-case view)
Nogales - UC3M Optimization & Finance BCAM 2013 67 / 112
Optimization under Uncertainty with Applications
• Up to now, we made the implicit assumption that the data of theproblem are all known
• Often, the problem parameters will only be known in the future, orcannot be known exactly before the problem is solved
• Two fundamentally different approaches that address optimizationunder uncertainty:
• Stochastic programming: data uncertainty is random and can beexplained by some probability distribution (average view)
• Robust optimization: data uncertainty is deterministic, but unknown.Solution behaves well under all possible realizations of the data(worst-case view)
Nogales - UC3M Optimization & Finance BCAM 2013 67 / 112
Introduction
• Optimization under uncertainty is a decision-making framework todeal with model parameters (data) that are unknown or stochastic
• Some applications:
• Transport and logistic (unknown demand)
• Electricity markets (prices and demands are stochastic)
• Finance (asset prices and demands are unknown)
• Nuclear engineering (accident probability)
• Environment (probability of non-satisfying regulations, i.e. Kyoto)
• Airline companies (passenger demands, no shows)
Nogales - UC3M Optimization & Finance BCAM 2013 68 / 112
Example
• Portfolio Optimization:
For a given vector of portfolio weights, xThe (empirical) distribution of the portfolio return is:
Optimize, over x , some characteristic of this random variable
Nogales - UC3M Optimization & Finance BCAM 2013 69 / 112
Portfolio Optimization
• General Framework:
Given N risky assets, select the vector of portfolio weights, x, thatsolves:
max or min c(RT x)
subject to xT e = 1,
where R is the random (next-period) return vector and c(·) measuresa characteristic of the r. v. (mean-var, expected utility, risk measure,etc.)
• This is an Optimization Problem with Uncertainty
It can be solved:
• By Stochastic Programming techniques
• By Robust Optimization techniques
Nogales - UC3M Optimization & Finance BCAM 2013 70 / 112
Portfolio Optimization
• General Framework:
Given N risky assets, select the vector of portfolio weights, x, thatsolves:
max or min c(RT x)
subject to xT e = 1,
where R is the random (next-period) return vector and c(·) measuresa characteristic of the r. v. (mean-var, expected utility, risk measure,etc.)
• This is an Optimization Problem with Uncertainty It can be solved:
• By Stochastic Programming techniques
• By Robust Optimization techniques
Nogales - UC3M Optimization & Finance BCAM 2013 70 / 112
Example
• To feed his cattle, a farmer can plant his land with either wheat, cornand rice
Excess or shortfall production can be sold or bought from a wholesaler
Available land for planting: 500 acres
Wheat Corn Rice
Yield (Ton/acre) 2.5 3 20Planting cost ($/acre) 150 230 260Selling price ($/Ton) 170 150 36 (under 6000 Ton)
10 (above 6000 Ton)Purchase price ($/Ton) 238 210 -
Minimum requirement for cattle (Ton) 200 240 -
Nogales - UC3M Optimization & Finance BCAM 2013 71 / 112
Example
• xW ,C ,R : Acres of land devoted to wheat, corn and rice
• wW ,C ,R,r : Tons of wheat, corn and rice (favorable/lower price) sold
• yT ,M : Tons of wheat and corn purchased
minimizex,w ,y + 150xW + 230xC + 260xR − 170wW − 150wC − 36wR − 10wr
+ 238yW + 210yC
subject to xW + xC + xR ≤ 500
2.5xW + yW − wW ≥ 200
3xC + yC − wC ≥ 240
20xR + yR − wR − wr ≥ 0
wR ≤ 6000
x , y ,w ≥ 0
Nogales - UC3M Optimization & Finance BCAM 2013 72 / 112
Example
• Solution:
Cost: -$118600
Wheat Corn Rice
Plant (acres) 120 80 300Production (Ton) 300 240 6000
Sales (Ton) 100 - 6000Purchase (Ton) - - -
• It is profitable to have an excess of 100 wheat tons for selling
• Corn is little profitable, plant only necessary to meet minimumrequirements
• Rice is only profitable if sold at favorable price
• But, is it a reasonable solution (model)?
Nogales - UC3M Optimization & Finance BCAM 2013 73 / 112
Approaches
The previous solution is based on expected estimations of:
• Yield for each plant (it is weather-dependent)
• Sale and purchase prices (one year ahead)
• Minimum requirement (one year in advance)
But these quantities are unknown at the time to make the decision
So, how to deal with this uncertainty?
Nogales - UC3M Optimization & Finance BCAM 2013 74 / 112
Approaches
Alternatives:
• Sensitivity analysis: changes in the solution and objective respect tochanges in parameters. In general, this is a local approach (only oneparameter can be changed at the same time) and it is only effective insmall problems
• Scenario analysis or wait-and-see: The problem is solved for eachparameter scenario and the obtained solutions are studied. Thedifferent optimal solutions and objectives are aggregated and the finaloptimal solution is obtained by an heuristic
• Stochastic programming or here-and-now: All parameter scenarios aretaken into account at the same time
Nogales - UC3M Optimization & Finance BCAM 2013 75 / 112
Example: Scenario Analysis
• Consider uncertainty only in the weather. Farmer considers two morescenarios: if weather is better then yield is 20% more than that ofexpected weather; if weather is worse then yield is 20% less than thatof expected weather.
• Wait and see solutions:
Scenario Best scenario Worst scenario
Cost -$167667 -$59950
Wheat Corn Rice Wheat Corn Rice
Plant (acres) 183.33 66.67 250 100 25 375Production (Ton) 550 240 6000 200 60 6000
Sales (Ton) 350 - 6000 - - 6000Purchases (Ton) - - - - 180 -
Nogales - UC3M Optimization & Finance BCAM 2013 76 / 112
Stochastic Programming
Note that:
• the solution is quite dependent on the weather
• impossible to make an optimal decision here and now: i.e., if heplants 300 acres of rice and weather is good then he will have to sell1200 Ton at unfavorable price
The best alternative is to minimize the expected cost considering at thesame time ALL the possible scenarios: Stochastic Programming
In our example, we have two types of decisions:
• first-stage decisions (now): amount of wheat, corn and rice to plant
• second-stage decisions (once the uncertainty is known): sales andpurchases
Nogales - UC3M Optimization & Finance BCAM 2013 77 / 112
Stochastic Programming
• Suppose the three different scenarios (s = 1 if better weather, s = 2if expected weather, s = 3 if worse weather) occur with the sameprobability
• Then the stochastic programming model is the following:
minimizex,w ,y + 150xW + 230xC + 260xR +1
|S |∑s∈S
(−170wWs − 150wCs
− 36wRs − 10wrs + 238yWs + 240yCs)
subject to xW + xC + xR ≤ 500
dWsxW + yWs − wWs ≥ 200 ∀s ∈ S
dCsxC + yCs − wCs ≥ 240 ∀s ∈ S
dRsxR + yRs − wRs − wrs ≥ 0 ∀s ∈ S
wRs ≤ 6000 ∀s ∈ S
x , y ,w ≥ 0
Nogales - UC3M Optimization & Finance BCAM 2013 78 / 112
Stochastic Programming
• Solution here-and-now:
Expected cost: -$108390
Wheat Corn Rice
First stage Land (acres) 170 80 250
s = 1 Production (Ton) 510 288 6000Good Sales (Ton) 310 48 6000
Purchases (Ton) - - -
s = 2 Production (Ton) 425 240 5000Average Sales (Ton) 225 - 5000
Purchases (Ton) - - -
s = 3 Production (Ton) 340 192 4000Bad Sales (Ton) 140 - 4000
Purchases (Ton) - 48 -
Nogales - UC3M Optimization & Finance BCAM 2013 79 / 112
Stochastic Programming
• Summary:
Expected profit with expected weather $118600Expected profit with good weather $167667Expected profit with bad weather $59950
Expected profit with stochastic solution $108390
With stochastic solution:
• Rice is never sold at unfavorable price.
• Corn is only purchased if weather is bad.
• Most important: the stochastic solution is not optimal under EACHscenario but it is hedged under the risk associated to ALL the scenarios
Nogales - UC3M Optimization & Finance BCAM 2013 80 / 112
Stochastic Programming: General Framework
• Given a probability space for the parameters with uncertainty,(Ω,A,P), an stochastic optimization problem can be written asfollows:
minimizex f (x) = Eω(F (x , ω))
subject to x ∈ X .
• The probability distribution F is supposed to be known (or estimatedthrough available data)
• This is the best approach although the size of the problem can beincreased considerably
• Let x∗ and z∗ = f (x∗) be the stochastic solution and thecorresponding optimal value
Nogales - UC3M Optimization & Finance BCAM 2013 81 / 112
Wait-and-see approach
• For each possible scenario ω, the following problem is solved:
minimize F (x , ω)
subject to x ∈ X
and solutions xω and zω = F (xω, ω) are obtained. Then, we cancompute the expected cost under perfect information:
zws =∑ω
zωp(ω),
where p(ω) denotes the probability of scenario ω
• In our example, zws = −118600−167667−599503 = −115406
Nogales - UC3M Optimization & Finance BCAM 2013 82 / 112
Wait-and-see approach: EVPI
• With this approach, we can define the expected value of perfectinformation (EVPI):
EVPI = z∗ − zws,
that measures how much farmer would be willing to pay for thisperfect information: or for a good forecasting method
• An small value of EVPI indicates that more accurate forecasts do notguarantee significant profits. On the other hand, a large value ofEVPI indicates that bad forecasts will have associated a great cost
• In our example, EVPI = −108390 + 115406 = $7016. This is theamount of money that farmer would have to pay to obtain perfectinformation (via a fortune teller?)
Nogales - UC3M Optimization & Finance BCAM 2013 83 / 112
Predictive approach
• The parameter values ω are estimated (predicted) by ω and thefollowing deterministic optimization problem is solved:
minimize F (x , ω)
subject to x ∈ X
• We obtain solutions xω and zd = F (xω, ω)
• Given the solution xω, the expected cost of this approach is denotedby zd:
zd = Eω(F (xω, ω))
• In our example, because we have two-stage decisions, zd is obtainedas follows: first, the original problem is solved using ω instead of ω.Then, the first-stage decisions are fixed and a second-stage problem issolved for each scenario. Finally, the average objective function forthese problems is computed: zd = −118600−148000−55120
3 = −107240
Nogales - UC3M Optimization & Finance BCAM 2013 84 / 112
Predictive approach: VSS
• With this approach, the value of the stochastic solution (VSS) can bedefined:
VSS = zd − z∗,
that measures the value of including uncertainty in the optimizationproblem
• In other words, it measures our profit if we use the stochasticprogramming approach instead of the predictive one
In our example, VSS = −107240 + 108390 = $1150 is the gainingfrom using the stochastic solution rather than the predictive solution
• As a summary, if F (xω, ω) is convex, then:
zws ≤ z∗ ≤ zd
Nogales - UC3M Optimization & Finance BCAM 2013 85 / 112
Two-stage Stochastic Programs
• The most used stochastic program is the following:
minimizex ,y cT x + Eξ
[q(ω)T y(ω)
]subject to Ax = b,
T (ω)x + Wy(ω) = h(ω),
x , y(ω) ≥ 0,
where T (ω) is the technology matrix, W is the recourse matrix, xdenotes the first-stage decisions, and y denotes the second-stagedecisions
• If matrix W does not have uncertainty, then the problem has fixedrecourse
Nogales - UC3M Optimization & Finance BCAM 2013 86 / 112
Two-stage Stochastic Programs
• The deterministic equivalent program is the following:
minimizex ,y cT x +Q(x)
subject to Ax = b,
x ≥ 0,
whereQ(x) = Eξ [Q(x , ξ(ω))]
is the expected value of the objective function at the second stage and
Q(x , ξ(ω)) = minyq(ω)T y(ω)| Wy(ω) = h(ω)− T (ω)x , y ≥ 0
is the value for the function at the second stage given a scenario ωand a first-stage decision x
Nogales - UC3M Optimization & Finance BCAM 2013 87 / 112
Properties
• The deterministic equivalent is a deterministic nonlinear problem
• The computational cost in these problems is very expensive:evaluation of Q(x)
• Properties of Q(x) are very important when designing efficientsolution methods:
• It is continuous, but non linear. But if the probability distributions arediscrete, then it is piecewise linear
• It is convex and differentiable (if the probability distributions arecontinuous)
Nogales - UC3M Optimization & Finance BCAM 2013 88 / 112
Financial Application
• Dynamic Portfolio Selection (Asset Allocation)
Given an initial budget w0, decide (for t = 0, . . . ,T − 1)xit ≡ the amount of money to invest in each available asset i ,i = 1, . . . , n to maximize the final profit (en t = T )
• The decisions depend on the asset returns, Rit , t = 1, . . . ,T , andthey will be known progressively. Hence, the decisions are maderecursively:
decision x0 R1 is known decision x1 R2 is known · · · decision xT−1
Nogales - UC3M Optimization & Finance BCAM 2013 89 / 112
Dynamic Portfolio Selection
• The dynamic stochastic problem is the following:
maximizex ,w E (U(wT ))
subject to wt =n∑
i=1
(1 + Rit)xi ,t−1, t = 1, . . . ,T
n∑i=1
xit = wt , t = 0, . . . ,T − 1
wt ≥ 0, t = 0, . . . ,T − 1
• The variables (states) wt are random, except w0
• The variables (controls) xt are random, except x0 (unknown butdeterministic)
Nogales - UC3M Optimization & Finance BCAM 2013 90 / 112
Chance Constraints
• Portfolio Optimization and Risk Management
Now, risk is not modeled through the variance (or volatility)
Now, focus on losses (instead of profits)
Nogales - UC3M Optimization & Finance BCAM 2013 91 / 112
Chance Constraints
• Up to now, decisions feasible for (almost) all the uncertainty
• Relax this assumption to improve the objective function
Example: allow the constraints to be satisfied with certain probability(probability of losing more than 5% of invested money less than 1%)
• Portfolio Optimization and Risk Management
maximizex µT x
subject to P(RT x < −b) ≤ αxT e = 1
• With this constraint relaxation, we can achieve a larger profit butassuming a higher loss risk
Nogales - UC3M Optimization & Finance BCAM 2013 92 / 112
Chance Constraints
• Up to now, decisions feasible for (almost) all the uncertainty
• Relax this assumption to improve the objective function
Example: allow the constraints to be satisfied with certain probability(probability of losing more than 5% of invested money less than 1%)
• Portfolio Optimization and Risk Management
maximizex µT x
subject to P(RT x < −b) ≤ αxT e = 1
• With this constraint relaxation, we can achieve a larger profit butassuming a higher loss risk
Nogales - UC3M Optimization & Finance BCAM 2013 92 / 112
Chance Constraints
• Constraint P(RT x < −b) ≤ α is equivalent to VaR1−α(−RT x) ≤ b,where
VaRα(Z ) = minγ : P(Z ≤ γ) ≥ α,
represents the 100α% percentile of the variable Z
• VaR is the most-used risk measure in Finance and Insurance
• But in general, the optimization problem is non-differentiable andnonconvex. Moreover, it is not coherent
Nogales - UC3M Optimization & Finance BCAM 2013 93 / 112
Coherent Risk Measures
• A risk measure, ρ(Z ) (for a loss Z ), is coherent if:
1 it is convex: ρ(λZ1 + (1− λ)Z2) ≤ λρ(Z1) + (1− λ)ρ(Z2)
2 it is monotone: If Z2 ≥ Z1, then ρ(Z2) ≥ ρ(Z1)
3 it is positive homogeneous: ρ(λZ ) = λρ(Z ) si λ > 0
4 it is equivariant to traslations: ρ(Z + a) = ρ(Z ) + a
• In addition, coherent risk measures have very good optimizationproperties:
If f (·, ω) is convex ∀ω, then ρ(f (x , ω)) is convex
Nogales - UC3M Optimization & Finance BCAM 2013 94 / 112
Coherent risk measures
• Conditional Value-at-risk:
CVaRα(Z ) = E (Z | Z ≥ VaRα(Z ))
CVaR is more conservative than VaR: CVaRα ≥ VaRα
• But it is a coherent risk-measure. Hence, the associated RiskManagement problem is convex. Indeed, it can be reformulated as aLP !
• But it is not widely used in Finance (at least compared to VaR)
Nogales - UC3M Optimization & Finance BCAM 2013 95 / 112
Robust Optimization
Now, focus is on the worst-case scenario (under a bounded set)
Nogales - UC3M Optimization & Finance BCAM 2013 96 / 112
Robust Optimization
Stochastic Programming:
minimize f (x) = Eω(F (x , ω))
subject to x ∈ X
Robust Optimization:
minimize f (x) = supω∈Ω
F (x , ω)
subject to x ∈ X
• SP: uncertainty is described through a probability function. Itprovides good solutions in practice, but it suffers from the curse ofdimensionality
• RO: uncertainty is not stochastic (no random variables), it isdeterministic and based on bounded and convex sets. It is aworst-case scheme but the associated problems can be solved in anefficient way: they are tractable
Nogales - UC3M Optimization & Finance BCAM 2013 97 / 112
Robust Portfolio Optimization
• Portfolio Selection:
maximizex µT x − λxTΣx
subject to xT e = 1
Now suppose µ and Σ are unknown, but not stochastic
For instance, we assume the expected return of a given asset will be inthe interval 15%± 7% and the associated volatility inside 12%± 2%
• Suppose for a moment that Σ is known, and consider the followingellipsoidal uncertainty set for the unknown µ:
Ω = µ : (µ− µ)TΣ−1(µ− µ) ≤ k2
Nogales - UC3M Optimization & Finance BCAM 2013 98 / 112
Robust Portfolio Optimization
• The robust approach is:
maximizex minµ∈Ω
µT x − λxTΣx
subject to xT e = 1
or equivalently
maximizex t − λxTΣx
subject to minµ∈Ω
µT x ≥ t
(µ− µ)TΣ−1(µ− µ) ≤ k2
xT e = 1
• This is a problem with an infinite number of constraintsWe would like to deal with a tractable reformulation
Nogales - UC3M Optimization & Finance BCAM 2013 99 / 112
Robust Portfolio Optimization
• The KKT conditions of the inner problem
minimizeµ µT x
subject to (µ− µ)TΣ−1(µ− µ) ≤ k2
are
x − 2λΣ−1(µ− µ) = 0
(µ− µ)TΣ−1(µ− µ) ≤ k2
• Then, µ = µ+ 1λΣx and (assuming the constraint is active)
λ = − 1k
√xTΣx
Nogales - UC3M Optimization & Finance BCAM 2013 100 / 112
Robust Portfolio Optimization
• Hence, the optimal objective value for
minimizeµ µT x
subject to (µ− µ)TΣ−1(µ− µ) ≤ k2
is µT x − k‖Σ1/2x‖
• Now, we write this expression in our original robust formulation:
maximizex minµ∈Ω
µT x − λxTΣx
subject to xT e = 1
Nogales - UC3M Optimization & Finance BCAM 2013 101 / 112
Robust Portfolio Optimization
• Finally, the robust scheme is equivalent to the following tractableframework:
maximizex µT x − λxTΣx − k‖Σ1/2x‖subject to xT e = 1
This is a convex (non-quadratic) program:Second Order Cone Program≡ SOCP
• Now, compare with Markowitz approach:
maximizex µT x − λxTΣx
subject to xT e = 1
This is a convex QP
• The robust formulation is more conservative(new term penalizing estimation risk in µ)
Nogales - UC3M Optimization & Finance BCAM 2013 102 / 112
Robust Portfolio Optimization
• Now what happens if Σ is unknown too?
• The robust formulation is:
maximizex min(µ,Σ)∈Ω
µT x − λxTΣx
subject to xT e = 1,
where now Ω represents the uncertainty in (µ,Σ)
• For Σ, the following set leads to a tractable formulation (SOCP):
Ω = Σ 0 : ||Σ−1/2target(Σ− Σ)Σ
−1/2target ||2 ≤ δ2,
where || · || denotes the 2- or Frobenius norm and Σtarget is a targetmatrix (identity, or 1-factor matrix, or even Σ)
Nogales - UC3M Optimization & Finance BCAM 2013 103 / 112
Robust Portfolio Optimization
• In this case, the corresponding tractable formulation is equivalent tothe following norm-constrained portfolio formulation:
maxx
µT x− k‖Σ1/2x‖ − γ
2xT Σx− δ
2xT Σtargetx
subject to x>e = 1
• Solution even more conservative: another new term penalizingestimation risk in Σ
Nogales - UC3M Optimization & Finance BCAM 2013 104 / 112
Robust Optimization
• In general, robust optimization problems are typically solved withsecond-order cone programs (SOCP) (convex problems that aresolved efficiently)
• Advantages: no need of probability distribution for uncertainty.Problems solved in an efficient way
• Main limitation: conservative approach (worst-case view)
• Applications: finance (robust portfolio optimization), statistics (leastsquares with noise in explanatory variables), engineering (robustdesign), telecommunications (robust networks), signal processing,supply chain management, air traffic control, dynamic systems, etc.
• Active research topic: especially in portfolio optimization and (robust)optimal control
Nogales - UC3M Optimization & Finance BCAM 2013 105 / 112
Robust Optimization
• Robust optimization is more than a minimax approachThe solution methods are efficient!
• Chance constraints and robust optimization are related:
• Remember the chance constraints view: P(ci (x , ωi ) ≤ 0) ≥ 1− αHere, we look for feasible solutions for all the uncertainty and optimalin some sense. Hence, we look for robust solutions against uncertainty
• Robust optimization view: ci (x , ωi ) ≤ 0, ∀ωi ∈ Ωi
Nogales - UC3M Optimization & Finance BCAM 2013 106 / 112
Robust Optimization
• General formulation:
minimize f (x)
subject to ci (x , ωi ) ≤ 0, ∀ωi ∈ Ωi , i = 1, . . . ,m.
• Note in this formulation there is no probability
• Uncertainty is modelled through the uncertainty sets Ωi
(closed, bounded and typically convex)
• But the general formulation has an infinite number of constraintsWe need efficient solution methods ≡ tractable formulations
Nogales - UC3M Optimization & Finance BCAM 2013 107 / 112
Robust Optimization
• Objective of RO: obtain an equivalent reformulation that is tractable
We can achieve this objective if we restrict the form of the sets Ωi :typically ellipsoids (tractable convex sets)
• The objective function may depend on unknown parameters ω0, too.But introducing an auxiliary variable, t, it fits with the generalformulation:
minimizex ,t t
subject to ci (x , ωi ) ≤ 0, ∀ωi ∈ Ωi , i = 1, . . . ,m
maxω0∈Ω0
f (x , ω0) ≤ t
Nogales - UC3M Optimization & Finance BCAM 2013 108 / 112
Robust Linear Programming
• General formulation:
minimize cT x
subject to Ax ≤ b, ∀aTi ∈ Ωi , i = 1, . . . ,m
• It is equivalent to
minimize cT x
subject to maxai∈Ωi
aTi x ≤ bi , i = 1, . . . ,m
But this formulation is not tractable for general Ω
Nogales - UC3M Optimization & Finance BCAM 2013 109 / 112
Robust Optimization
• Ellipsoidal uncertainty: if Ω is restricted to be
Ω = A : ai = a0i + ∆iui , i = 1, . . . ,m, ‖u‖2 ≤ ρ,
then, the equivalent robust counterpart is
minimize cT x
subject to (a0i )T x + ρ‖∆ix‖2 ≤ bi , i = 1, . . . ,m
And this is a SOCP (tractable formulation)
Nogales - UC3M Optimization & Finance BCAM 2013 110 / 112
Robust Optimization
• Polyhedral uncertainty: if Ω is restricted to be a polyhedra, then theequivalent robust counterpart is a LP
• More general cases:
• For certain quadratic problems, the equivalent robust counterpart is aSDP
• For certain SOCP problems, the equivalent robust counterpart is a SDP
• But for SDP problems, the equivalent robust counterpart is in general aNP-hard problem
Nogales - UC3M Optimization & Finance BCAM 2013 111 / 112
That’s all
Thanks for your attention!!
Javier Nogaleswww.est.uc3m.es/Nogales
@fjnogales
Nogales - UC3M Optimization & Finance BCAM 2013 112 / 112