lecture 9 – nonlinear programming models topics convex sets and convex programming first-order...
TRANSCRIPT
Lecture 9 – Nonlinear Programming Models
Topics
• Convex sets and convex programming
• First-order optimality conditions
• Examples
• Problem classes
Minimize f(x)
s.t. gi(x) (, , =) bi, i = 1,…,m
x = (x1,…,xn)T is the n-dimensional vector of decision variables
f (x) is the objective function
gi(x) are the constraint functions
bi are fixed known constants
General NLP
Convex Sets
Definition: A set S n is convex if every point on the line
segment connecting any two points x1, x2 S is also in S.
Mathematically, this is equivalent to
x0 = x1 + (1–)x2 S for all such 0 ≤ ≤ 1.
x1
x2
x1x1x2
x2
S = {(x1, x2) : (0.5x1 – 0.6)x2 ≤ 1
2(x1)2 + 3(x2)2 ≥ 27; x1, x2 ≥ 0}
(Nonconvex) Feasible Region
x
1
x
2
Convex Sets and Optimization
Let S = { x n : gi(x) bi, i = 1,…,m }
Fact: If gi(x) is a convex function for each i = 1,…,m then S is a convex set.
Convex Programming Theorem: Let x n and let f (x) be a
convex function defined over a convex constraint set S. If a
finite solution exists to the problem
Minimize { f (x) : x S }
then all local optima are global optima. If f (x) is strictly
convex, the optimum is unique.
Max f (x1,…,xn)
s.t. gi(x1,…,xn) bi
i = 1,…,mx1 0,…,xn 0
is a convex program if f is concave and each gi is convex.
Convex Programming
Min f (x1,…,xn)
s.t. gi(x1,…,xn) bi
i = 1,…,mx1 0,…,xn 0
is a convex program if f is convex and each gi is convex.
x11 2 3 4 5
1
2
3
4
5
x2
Maximize f (x) = (x1 – 2)2 + (x2 – 2)2
subject to –3x1 – 2x2 ≤ –6
–x1 + x2 ≤ 3
x1 + x2 ≤ 7
2x1 – 3x2 ≤ 4
Linearly Constrained Convex Function with Unique Global Maximum
(Nonconvex) Optimization Problem
First-Order Optimality Conditions
Minimize { f (x) : gi(x) bi, i = 1,…,m }
Lagrangian: L(x,μ) = f (x) + μi gi (x) −bi( )
i=1
m
∑
∇L(x,μ) =∇f (x) + μi∇gi (x)
i=1
m
∑ =0
Optimality conditions
• Stationarity:
• Complementarity: μigi(x) = 0, i = 1,…,m
• Feasibility: gi(x) bi, i = 1,…,m
• Nonnegativity: μi 0, i = 1,…,m
Commercial optimization software cannot guarantee that a solution is globally optimal to a nonconvex program.
Importance of Convex Programs
NLP algorithms try to find a point where the gradient of the Lagrangian function is zero – a stationary point – and complementary slackness holds.
Given L(x,μ) = f(x) + μ(g(x) – b)
we want L(x,μ) = f(x) + μg(x) = 0
μg(x) – b) = 0
g(x) – b ≤ 0, μ 0
For a convex program, all local solutions are global optima.
We want to build a cylinder (with a top and a bottom) of maximum volume such that its surface area is no more than s units.
Max V(r,h) = r2h
s.t. 2r2 + 2rh = s
r 0, h 0
r
h
There are a number of ways to approach this problem. One way is to solve the surface area constraint for h and substitute the result into the objective function.
Example: Cylinder Design
h =s 2r
2
2rVolume = V = r2
[
s 2r 2
2r] =
rs 2
r 3
dVdr
= 0 r = (s
6)1/2
h = s
2r r =2(s
6)1/2
V = r 2h = 2(
s 6)
3/2
r = (s
6)1/2
h = 2(s
6)1/2
Is this a global optimal solution?
Solution by Substitution
V(r ) = rs 2
r 3 dV(r) dr =
s2
3r 2 d2V(r )
dr 2 r
d2V dr 2
0 for all r 0
Thus V(r ) is concave on r 0 so the solution is a global maximum.
Test for Convexity
• A company wants to advertise in two regions.
• The marketing department says that if $x1 is spent in region 1, sales volume will be 6(x1)1/2.
• If $x2 is spent in region 2, sales volume will be 4(x2)1/2.
• The advertising budget is $100.
Advertising (with Diminishing Returns)
Model: Max f (x) = 6(x1)1/2 + 4(x2)1/2
s.t. x1 + x2 100, x1 0, x2 0
Solution: x1* = 69.2, x2* = 30.8, f (x*) = 72.1
Is this a global optimum?
Excel Add-in Solution
1
23456789
10111213141516171819
A B C D E F G H I J K L M N O
Nonlinear Model Name: Adv100 Objective Terms Solver: Excel Solver
72.111 Type: NLP1 Linear: 0 Type: Nonlinear2 Change Goal: Max NonLinear 1: 72.111 Sens.: Yes Comp. Time 00:00
TRUE Objective: 72.111NonLinear 2: 0 Status OptimalTRUE Solve100 Variables 1 2
Change Relation Name: X1 X2Values: 69.231 30.769
Lower Bounds: 0 0
Linear Obj. Coef.: 0 0Nonlinear Obj. Terms: 8.3205 5.547Nonlinear Obj. Coef.: 6 4
ConstraintsNum. Name Value Rel. RHS Linear Constraint Coefficients
1 Con1 100 <= 100 1 12 Con2 0 <= 10000 0 0
Let rj = random variable associated with return on stock j
μj = expected return on stock j
jj variance of return for stock jWe are also concerned with the covariance terms:
ij = cov(ri, rj)
If ij > 0 then returns on i and j are positively correlated.
If ij < 0 returns are negatively correlated.
Portfolio Selection with Risky Assets (Markowitz)
• Suppose that we may invest in (up to) n stocks.
• Investors worry about (1) expected gain (2) risk.
Decision Variables: xj = # of shares of stock j purchased
R(x) = μjxj
n
j =1Expected return of the portfolio:
V(x) = ijxixj
Variance (measure of risk): n
i =1
n
j =1
V(x) = 11x1x1 + 12x1x2 + 21x2x1 + 22x2x1
= 1 + (1) + (1) + 1 = 0
Thus we can construct a “risk-free” portfolio (from variance point of view) if we can find stocks “fully” negatively correlated.
Example: If x1 = x2 = 1, we get
11 12
21 22
⎡
⎣⎢⎢
⎤
⎦⎥⎥=
1 −1−1 1
⎡
⎣⎢
⎤
⎦⎥
Nonlinear optimization models …Let pj = price of stock j
b = our total budget
risk-aversion factor (when 0 risk is not a factor)
Consider 3 different models:1) Max f (x) = R(x) – V(x)
s.t. pj xj b, xj 0, j = 1,…,n
where 0 is determined by the decision maker
n
j =1
If , then buying stock 2 is just like
buying additional shares of stock 1.
11 12
21 22
⎡
⎣⎢⎢
⎤
⎦⎥⎥=
1 11 1
⎡
⎣⎢
⎤
⎦⎥
2) Max f (x) = R(x)
s.t. V(x) , pjxj b, xj 0, j = 1,…,n
where 0 is determined by the investor. Smaller
values of represent greater risk aversion.
n
j =1
3) Min f (x) = V(x)
s.t. R(x) , pj xj b, xj 0, j = 1,…,n
where 0 is the desired rate of return
(minimum expectation) is selected by the investor.
n
j =1
Decision variables: Let (xj, yj), j = 1,…,n, be the
incremental horizontal and vertical displacement of
each link, where n 10.Constraints:
xj2 + yj
2 = 1, j = 1,…,n, each link has length 1
x1 + x2 + • • • + xn = 10, net horizontal displacement
y1 + y2 + • • • + yn = 0, net vertical displacement
Hanging Chain with Rigid Links
10ft
x
y
1 ft
each link
What is equilibrium shape of chain?
Objective: Minimize chain’s potential energy
Assuming that the center of the mass of each link is at the center of the link. This is equivalent to minimizing
12 y1 + (y1 + 1
2 y2) + (y1 + y2 + 1
2y3) + • • •
+ (y1 + y2+ • • • + yn-1 + 12 yn)
= (n 1 + 12 ]y1 + (n 2 +
12 )y2
+ (n 3 + 12)y3 + • • • +
32 yn-1 +
12 yn
Summary
Min (n j + ½)yj n
j =1s.t. xj
2 + yj2 = 1, j = 1,…,n
x1 + x2 + • • • + xn = 10
y1 + y2 + • • • + yn = 0
Is a local optimum guaranteed to be a global optimum?
Consider a chain with 4 links:
These solutions are both local minima.
Constraints xj2 + yj
2 = 1 for all j yield a nonconvex
feasible region so there may be several local
optima.
No!
Direct Current Network
Problem: Determine the current flows I1, I2,…,I7 so that the total content is minimized
Content: G(I) = 0
I
v(i)di for I ≥ 0 and G(I) = I
0
v(i)di for I < 0
Ω
2010Ω Ω100v I
I
I1 3
4
10 20Ω
I2
I5
I6 I7
Solution Approach
Electrical Engineering: Use Kirchoff’s laws to find currents when power source is given.
Operations Research: Optimize performance measure in network taking flow balance into account.Linear resistor: Voltage, v(I ) = IR
Content function, G(I ) = I 2R/2
Battery: Voltage, v(I ) = –E
Content function, G(I ) = –EI
Network Flow Model
Minimize Z = –100I1 + 5I22 + 5I3
2 + 10I42 +
10I52
subject to I1 – I2 = 0, I2 – I3 – I4 = 0, I5 – I6 = 0,
I5 + I7 = 0, I3 + I6 – I7 = 0, –I1 – I6 = 0 Solution: I1 = I2 = 50/9, I3 = 40/9, I4 = I5 = 10/9,
I6 = –50/9, I7 = –10/9
-100 I
I
I1
3
4I2
I5
I6 I71
2 3 5
64
5
5
10
10
0 0
2 2
22
Network diagram:
NLP Problem Classes• Constrained vs. unconstrained
• Convex programming problem
• Quadratic programming problem
f (x) = a + cTx + ½ xTQx, Q 0
• Separable programming problem
f (x) = j=1,n fj(xj)
• Geometric programming problem
g(x) = t=1,T ctPt(x), Pt(x) = (x1at1) . . . (xn
atn), xj > 0
• Equality constrained problems
What You Should Know About Nonlinear Programming
• How to identify a convex program.
• How to write out the first-order optimality conditions.
• The difference between a local and global solution.
• How to classify problems.