part 4 nonlinear programming 4.3 successive linear programming
TRANSCRIPT
![Page 1: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/1.jpg)
Part 4 Nonlinear Programming
4.3 Successive Linear Programming
![Page 2: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/2.jpg)
Basic Concept
0 0 0 0;f f f f
y y y y y y y
![Page 3: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/3.jpg)
Approach 1: Direct Use of Linear Programs
The simplest and most direct use of the linearization construction is to replace the general nonlinear problem with a complete linearization of all problem functions at some selected estimate solution.
The linearized problem takes the form of a linear program and can be solved as such.
![Page 4: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/4.jpg)
Case 1.1 The linearly constrained case
min
. .
Because ( ) is nonlinear, the optimal solution no
longer needs
f
s t
f
y
Ay b
y 0
y
to be confined to corner points of the
feasible region but can lie anywhere within the region.
![Page 5: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/5.jpg)
Case 1.1The approximate problem
0min ;
. .
f
s t
y y
Ay b
y 0
![Page 6: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/6.jpg)
*
*
0 0 * 0
0 0 0 * 0
0 * 0
0
How is , the solution of the approximate problem,
related to , the solution of the original problem?
Note,
; ;
or
0
Since is in the direction of steepest
f f
f f f
f
f
y
y
y y y y
y y y y y
y y y
y
* 0
ascent, the
vector is in a descent direction on the surface
of .f
y y
y
![Page 7: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/7.jpg)
Bounded Line Search
*
0
0
Since is a corner point of the feasible region,
and since is feasible, all points on the line between
the two will be feasible.
The line search can be confined to the bounded line
segment,
y
y
y y
* 0 where 0 1 y y
![Page 8: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/8.jpg)
Frank-Wolfe Algorithm
( )
( )
( )
Step 1: Calculate
Step 2: Solve the LP subproblem
min
. .
Let be the optimal solution.
k
k
k
f
f
s t
y
y z
Az b
z 0
z
![Page 9: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/9.jpg)
( )
( ) ( ) ( )
0 1
( 1) ( ) ( ) ( ) ( )
( 1)
( 1) ( )
( 1)
( 1) ( )
( 1)
Step 3: Find which solves
min
Step 4: Calculate
Step 5: Convergence check.
Otherwise, go t( ) ( )
( )
k
k k k
k k k k k
k
k k
k
k k
k
f
f
f f
f
y z y
y y z y
y
y y
y
y y
y
o step 1.
![Page 10: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/10.jpg)
Remark
( ) ( ) ( )
( ) ( ) ( )
( )
( ) ( ) ( )
If the objective function is convex, then
for all feasible points. This implies
min min ; ; ...lower bound
It is also clear that
min
Thus, ; g
k k k
k k k
k
k k k
f f f
f f f
f f
f f
y y y y y
y y y z y
y y
y z y
ives an estimate of the
improvement in objective function.
![Page 11: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/11.jpg)
Case 1.2The general LP case
min
. .
0 1, 2, ,
0 1, 2, ,
1, 2, ,
j
k
i i i
f
s t
g j J
h k K
U x L i N
y
y
y
![Page 12: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/12.jpg)
Direct Linear Approximation
( ) ( ) ( )
( ) ( ) ( )
( ) ( ) ( )
( )
min
. .
0 1, 2, ,
0 1, 2, ,
1, 2, ,
Note that even if is a feasible point of the original
nonlinear proble
t t t
t t tj j
t t tk k
i i i
t
f f
s t
g g j J
h h k K
U x L i N
y y y y
y y y y
y y y y
y
( 1)m, there is no assurance that will
be feasible!
ty
![Page 13: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/13.jpg)
Remark
In order attain convergence to the true optimum, it is sufficient that at each iteration an improvement be made in both the objective function and constraint infeasibility.
This type of monotonic behavior will occur if the problem functions are mildly nonlinear.
![Page 14: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/14.jpg)
Approach 2Separable Programming
The motivation for this technique stems from the observation that a good way of improving the linear approximation over a large interval is to partition the interval into subintervals and construct individual linear approximation over each subinterval, i.e., piecewise linear approximation.
![Page 15: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/15.jpg)
Case 2.1Single-Variable Functions
![Page 16: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/16.jpg)
Line Segment in Interval k
( 1) ( )
( ) ( )( 1) ( )
( ) ( 1)
The equation of line in interval is
k kk k
k k
k k
k
f ff x f x x
x x
x x x
![Page 17: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/17.jpg)
Line Segment in Interval k
( ) ( ) ( 1) ( 1)
( ) ( 1) ( ) ( 1)
( 1) ( )( ) ( ) ( ) ( 1) ( 1) ( )
( 1) ( )
( 1) ( )( ) ( 1) ( 1) ( 1) ( )
( 1) ( )
( ) ( 1) ( 1) ( )
(
where
1 and , 0
( )
k k k k
k k k k
k kk k k k k k
k k
k kk k k k k
k k
k k k k
k
x x x
f ff x f x x x
x x
f ff x x
x x
f f f
) ( ) ( 1) ( 1)k k kf f
![Page 18: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/18.jpg)
General Formula
(1) ( )
( ) ( )
1
( ) ( )
1
( ) ( )
1
( ) ( )
For any point in
where
1 and 0 1, 2, ,
0 if 1 and 1, 2, , 2
K
Kk k
k
Kk k
k
Kk k
k
i j
x x x
x x
f x f
k K
j i i K
![Page 19: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/19.jpg)
Case 2.2Multivariable Separable Functions
1
A function is said to be separable if it can be
expressed as the sum of the single-variable functions
that each involve only one of the N variables, i.e.,
N
i ii
f f x
x
![Page 20: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/20.jpg)
General Formula
1 2
(1) (2) ( )
1 2
( ) ( ) ( ) ( ) ( ) ( )1 1 2 2
1 1 1
( ) ( )
1 1
Let
where 1, 2, ,
Then
, , ,
N
i
Ki i i i i
N
KK Kk k k k k k
N Nk k k
KNk ki i
i k
L x x x U
i N
f x x x
f f f
f
![Page 21: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/21.jpg)
General Formula
( ) ( ) ( ) ( )
1
( )
1
( )
( ) ( )
In the previous slide,
and
1
0 1, 2, ,
0 if 1 and 1,2, , 2
and
1,2, ,
i
i
Kk k k k
i i i i i ik
Kki
k
ki i
m ni i i
x x f f x
k K
n m m K
i N
![Page 22: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/22.jpg)
Restricted Basis Entry
Prior to entering one lambda into the basis (which will make it nonzero), a check should be made to ensure that no more than one other lambda associated with the same x_i is in the basis.
If there is one such lambda in the basis, it has to be adjacent.
![Page 23: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/23.jpg)
Example
41 2 1 2
21 2 1 2
1
2
max ,
. .
, 9 2 3 0
0
0
f x x x x
s t
g x x x x
x
x
![Page 24: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/24.jpg)
1 2 1 1 2 2
41 1 1
2 2 2
1 2 1 1 2 2
21 1 1
2 2 2
,
,
2
9 3
f x x f x f x
f x x
f x x
g x x g x g x
g x x
g x x
![Page 25: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/25.jpg)
1Construct the approximation over 0 3
and use 4 equidistant points
x
k
1 0 0 0
2 1 1 -2
3 2 16 -8
4 3 81 -18
( )1kx ( )
1kf ( )
1kg
![Page 26: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/26.jpg)
(1) (2) (3) (4)1 1 1 1 1 1
(1) (2) (3) (4)1 1 1 1 1 1
(1) (2) (3) (4)1 1 1 1
Thus,
0 1 16 81
0 2 8 18
1
f x
g x
![Page 27: Part 4 Nonlinear Programming 4.3 Successive Linear Programming](https://reader036.vdocument.in/reader036/viewer/2022062309/56649ea45503460f94ba8958/html5/thumbnails/27.jpg)
1 2 1 1 2 2
1 2 1 1 2 2
(2) (3) (4)1 1 1 2
(2) (3) (4)1 1 1 2 3
(1) (2) (3) (4)1 1 1 1
(1) (2) (3) (4)1 1 1 1 2 3
max ,
. .
,
2 8 18 3 9 0
( 2 8 18 3 9)
1
, , , , , 0
f x x f x f x
s t
g x x g x g x
x
x x
x x
Homework