introduction to nonlinear programming...
TRANSCRIPT
![Page 1: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/1.jpg)
Introduction to Nonlinear Programming (NLP)This lecture was adapted from Thomas W. Reiland, North Carolina State
University and from Daniel P. Loucks & Eelco van Beek, UNESCO-Delft
Hydraulics
Dr. Arturo S. Leon, BSU (Spring 2010)
1© Arturo S. Leon, BSU, Spring
2010
![Page 2: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/2.jpg)
7-
2
Nonlinear Programming (NLP)
� An NLP problem has a nonlinear objective function and/or one or more nonlinear constraints.
� The mathematics involved in solving NLPs is quite different than for LPs.
![Page 3: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/3.jpg)
7-
3
Possible Optimal Solutions to NLPs (not occurring at corner points)
objective function level curve
optimal solution
Feasible Region
linear objective,nonlinear constraints
objective function level curve
optimal solution
Feasible Region
nonlinear objective,nonlinear constraints
objective function level curve
optimal solution
Feasible Region
nonlinear objective,linear constraints
objective function level curves
optimal solution
Feasible Region
nonlinear objective,linear constraints
![Page 4: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/4.jpg)
Steepest Descent Method
Contours are shown below
2 2
1 2 1 2( , ) 5f x x x x= +
![Page 5: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/5.jpg)
The gradient at the point is
If we choose x11 = 3.22, x2
1 = 1.39 as the starting point
represented by the black dot on the figure, the black line shown
in the figure represents the direction for a line search.
Contours represent from red (f = 2) to blue (f = 20).
Steepest Descent
1 1 1 1
1 2 1 2( , ) (2 10, )
Tf x x x x∇ =
1 1
1 2( , )x x
1 1
1 2( , ) ( 6.44, 13.9)
Tf x x−∇ = − −
![Page 6: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/6.jpg)
Now, the question is how big should the step be along the
direction of the gradient? We want to find the minimum
along the line before taking the next step.
The minimum along the line corresponds to the point where the new direction is orthogonal to the original
direction.
The new point is (x12, x2
2) = (2.47,-0.23) shown in blue .
Steepest Descent
![Page 7: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/7.jpg)
By the third iteration we can see that from the point (x12, x2
2) the
new vector again misses the minimum, and here it seems that
we could do better because we are close.
Steepest descent is usually used as the first technique in aminimization procedure, however, a robust strategy that
improves the choice of the new direction will greatly enhance
the efficiency of the search for the minimum.
Steepest Descent
![Page 8: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/8.jpg)
7-
8
The GRG Algorithm (Used in solver - Excel)
� GRG can also be used on LPs but is slower than the Simplex method.
� Solver uses the Generalized Reduced Gradient (GRG) algorithm to solve NLPs.
� The following discussion gives a general (but somewhat imprecise) idea of how GRG works.
![Page 9: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/9.jpg)
7-
9
An NLP Solution Strategy
Feasible Region
A (the starting point)
B
C D
E
objective function level
curves
X1
X2
![Page 10: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/10.jpg)
7-
10
Local vs. Global Optimal Solutions
A
C
B
Local optimal solution
Feasible Region
D
EF
G
Local and global
optimal solution
X1
X2
![Page 11: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/11.jpg)
7-
11
Comments About NLP Algorithms
� It is not always best to move in the
direction producing the fastest rate of
improvement in the objective.
� NLP algorithms can terminate at local
optimal solutions.
� The starting point influences the local
optimal solution obtained.
![Page 12: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/12.jpg)
7-
12
Comments About Starting Points
� The null starting point should be
avoided.
� When possible, it is best to use
starting values of approximately
the same magnitude as the
expected optimal values.
![Page 13: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/13.jpg)
7-
13
A Note About “Optimal” Solutions
� When solving a NLP problem, Solver normally stops when the first of three numerical tests is satisfied, causing one of the following three completion messages to appear:
1) “Solver found a solution. All constraints and
optimality conditions are satisfied.”
This means Solver found a local optimal solution, but does not guarantee that the solution is the global optimal solution.
![Page 14: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/14.jpg)
7-
14
A Note About “Optimal” Solutions
� When solving a NLP problem, Solver normally stops when the first of three numerical tests is satisfied, causing one of the following three completion messages to appear:
2) “Solver has converged to the current solution. All constraints are satisfied.”
This means the objective function value changed very slowly for the last few iterations.
![Page 15: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/15.jpg)
7-
15
A Note About “Optimal” Solutions
� When solving a NLP problem, Solver normally stops
when the first of three numerical tests is satisfied,
causing one of the following three completion
messages to appear:
3) “Solver cannot improve the current solution.
All constraints are satisfied.”
This rare message means the your model is degenerate and the Solver is cycling. Degeneracy can often be eliminated by removing redundant constraints in a model.
![Page 16: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/16.jpg)
7-
16
Location Problems
� Many decision problems involve determining optimal locations for facilities or service centers. For example,
– Manufacturing plants
– Warehouse
– Fire stations
– Ambulance centers
� These problems usually involve distance measures in the objective and/or constraints.
• The straight line (Euclidean) distance between two points (X1, Y1) and (X2, Y2) is:
( ) ( )2212
21 YYXXDistance −+−=
![Page 17: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/17.jpg)
7-
17
A Location Problem:Rappaport Communications
� Rappaport Communications provides
cellular phone service in several mid-
western states.
� They want to expand to provide inter-city
service between four cities in northern Ohio.
� A new communications tower must be built
to handle these inter-city calls.
� The tower will have a 40 mile transmission
radius.
![Page 18: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/18.jpg)
7-
18
Graph of the Tower Location Problem
Cleveland
AkronYoungstown
Canton
x=5, y=45
x=12, y=21
x=17, y=5
x=52, y=21
0 20 30 40 50 60
10
20
30
40
50
X
Y
0
10
![Page 19: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/19.jpg)
7-
19
Defining the Decision Variables
X1 = location of the new tower with respect to the X-axis
Y1 = location of the new tower with respect to the Y-axis
![Page 20: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/20.jpg)
7-
20
Defining the Objective Function
� Minimize the total distance from the new tower to the existing towers
( ) ( ) ( ) ( )21
22
1
2 Y21Y45 −++−+11
X-12X-5
( ) ( ) ( ) ( )21
22
1
2 Y21Y5 −++−++11
X-52X-17
MIN:
![Page 21: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/21.jpg)
7-
21Problem solved in class
� Cleveland
� Akron
� Canton
� Youngstown
( ) ( ) 4045 2
1
2 ≤−+ Y 1
X-5
( ) ( )12- X1
2
1
2
21 40+ − ≤Y
( ) ( )17 - X1
2
1
2
5 40+ − ≤Y
( ) ( )52- X1
2
1
2
21 40+ − ≤Y
Defining the Constraints
![Page 22: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/22.jpg)
7-
22
Analyzing the Solution
� The optimal location of the “new tower” is in virtually the same location as the existing Akron tower.
� Maybe they should just upgrade the Akron tower.
� The maximum distance is 39.8 miles to Youngstown.
� This is pressing the 40 mile transmission radius.
![Page 23: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/23.jpg)
7-
23
Comments on Location Problems
� The optimal solution to a location problem may not work:
– The land may not be for sale.
– The land may not be zoned properly.
– The “land” may be a lake.
� In such cases, the optimal solution is a good starting point in the search for suitable property.
� Constraints may be added to location problems to eliminate infeasible areas from consideration.
![Page 24: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/24.jpg)
7-
24
Another example (Taken from Daniel P. Loucks & Eelco van Beek)
Consider the system shown below where a reservoir is upstream of three demand sites along a river.
Demand sites
Reservoir
![Page 25: Introduction to Nonlinear Programming (NLP)aleon3/courses/Optimization/Lectures/Non_linear_programming.pdfIntroduction to Nonlinear Programming (NLP) This lecture was adapted from](https://reader030.vdocument.in/reader030/viewer/2022040112/5e6d71dbcef10e00dd755058/html5/thumbnails/25.jpg)
7-
25
Example (Cont. )
The net benefits derived from each use depend on the
reliable amounts of water allocated to each use. Letting
xit be the allocation to use i in period t, the net benefits for each period t equal
� 1. 6x1t– x1t2
� 2. 7x2t – 1.5 x2t2
� 3. 8x3t – 0.5 x3t2
Assume the average inflows to the reservoir in each of
four seasons of the year equal 10, 2, 8, 12 units per
season and that the reservoir capacity is 5 volume units.
Find the optimal operating policy for this reservoir
that maximizes the total (four season) allocation
benefits for the users. Problem solved in class