(iii) lagrange multipliers and kuhn-tucker conditions d nagesh kumar, iisc introduction to...
Post on 19-Jan-2016
237 Views
Preview:
TRANSCRIPT
(iii) Lagrange Multipliers and Kuhn-tucker Conditions
D Nagesh Kumar, IISc
Introduction to Optimization
Water Resources Systems Planning and Management: M2L3
Objectives
D Nagesh Kumar, IISc2
To study the optimization with multiple decision variables
and equality constraint : Lagrange Multipliers.
To study the optimization with multiple decision variables
and inequality constraint : Kuhn-Tucker (KT) conditions
Water Resources Systems Planning and Management: M2L3
Constrained optimization with equality constraints
D Nagesh Kumar, IISc3
A function of multiple variables, f(x), is to be optimized subject to one or more
equality constraints of many variables. These equality constraints, gj(x), may or may
not be linear. The problem statement is as follows:
Maximize (or minimize) f(X), subject to gj(X) = 0, j = 1, 2, … , m
where
(1)
1
2
n
x
x
x
X
Water Resources Systems Planning and Management: M2L3
Constrained optimization…
D Nagesh Kumar, IISc4
With the condition that ; or else if m > n then the problem
becomes an over defined one and there will be no solution. Of the
many available methods, the method of constrained variation and the
method of using Lagrange multipliers are discussed.
m n
Water Resources Systems Planning and Management: M2L3
Solution by method of Lagrange multipliers
D Nagesh Kumar, IISc5
Continuing with the same specific case of the optimization problem with
n = 2 and m = 1 we define a quantity λ, called the Lagrange multiplier as
(2)
Using this in the constrained variation of f [ given in the previous lecture]
And (2) written as (3)
(4)
* *1 2
2
2 (x , x )
/
/
f x
g x
* *1 2
1 1 (x , x )
0f g
x x
* *1 2
2 2 (x , x )
0f g
x x
1 2
11
1 2 2 (x *, x *)
/0
/
g xf fdf dx
x g x x
Water Resources Systems Planning and Management: M2L3
Solution by method of Lagrange multipliers…
D Nagesh Kumar, IISc6
Also, the constraint equation has to be satisfied at the extreme point
(5)
Hence equations (2) to (5) represent the necessary conditions for the point
[x1*, x2*] to be an extreme point.
λ could be expressed in terms of as well has to be non-
zero.
These necessary conditions require that at least one of the partial
derivatives of g(x1 , x2) be non-zero at an extreme point.
* *1 2
1 2 ( , )( , ) 0
x xg x x
1/g x 1/g x
Water Resources Systems Planning and Management: M2L3
D Nagesh Kumar, IISc7
The conditions given by equations (2) to (5) can also be generated by
constructing a functions L, known as the Lagrangian function, as
(6)
Alternatively, treating L as a function of x1,x2 and , the necessary
conditions for its extremum are given by
(7)
1 2 1 2 1 2( , , ) ( , ) ( , )L x x f x x g x x
1 2 1 2 1 21 1 1
1 2 1 2 1 22 2 2
1 2 1 2
( , , ) ( , ) ( , ) 0
( , , ) ( , ) ( , ) 0
( , , ) ( , ) 0
L f gx x x x x x
x x x
L f gx x x x x x
x x x
Lx x g x x
Solution by method of Lagrange multipliers…
Water Resources Systems Planning and Management: M2L3
Necessary conditions for a general problem
D Nagesh Kumar, IISc8
For a general problem with n variables and m equality constraints the
problem is defined as shown earlier
Maximize (or minimize) f(X), subject to gj(X) = 0, j = 1, 2, … , m
where
In this case the Lagrange function, L, will have one Lagrange multiplier j
for each constraint as
(8)1 2 , 1 2 1 1 2 2( , ,..., , ,..., ) ( ) ( ) ( ) ... ( )n m m mL x x x f g g g X X X X
1
2
n
x
x
x
X
Water Resources Systems Planning and Management: M2L3
D Nagesh Kumar, IISc9
L is now a function of n + m unknowns, , and the
necessary conditions for the problem defined above are given by
(9)
which represent n + m equations in terms of the n + m unknowns, xi and j.
The solution to this set of equations gives us
and (10)
1 2 , 1 2, ,..., , ,...,n mx x x
1
( ) ( ) 0, 1, 2,..., ; 1, 2,...,
( ) 0, 1, 2,...,
mj
jji i i
jj
gL fi n j m
x x x
Lg j m
X X
X
*1
*2
*n
x
x
x
X
*1
** 2
*m
Necessary conditions for a general problem…
Water Resources Systems Planning and Management: M2L3
Sufficient conditions for a general problem
D Nagesh Kumar, IISc10
A sufficient condition for f(X) to have a relative minimum at X* is that each
root of the polynomial in Є, defined by the following determinant equation
be positive.
(11)
11 12 1 11 21 1
21 22 2 12 22 2
1 2 1 2
11 12 1
21 22 2
1 2
0
0 0
0 0
n m
n m
n n nn n n mn
n
n
m m mn
L L L g g g
L L L g g g
L L L g g g
g g g
g g g
g g g
Water Resources Systems Planning and Management: M2L3
D Nagesh Kumar, IISc11
where
(12)
Similarly, a sufficient condition for f(X) to have a relative maximum at X*
is that each root of the polynomial in Є, defined by equation (11) be
negative.
If equation (11), on solving yields roots, some of which are positive and
others negative, then the point X* is neither a maximum nor a minimum.
2* *
*
( , ), for 1,2,..., 1,2,...,
( ), where 1,2,..., and 1,2,...,
iji j
ppq
q
LL i n and j m
x x
gg p m q n
x
X
X
Sufficient conditions for a general problem…
Water Resources Systems Planning and Management: M2L3
Example
D Nagesh Kumar, IISc12
Minimize ,
Subject to
Solution
with n = 2 and m = 1
L =
or
2 21 1 2 2 1 2( ) 3 6 5 7 5f x x x x x x X
1 2 5x x
1 1 2( ) 5 0g x x X
1 2 , 1 2 1 1 2 2( , ,..., , ,..., ) ( ) ( ) ( ) ... ( )n m m mx x x f g g g L X X X X
2 21 1 2 2 1 2 1 1 23 6 5 7 5 ( 5)x x x x x x x x
1 2 11
1 2 1
1
6 6 7 0
1(7 )
61
5 (7 )6
x xx
x x
L
1 23
Water Resources Systems Planning and Management: M2L3
Example…
D Nagesh Kumar, IISc13
1 2 12
1 2 1
1 2 2 1
6 10 5 0
13 5 (5 )
21
3( ) 2 (5 )2
x xx
x x
x x x
L
2
1
2x
1
11
2x
11 1* , ; * 23
2 2
X λ
11 12 11
21 22 21
11 12
0
0
L L g
L L g
g g
Hence,
Water Resources Systems Planning and Management: M2L3
Example…
D Nagesh Kumar, IISc14
2
11 21 ( )
6Lx
X*,λ*
L 2
12 211 2 ( )
6L Lx x
X*,λ*
L 2
22 22 ( )
10Lx
X*,λ*
L
111
1 ( )
112 21
2 ( )
1
1
gg
x
gg g
x
X*,λ*
X*,λ*
6 6 1
6 10 1 0
1 1 0
( 6 )[ 1] ( 6)[ 1] 1[ 6 10 ] 0
2
The determinant becomes
or
*λSince is negative, X*, correspond to a maximum
Water Resources Systems Planning and Management: M2L3
Kuhn – Tucker Conditions
D Nagesh Kumar, IISc15
KT condition: Both necessary and sufficient if the objective function
is concave and each constraint is linear or each constraint function is
concave, i.e., the problems belongs to a class called the convex
programming problems.
Water Resources Systems Planning and Management: M2L3
Kuhn-Tucker Conditions: Optimization Model
Consider the following optimization problem
Minimize f(X)
subject to
gj(X) ≤ 0 for j=1,2,…,p
where the decision variable vector
X=[x1,x2,…,xn]
D Nagesh Kumar, IIScWater Resources Systems Planning and Management: M2L316
Kuhn-Tucker Conditions
1
0 1, 2,...,
0 1, 2,...,
0 1, 2,...,
0 1, 2,...,
m
jji i
j j
j
j
f gi n
x x
g j m
g j m
j m
Kuhn-Tucker conditions for X* = [x1* , x2
* , . . . xn*] to be a local minimum are
D Nagesh Kumar, IIScWater Resources Systems Planning and Management: M2L317
Kuhn Tucker Conditions …
In case of minimization problems, if the constraints are of
the form gj(X) ≥ 0, then λj have to be non-positive
If the problem is one of maximization with the constraints
in the form gj(X) ≥ 0, then λj have to be nonnegative.
D Nagesh Kumar, IIScWater Resources Systems Planning and Management: M2L318
Example 1
D Nagesh Kumar, IISc19
Minimize
subject to
2 2 21 2 32 3f x x x
1 1 2 3
2 1 2 3
2 12
2 3 8
g x x x
g x x x
Water Resources Systems Planning and Management: M2L3
D Nagesh Kumar, IISc20
1 21 2 0
i i i
g gf
x x x
0 j jg
0 jg
0j
Kuhn – Tucker Conditions
Example 1…
1 1 2
2 1 2
3 1 2
2 0 (14)
4 2 0 (15)
6 2 3 0
x
x
x
(16)
1 1 2 3
2 1 2 3
( 2 12) 0 (17)
( 2 3 8) 0 (18)
x x x
x x x
1 2 3
1 2 3
2 12 0 (19)
2 3 8 0 (20)
x x x
x x x
1
2
0 (21)
0 (22)
Water Resources Systems Planning and Management: M2L3
From (17) either = 0 or ,
Case 1: = 0
From (14), (15) and (16) we have x1 = x2 = and x3 =
Using these in (18) we get
From (22), , therefore, =0,
Therefore, X* = [ 0, 0, 0 ]
This solution set satisfies all of (18) to (21)
1 1 2 32 12 0x x x
2 / 2 2 / 222 2 28 0, 0 8or
2 0 2
Example 1…
1
D Nagesh Kumar, IISc21 Water Resources Systems Planning and Management: M2L3
Case 2:
Using (14), (15) and (16), we have
or
But conditions (21) and (22) give us and
simultaneously, which cannot be
possible with
Hence the solution set for this optimization problem is
X* = [ 0 0 0 ]
1 2 32 12 0x x x
1 2 1 2 1 22 2 312 0
2 4 3
1 217 12 144
1 0 2 0
1 217 12 144
Example 1…
D Nagesh Kumar, IISc22 Water Resources Systems Planning and Management: M2L3
Minimize
subject to
2 21 2 160f x x x
1 1
2 1 2
80 0
120 0
g x
g x x
Example 2
D Nagesh Kumar, IISc23 Water Resources Systems Planning and Management: M2L3
1 21 2 0
i i i
g gf
x x x
0 j jg
0 jg
0j
Kuhn – Tucker Conditions
Example 2…
1 1 2
2 2
2 60 0 (23)
2 0 (24)
x
x
1 1
2 1 2
( 80) 0 (25)
( 120) 0 (26)
x
x x
1
1 2
80 0 (27)
120 0 (28)
x
x x
1
2
0 (29)
0 (30)
D Nagesh Kumar, IISc24 Water Resources Systems Planning and Management: M2L3
From (25) either = 0 or ,
Case 1
From (23) and (24) we have and
Using these in (26) we get
Considering , X* = [ 30, 0]. But this solution set violates (27)
and (28)
For , X* = [ 45, 75]. But this solution set violates (27)
1 1( 80) 0x
21 302x 2
2 2x
2 2 150 0
2 0 150or
2 0
2 150
Example 2…
D Nagesh Kumar, IISc25 Water Resources Systems Planning and Management: M2L3
Case 2:
Using in (23) and (24), we have
Substitute (31) in (26), we have
For this to be true, either
1( 80) 0x
1 80x
2 22 40 0x x
2 20 40 0x or x
Example 2…
2 2
1 2
2
2 220 (31)
x
x
D Nagesh Kumar, IISc26 Water Resources Systems Planning and Management: M2L3
For ,
This solution set violates (27) and (28)
For ,
This solution set is satisfying all equations from (27) to (31) and hence
the desired
Thus, the solution set for this optimization problem is
X* = [ 80, 40 ]
2 0x 1 220
2 40 0x 1 2140 80and
Example 2…
D Nagesh Kumar, IISc27 Water Resources Systems Planning and Management: M2L3
BIBLIOGRAPHY / FURTHER READING
1. Rao S.S., Engineering Optimization – Theory and Practice, Fourth
Edition, John Wiley and Sons, 2009.
2. Ravindran A., D.T. Phillips and J.J. Solberg, Operations Research –
Principles and Practice, John Wiley & Sons, New York, 2001.
3. Taha H.A., Operations Research – An Introduction, 8th edition, Pearson
Education India, 2008.
4. Vedula S., and P.P. Mujumdar, Water Resources Systems: Modelling
Techniques and Analysis, Tata McGraw Hill, New Delhi, 2005.
D Nagesh Kumar, IISc28 Water Resources Systems Planning and Management: M2L3
Thank You
D Nagesh Kumar, IIScWater Resources Systems Planning and Management: M2L3
top related