chapter 16. calculus methods of optimization

Min Soo KIM

Department of Mechanical and Aerospace EngineeringSeoul National University

Chapter 16. Calculus Methods of Optimization

Optimal Design of Energy Systems (M2794.003400)


16.1 Continued exploration of calculus methods

- A major portion of this chapter deals with principles of calculus method

- Substantiation for the Largrange multiplier equations will be provided

- Physical interpretation of λ, and test for maxima-minima are also provided

/ 152


16.1 Continued exploration of calculus methods

- Lagrange multipliers (from Chap.8)

1 2( , , , )ny y x x x

1 1 2

1 2

( , , , ) 0

( , , , ) 0

n

m n

x x x

x x x

1

0m

i i

i

y

1 1 2

1 2

( , , , ) 0

( , , , ) 0

n

m n

x x x

x x x

n+m scalarequations

/ 153


16.2 The nature of the gradient vector ( )

1. normal to the surface of constant y

1 1 2 2 3 3ˆ ˆ ˆdx i dx i dx i

1 2 3

1 2 3

0y y y

dy dx dx dxx x x

2 2 3 31

1

( / ) ( / )

/

y x dx y x dxdx

y x

2 2 3 31 2 2 3 3

1

( / ) ( / ) ˆ ˆ ˆ/

y x dx y x dxT i dx i dx i

y x

0T y

1 2 3

1 2 3

ˆ ˆ ˆy y yy i i i

x x x

For arbitrary vector :

To be tangent to the surface :

Tangent vector :

Gradient vector :

→ gradient vector is normal to all tangent vectors

→ gradient vector normal to the surface

y

/ 154



2. indicate direction of maximum rate of change of y with respect to x

1 2

1 2

n

n

y y ydy dx dx dx

x x x

2 2 2 2

1 2( ) ( ) ( )ndx dx dx r constraints :

to find maximum dy :

y

The constraint indicates a circle of radious for 2 dimensions, and a sphere for 3 dimensions

/ 155



12 0

2i i

i i

y ydx dx

x x

using Lagrange multipliers :

1 1 2 2 1 2

1 2

1 1ˆ ˆ ˆ ˆ ˆ ˆ2 2

n n n

n

y y ydx i dx i dx i i i i y

x x x

y

In vector form :

→ indicates the direction of maximum change for a given distance in the spacey

/ 156



3. points in the direction of increasing y

y

small move in the x1-x2 space : 1 2

1 2

y ydy dx dx

x x

2 2

1 2

0y y

dy cx x

If the move is made in the direction of :y

1 1 2 2( / ), ( / )/

i

i

dxc dx c y x dx c y x

y x

→ dy is equal or greater than zero

→ y always increase in the direction of y

/ 157

Optimize subject to


16.5 Two variables and one constraint (prove Lagrange multiplier method)

1 2

1 2

y yy x x

x x

21 2 1 2

1 2 1

/0

/

xd x x x x

x x x

22

1 1 2

/

/

xy yy x

x x x

1 2( , )y x x 1 2( , ) 0x x

Taylor expansion :

substituting the result for ∆𝑥1

/ 158

In order for no improvements of Δy,


16.5 Two variables and one constraint (prove Lagrange multiplier method)

2 2

0y

x x

1 1

0y

x x

∆𝑦 = −𝜕𝑦

𝜕𝑥1

Τ𝜕𝜙 𝜕𝑥2Τ𝜕𝜙 𝜕𝑥1

+𝜕𝑦

𝜕𝑥2∆𝑥2 = 0 −

𝜕𝑦

𝜕𝑥1


+𝜕𝑦

𝜕𝑥2= 0

If λ is defined as 𝜕𝑦𝜕𝑥1𝜕𝜙𝜕𝑥1

/ 159


16.6 Three variables and one constraint

As a same manner with 2 variables, 𝑦 𝑥1, 𝑥2, 𝑥3 is need to be optimized subject to the constraint, 𝜙 𝑥1, 𝑥2, 𝑥3

The first degree terms in the Taylor seriers are

𝜕𝑦

𝜕𝑥1∆𝑥1 +

𝜕𝑦

𝜕𝑥2∆𝑥2 + (

𝜕𝑦

𝜕𝑥3)∆𝑥3

𝜕𝜙

𝜕𝑥1∆𝑥1 +

𝜕𝜙

𝜕𝑥2∆𝑥2 + (

𝜕𝜙

𝜕𝑥3)∆𝑥3

/ 1510



Any two of the three variables can be moved independently, but the motion of the third variables must abide by the constraint. Arbitraily choosing 𝑥2 as a dependant one,

∆𝑦 =𝜕𝑦

𝜕𝑥1∆𝑥1 −

𝜕𝑦

𝜕𝑥2


∆𝑥1 +Τ𝜕𝜙 𝜕𝑥3Τ𝜕𝜙 𝜕𝑥2

∆𝑥3 +𝜕𝑦

𝜕𝑥3∆𝑥3 = 0

In order for no improvement to be possible

𝜕𝑦

𝜕𝑥1−

𝜕𝑦

𝜕𝑥2

𝜕𝜙𝜕𝑥1𝜕𝜙𝜕𝑥2

= 0𝜕𝑦

𝜕𝑥3−

𝜕𝑦

𝜕𝑥2

𝜕𝜙𝜕𝑥3𝜕𝜙𝜕𝑥2

= 0

/ 1511



Define 𝜆 as𝜕𝑦𝜕𝑥1𝜕𝜙𝜕𝑥1

𝜕𝑦

𝜕𝑥1− 𝜆

𝜕𝜙

𝜕𝑥1= 0

Then𝜕𝑦

𝜕𝑥2− 𝜆

𝜕𝜙

𝜕𝑥2= 0

𝜕𝑦

𝜕𝑥3− 𝜆

𝜕𝜙

𝜕𝑥3= 0

/ 1512



Define 𝜆 as𝜕𝑦𝜕𝑥1𝜕𝜙𝜕𝑥1

𝜕𝑦

𝜕𝑥1− 𝜆

𝜕𝜙

𝜕𝑥1= 0

Then𝜕𝑦

𝜕𝑥2− 𝜆

𝜕𝜙

𝜕𝑥2= 0

𝜕𝑦

𝜕𝑥3− 𝜆

𝜕𝜙

𝜕𝑥3= 0

/ 1513


16.8 Alternate expression of constrained optimization problem

optimize 1 2( , )y x x

1 2( , )x x b subject to

1 2 1 2 1 2( , ) ( , ) [ ( , ) ]L x x y x x x x b unconstrained function

optimum occurs where 0L

1 1 1

2 2 2

1 2

0

0

( , ) 0

L y

x x x

L y

x x x

x x b

find optimum point

/ 1514


16.9 Interpretation λ of as the sensitivity coefficient

* * *

1 21 2

1 2

( , ) (1)x xy y y

y x x SCb x b x b

1 21 2

1 2

( , ) 1 0 (2)x x

x x bb x b x b

* * * *

1 2

* *

1 2

( / ) ( / )

( / ) ( / )

y x y x

x x

1 2

1 2

(2) 0x xy y

x b x b

sensitivity coefficient (SC) :

(1) SC

/ 1515

chapter 16. calculus methods of optimization

Documents