continuous optimization techinique

8/18/2019 continuous optimization techinique

1/32

EPE-821

1

Dr. Muhammad Naeem

Dr. Ashfaq Ahmed

Continuous Optimization Techniques


2/32

2

Outline

Review of Math

Basics of Optimization

Power systems basics

Review of Matab

!"ampes of Optimization in Power systems

#raphica Optimization

Optimization $ypes %onstraint and &nconstraint

Optimization Probem $ypes 'inear Non'inear etc

'inear Optimization and Power (ystems Appications

Non 'inear Optimization and Power (ystems Appications

)nte*er and Mi"ed inte*er pro*rammin* and Power (ystems Appications

%ompe"ity Anaysis

+uiz ne"t wee,


3/32

Text%acuus and Anaytic #eometry By $homas and -inney th !dition

)ntroduction of Optimum Desi*n By /asbir (. Arora

(ome fi*ure from 0eb

Applied Numerical Methods ith MAT!A"# $or En%ineers and

&cientists 'rd Edition( &te)en C* Chapra

http122www.ece.mcmaster.ca23"wu2part4.pdf

Noninear Optimization with -inancia Appications by Michae Barthoomew5

Bi**s

'


4/32

+

Optimialit, Condition(uppose that - 6"7 is a continuousy differentiabe function of the scaar

variabe "8 and that8 when " 9 ":8

2

20 0 (1)

dF d F and

dx dx= ≥

Above two conditions are ,nown as optimaity conditions

%onditions 6;7 impy that -6":7 is the smaest vaue of - in some re*ion

near ": . )t may aso be true that - 6": 7 < - 6"7 for a " but condition 6;7

does not *uarantee this.

Definition )f conditions 6;7 hod at " 9 ": and if -6":7 = -6"7 for a "

then ": is said to be the *oba minimum.


5/32

Necessar, and &ucient Conditions

.

Example1 )f a number is divisibe by 46ca this >78 then it is divisibe by ?

6ca this %7. > impies % but % is not stron* enou*h to impy > for e"ampe

@ is divisibe by ? but not by 4. $herefore % is necessary for > but notsufficient for >.

Example: )n !ucidean *eometry8 a trian*e has equa sides 6ca this >7 if

and ony if the trian*e has equa an*es 6ca this %7. $his means

> if %1 > is necessary for % since % impies >

> ony if %1 % is necessary for > since > impies %

> if and ony if %1 > and % impy each other and are both necessary and

sufficient conditions for each other.


6/32

Necessar, and &ucient Conditions

/

Example1 )f a number is divisibe by 46ca this >78 then it is divisibe by ?

6ca this %7. > impies % but % is not stron* enou*h to impy > for e"ampe

@ is divisibe by ? but not by 4. $herefore % is necessary for > but notsufficient for >.

Example: )n !ucidean *eometry8 a trian*e has equa sides 6ca this >7 if

and ony if the trian*e has equa an*es 6ca this %7. $his means

> if %1 > is necessary for % since % impies >

> ony if %1 % is necessary for > since > impies %

> if and ony if %1 > and % impy each other and are both necessary and

sufficient conditions for each other.

)n the conte"t of smooth functions 6Differentiabe functions7 the condition f 6"7 is a necessary condition for a reative ma"imum or minimum. 6$his is

not sufficient because zero sope may aso at infection point.

)n the conte"t of smooth functions 6Differentiabe functions7 the condition

f6"7 is a sufficient condition if f6"7 9 C.


7/320

irect search and %radient methods

Both are iterative methods

direct search techniques are based on simpe comparison of function

vaues at tria points.

*radient methods. $hese use derivatives of the obective function and can

be viewed as iterative a*orithms for sovin* the noninear equation

#radient methods tend to conver*e faster than direct search methods.

$hey aso have the advanta*e that they permit an obvious conver*ence

test 5 namey stoppin* the iterations when the *radient is near zero.

0

dF

dx =


8/328

Example

(oution1

-ind the minimum and ma"imum of for3 23 x x− [ ]1, 4 x ∈ −


9/32

Example-ind the minimum and ma"imum of for

3 23 x x−

(oution1

3 2

2

3

3 6 0 0 2

6 6

(0) 0

(2) 0

F x x

F x x at x and x

F x

F Local Maximum

F Local Minimum

= −

′ = − = ⇒ = =

′′ = −

′′ ≤

′′ ≥

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15

-10

-5

0

5

10

15

20

25x3-3*x

2

Orig

1st Derv.

2nd Derv.

[ ]1, 4 x ∈ −


10/3213

"isection Method to 4ind the 5oot

Ony suitabe for functions that has ony one minimum2ma"imum in the ran*e Ea8 bF

#iven a brac,eted root8 the method repeatedy haves the interva whie

continuin* to brac,et the root and it wi conver*e on the soution.


11/3211

"isection Method to 4ind the 5oot1.Choose x

l and x

u as two guesses for the root such

that f ( xl ) f( x

u) < 0, or in other words, f(x) changes

sign between xl and xu.2.Estimate the root , x

m of the equation f (x) 0 as the

mid!"oint between xl and x

u as

3. #f f ( xl ) f ( x

m ) $ 0, then the root %ies between x

l and

xm& then x

l = x

l ; x

u = x

m.

E%se #f f( xl ) f( x

m) ' 0, then the root %ies between x

m

and u& then x

l = x

m; x

u = x

u.

E%se #f f (%) f(

m) 0& then the root is x

m.to" the

a%gorithm if this is true.

4. *et new estimate

+. bso%ute -e%atie ""roimate Error

6. Chec/ if error is %ess than "re!s"ecified to%eranceor if maimum number of iterations is reached

2

l um

x x x

+=

f()

u

m

2l u

m x x x +=100×

−=∈

new

m

old

m

new

a x

x xm

rootof estimatecurrent=new

m x

rootof estimate "reious=old m x


12/3212

"isection Method to 4ind the 5oot


13/32

1'

"isection Method( ) 423 10(3316+0 -.+ x.- x x f =

Choose the 6rac7et

( )

( ) 4

4

10662.211.0

103.30.011.0

00.0

−

−

−=

==

=

f

f x

x

u


14/32

1+

"isection Method

0++.02

11.00

11.0,0

=+

=

==

m

u

x

x x

( )( )

( ) +

4

4

106++.60++.0

10662.211.0

103.30

−

−

−

=

−==

f

f

f

11.0

0++.0

=

=

u x

x

teration 91


15/32

1.

"isection Methodteration 92

33.33

02+.02

11.00++.0

11.0,0++.0

=∈

=+

=

==

a

m

u

x

x x

( )

( )

( )02+.0,0++.0

10(62216.102+.0

10(662.211.0

10(6++.60++.0

4

4

+

==−=

−=

=

−

−

−

u x x

f

f

f


16/32

1/

"isection Methodteration 9'

06+.02

02+.00++.0

02+.0,0++.0

=+

=

==

m

u

x

x x

( )

( )

( ) +

4

+

10+632.+06+.0

1062216.102+.0

106++.60++.0

20

−

−

−

−=

−=

=

=∈

f

f

f

a


17/32

10

"isection MethodCon)er%ence

$abe ;1 Root of f6"79C as function of number of iterations for bisection method.


18/32

18

"isection Method Advantages:

Aways conver*ent

$he root brac,et *ets haved with each iteration 5 *uaranteed. Disadvantages:

(ow conver*ence

)f one of the initia *uesses is cose to the root8 the conver*ence is sower

)f a function f6"7 is such that it ust touches the "5a"is it wi beunabe to find the ower and upper *uesses.

( ) 2 x x f =

5;C 5G C G ;C

C

?C

4C

@C

HC

;CC

y ( x )

x

y = x?


19/32

1

"isection Method

4unction chan%es si%n 6ut root does not exist

( ) x

x f 1=

5C. 5C.@ 5C.I C.C C.I C.@ C.

5;CC

5HC

5@C

54C

5?C

C

?C

4C

@C

HC

;CC

y ( x )

x

y = 1/x


20/32

23

"isection method $or optimization-ind the minimum of for . Aso write matab code for

verification.

3 23 x x− [ ]0,3 x ∈

%an we appy the bisection method to

find the minimum to this probemJ


21/32

21

"isection method $or optimization-ind the minimum of for . . Aso write matab code for

verification.

3 23 x x− [ ]1.+,3 x ∈

%an we appy the bisection method tofind the root to this probemJ

No. 0hyJ. >ow to use root findin* a*orithms for optimization

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15

-10

-5

0

5

10

15

20

25x3-3*x

2

Orig

1st Derv.

2nd Derv.


22/32

i:erence 6eteen roots and optima


23/32

2'

"isection method $or optimization-ind the minimum of for . Aso write matab code for

verification.

3 23 x x− [ ]1.+,3 x ∈

%an we appy the bisection method tofind the root to this probemJ

No. 0hyJ. >ow to use root findin* a*orithms for optimization

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15

-10

-5

0

5

10

15

20

25x3-3*x2

Orig

1st Derv.

2nd Derv.

$o *et the minimum we need to

appy bisection method on the

derivative of x

-x!

otherwisewe wi *et the wron* answer.

Root findin* a*orithms find the

point where the function is zero.

0e want to see where

derivative is zero.


24/32

2+

&ecant method to ;nd rootEquation o$ a !ine $rom 2 Points

,

2 1

2 1

5 ,

1 2 1 1 2 1

2 11 1

2 1

F"om t#e definition of $lo%ewe can dete"mine t#e $lo%e in t#i$ ca$e a$

y ym

x x

&lu''in' in t#i$ fo" m in $lo%e %oint fo"mula

y y y y x x x x

y y y y x x

x x

#i$ i$ called a$

−=

−−

− −=− −

− ⇒ − = − ÷− .two %oint fo"mula−


25/32

2.

&ecant method to ;nd rooteri)ation o$ the method

e then use this new a%ue of x as x! and re"eat the "rocess

using x1 and x! instead of x0 and x1 . e continue this

"rocess, so%ing for x , x , etc., unti% we reach a

sufficient%7 high %ee% of "recision (a sufficient%7 sma%%

difference between xn and xn-1 ).


26/32

2/

&ecant method to ;nd root

$he first two iterations of the secant method. $he red curve

shows the function f and the bue ines are the secants. -or this

particuar case8 the secant method wi not conver*e.


27/32

20

&ecant method to ;nd root

nitialize x1(

x2( ε


28/32

28

&ecant method $or Optimization

nitialize x1(

x2( ε


29/32

2

&ecant method $or Optimization-ind the minimum of for . Aso write matab code for

verification.

3 23 x x− [ ]1.+,3 x ∈

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4-15

-10

-5

0

5

10

15

20

25x3-3*x

2

Orig1st Derv.

2nd Derv.4 < x' - 'Dx2 F $unctiond4 < 'Dx2 - /Dx F eri)ati)e

2 is 1.00000 and d8 is !1.00000

2 is 1.2+1 and d8 is !0.41326+

2 is 2.00264 and d8 is 0.042

2 is 1.6+ and d8 is !0.00122 is 1. and d8 is !0.00000

2 is 2.000000 and d8 is 0.000000


30/32

'3

&ecant method $or Optimization-ind the minimum of for . Aso write matab code for

verification.

3 23 x x− [ ]1.+,3 x ∈

4 < x' - 'Dx2 F $unctiond4 < 'Dx2 - /Dx F eri)ati)e

2 is 1.00000 and d8 is !1.00000

2 is 1.2+1 and d8 is !0.41326+

2 is 2.00264 and d8 is 0.042

2 is 1.6+ and d8 is !0.00122 is 1. and d8 is !0.00000

2 is 2.000000 and d8 is 0.000000

0 0.5 1 1.5 2 2.5 3-20

-15

-10

-5

0

5

10

15

20

Orig

1st Derv.

Iter = 1

Iter = 2

Iter = 3

Iter = 4

Iter = 5


31/32

'1

Neton-5aphson or Neton method

-e"eat the "rocess unti% 9 f ( xi) 9 is

sufficient%7 sma%%

( ) ( )

1

0ii

i i

f x f x x x +

−′ = −

0hich can be rearran*e as

( )

( )1i

i ii

f x

x x f x+ = −

′

Root


32/32

'2

Neton-5aphson or Neton method

-e"eat the "rocess unti% 9 f ( xi) 9 is

sufficient%7 sma%%

( ) ( )

1

0ii

i i

f x f x x x +

−′ = −

0hich can be rearran*e as

( )

( )1i

i ii

f x

x x f x+ = −

′

Root

Optimization( )

( )1

i

i i

i

f x x x

f x+

′= −

′′

continuous optimization techinique

Documents