L7 Optimal Design concepts pt C• Homework• Review• Positive definite tests• SVO example• MVO example• Summary• Test 1

1

Single variable optimization

First-order necessary condition

0*)( xf

0*)( xf

Second-order sufficientcondition for a minimum

Second-order sufficientcondition for a maximum

0*)( xf

“stationary point(s)”

SVO example

3

44)( 23 xxxxf

423)( 2 xxxf

MaxfMinf

xxf

)0211.7)8685/0(0211.7)535.1(

26)(

pt B8685.0pt A535.1

2

40423..0)(

2

1

2

2,1

2

xx

a

acbbx

xxeixf

Necessary condition

Sufficient condition

What happens when f ″(x)=0 ?i.e. x=2/6=1/3

MV Optimization

4

0x *)( Tf

For x* to be a local minimum:

2

1*)( dHddx TTff

0 *)()( xx fff

1rst orderNecessaryCondition

0 dHdT

2nd orderSufficientCondition

i.e. H(x*) must be positive definite

Positive definiteness Tests?

• By inspection• Leading principal minors• Eigenvalues

5

23

22

212

1)( xxxF xe.g. by inspection

symmetricis where2

1)(Given AxAxx TF

Find leading principal minors to check PD of A(x)

6

nnnnn

n

n

n

aaaa

aaaaaaaa

aaaa

321

3333231

2232221

1131211

A

333231

232221

131211

3

2221

12112

111

aaa

aaa

aaa

M

aa

aaM

aM

Principal Minors Test for PD

7

A matrix is positive definite if:

1.No two consecutive minors can be zero AND

2. All minors are positive, i.e. 0kM

If two consecutive minors are zeroThe test cannot be used.

Principal Minors Test for ND

8

A matrix is negative definite if:

1.No two consecutive minors can be zero AND

2. Mk<0 for k=odd

3. Mk>0 for k=even

If two consecutive minors are zeroThe test cannot be used.

Form Eigenvalue Test

Positive Definite (PD)

Positive Semi-def (PSD)

Indefinite

Eigenvalue test

9

0 IA

0i

0i

0i

0i

0x

xxAx

other than

allfor 0 ,T

0xxAx

xxAx

oneleast at for 0

and allfor 0T

T ,

xxAx

xxAx

other for 0

somefor 0

T

T

0i

0i

ND

NSD

Eigenvalue example

10

100

011

011

A 0

00

00

00

100

011

011

IA

0

100

011

011

IA

0]1)1)[(1( 2 IA

0)2(02

01)21(01)1(

2

2

2

2,0,1 321

Therefore A is NSD

Expanded on row3, col3

MVO example

11

42

22 *)(

*

22

2

21

2

22

2

22

2

21

2

21

2

2

x

xH

x

f

x

f

x

fx

f

x

f

x

f

f

0

0)142()222()(

21

21

*x

x xxxxf

5.15.2* x

8222)( 212221

21 xxxxxxf x

Necessary condition

Sufficient condition

H(x) is Pos Defx* is local min!

Effects of scaling f(x)or adding a constant

12

Figure 4.9 Graphs for Example 4.19. Effects of scaling or of adding a constant to a function. (a) A graph of f(x)=x2-2x+2. (b) The effect of addition of a constant to f(x). (c) The effect of multiplying f(x) by a positive constant. (d) Effect of multiplying f(x) by -1.

Summary

• Local min/max may exist• Necessary & Sufficient Conditions• “Positivity” – inspection, Mk, λi

13