Lecture 17-18Primal methods

Solmaz KiaMechanical and Aerospace Eng. Dept.,

University of California Irvine

Consult: Section 2.3 of Ref[1] and Sections 12.1,12.2 and 12.4 of Ref[2]

Primal Methods for constraint 4timizationWe consider the problem . . ?>7ô

. Iminf(x), LI f’k c

g(x)<O ‘ ‘,— )

I ?,q4ajItQ 1h(x)=O t,‘> P/i

/Y

—:---U::T—; d140By a primal method we mean a search method that works bysearching through the feasible region.First Order Necessary Condition for Optimality: x is a local minimizer then

Vf(x*)TL\x ; 0, for I\x e V(x*)

. Set of first order feasible variations at x

V(x) ={d e R f Vhj,(x)Td = 0, Vg(x)Td 0, j e A(x*)}

. Active inequality constraints at x

A(x) = {j E {1, . . . , r} gj (x) = O}

Primal Methods for constraint optimization

We consider the problem

mm f(x)g(x)Oh(x)=O

By a primal method we mean a search method that works bysearching through the feasible region.

; I •; i

I S

\C’ “ ‘1

Advantages “

. If the process is terminated before reaching the solution, the. . . 4-current point is feasible. V — ‘j

. It can often be guaranteed that if the sequence of pointsconverges, then the limit is a local constrained minimum. 7

• Most of the primal methods do not rely on special structure,such as convexity.

Disadvantages• Needs a feasible starting point.• It can be computationally hard to remain in the feasible region.

Feasible Direction MethodsThe idea of feasible direction methods is the same as with

) L

where dk 5 a feasible direction at xk, and ük 0.

is chosen to minimize f with the restriction that the point xk+1,and the line segment joining xk and xk+1 be feasible.

\>

unconstrained problems:,7)

Xk+1 Xk+kdkS

OPT: x =argmin f(x)xR

x E X (X is the set of constraints)for X = RU (problem becomes unconstrained) /

D RTh is a feasibledirection at x e X for OPTif(x+ocd) Xfor

&

LXEE [O,]

c)

A Feasible Direction Method: Simplified Zoutendijk method4Consider the a problem with linear constraints:

Given a feasible point Xk, let A(Xk) be the set of indices of active constraints,aj1xk b for 1 e A(Xk).

• The other constraints assure that Xk + dk will be feasible for sufficientlysmall k > 0.

minimize f(x)

u[xb, 1=1,••• ,rS t. fr- 4

The direction vector dk is then chosen as the solution of

5r\ )Lc 52.l minimize Vf(xk)Td

ci’

v

s.t.

i e A(xk),ad0,

Idj1=

LL-e

2Th ‘

;

j4CIL)

j=1

I

bounded solution

.-1_ irJ_/(

.,‘\Ot,’ k

c, LII’

• The last equation (which can be converted to linear constraints) ensures a

• The objective function makes d as close to Vf(Xk) as possible.

Feasible Direction Methods

There are two major shortcomings of feasible direction methods inthis form.. For general problems there may not exist any feasibledirections (see figure). ?‘ (‘‘

. They are also vulnerable to ,,jamming”, or ,,zigzagging”, whichwe have also seen with unconstrained problems, but here itmight converge to a point that is not even a constrained localminimum (the algorithmic map is not closed).

Feasibleset

S

c)

IV

:c-

I

S

r V

E r

1

2—.

0

V)

N

Ct

r

—r

f0

ci

Tp

11 0

2UJ

bj4J

9 p

(N

IL I-1-11

I’

(

i-cD

•

-

ii‘s--‘

4

\\-

1

11 S \co

+I, ±

q

+çC

>

F

c1

4-

3

00

1:___

r—cD-

%‘/

&c:s

_%$_J

9%1)

EDp

0

ac0U’

c

b

c_.

‘I

Ii

c)

/

y-

U

U

03

U

: $

±__k

r

k-

%rt-

r.

;:

‘I

4C:

:Lc

0

;,0

:S.

9S

454

r>-‘

..—

9

0

—

S

-11•’-r’O’

p-.

Icc;

‘:,

‘f\

S

r—i

-i--L_J

LJ

0

/

LE

00I-

7II

1ic)c0

0

N

*

cc *

I(C

a—T

‘%—

____

21

Th

--

r

ito

U4:_k c,Lt&(’L

Va

ft

‘I

‘

‘cicLQt

(‘1

0f’

ULI

(—_‘1Th

r

c±l

6

c’c

ra—’

6).

.,

ry__

4s

‘,

c

,;

.

Lc’

!

3_

.

;:3__r

>-,

:h,

s,

r’:r

,,c__

r’i

q:

f‘

%;:

-2—

•

_t-

.‘

3:(].

t?

.c;’.

tV

,‘\

.‘

0’?

\%—.,

.r?

(Cb

::ri

6•.

r’0

‘)—

r4

Lrc’

\‘—‘

S

S

rCI

\r

“

_‘_

-#

‘

r::

a•‘

c__%,

-rs

F-

11•T

?,

LJ

:

F

,

0 1’

T_

zN

(t\

z

0

cc

4-

1 LJ

t

oL ft

\:<c2 I

(%)

: 1 ,‘

cr T

r t.

<0) 3—

-r;

Th t

1

p A U )

Ac

jc (c

rOr

.4-r

(IL

mUI

.÷‘N

5

Cri

r

0 ci

x

1-

V

. (t.

‘

.

(cjJ

‘2

R,

O4-

%-c

J

jj

%)

aI0

n

L‘3

c;4DJ

IcL‘

(Dr

1:I

cJ1

0

Hø—Th:

4

fl4

04

.4

4.c4I’gLi

0

(Th-Jii

-‘

-Th

L

0Th

%

.—0c‘—h

%%J0

oJ

%%4-4

I;:

J‘

.%.-

a’

‘c-%

_)

ksi—

t)o

D,

‘S\_AT—

rL?(

4.-1’a

(Th

C,--

ct

o0

II>

-

‘1

V_

)-

II

“I

V)ì‘\J

_•%

A

U

TD

-Th

i

sJ

1—

L

=‘45

I

r

4

0

C

jp

o

Xk1ke ri hci

L

k().

(

‘:c1 r(L)

-2-

i) 4

c)

Ii 9r

-I’ o5 >

cc q

4

Os

‘

E

9 4

\1 4

\c

‘\I2\s \

c :‘s$\-°

2 L—

4”

Is

E

9 ‘\

11

\

N\‘

it

I1Hz

ii

S

0‘I

—1

>)

4’

F1

> b

11

&\

\\

ii

D

c aI%-

\\‘

‘F[

0

‘Ici.

0r

rCii

Uft

‘%__I

LA

k

__

4\

Csr

‘

Cl)%.

PRACTICAL AUGMENTED LAGRANGIAN METHODS:BOUND-CONSTRAINED FORMULATION

minimize f(x)

h(x)=O, l<x<u

LA (x,;tk) = f(x)+rn , rn

Akh(x)+kh(x)2

Bounded Gradient Lagrangian method

; Ik ) subject to

l<x<u

Ak+l Ak +kh(Xk)

Ak+1 Ak;

k+1 1OO;1 iaOA

11k+1Wk+1 l/&k+1;

(Bound-Constrained Lagrongian Method). I here is the projection operator tor boxed I. , . . . . * . 0 inequality (check your notes on gradient

Choose an initial point x0 and initial multipliers A ; projection method for further details)Choose convergence tolerances i4, and w;Set Co = 10, w = 1/ , and i = if ‘;fork = 0, 1, 2, .

ar0mate solution Xk subproble mm £A(x,Ak;) subject to I x U

IIxk (xkV1CA(xk,A’;Ck),1. <wk;

An efficient technique forsolving the nonlinear programwith bound constraints(forfixed .t andA) is the(nonlinear) gradient projectionmethod (see your notes fromlectures on primal methods)

ifllh(xk)1I 1IkNN

( * test for cwergence )if IIh(xk ) Ii s ‘7* Xk ‘ (Xk VItA (Xk , Ak ; k )‘ 1, u) II <

stop with apprimte solution xk;end(if)(* update multipliers, tighten tokq.ices )Ak+l Ak + kh(xk);

Remember that gradient projection method stopswhen the point generated from Xk by the gradientdescent algorithm gets projected back on Xk

Check the stopping condition of the gradientprojection method in your notes for more details.

else

L.)k+1Io9

Ilk-i-i = TIk/L..’k+i,WkI1 Wk/k+1

(* increase penalty parameter, tighten tolerances

If this condition holds, the penalty parameter is not changed forthe next iteration because the current value of Pk is producing anacceptable level of constraint violation. The Lagrange multiplierestimates are updated according to the update formula and thetolerances Wk and flk are tightened in advance of the nextiteration. If, on the other hand, this condition does nothold, then we increase the penalty parameter to ensure that thenext subproblem will place more emphasis on decreasing theconstraint violations. The Lagrange multiplier estimates are notupdated in this case; the focus is on improving feasibility.

end (if)end (for)

The constants 100, 0.1, 1 appearing hereare to some extent arbitrary;

—-—-———-—-—-—-———————-— other values can be used withoutcompromising theoretical convergenceproperties.