fdi_2013_opt_tool (1)

8/10/2019 fdi_2013_opt_tool (1)

http://slidepdf.com/reader/full/fdi2013opttool-1 1/37

Matlab-based Optimization:

theOptimization Toolbox

Gene Cliff (AOE/ICAM - [email protected] )3:00pm - 4:45pm, Monday, 11 February 2013.......... FDI ..........

AOE: Department of Aerospace and Ocean Engineering

ICAM: Interdisciplinary Center for Applied Mathematics

1 / 3 7

8/10/2019 fdi_2013_opt_tool (1)


Matlab’s Optimization Toolbox

Classifying Optimization Problems ⇐

A Soup Can Example

Intermezzo

A Trajectory Example

2nd Trajectory Example: fsolve

2 / 3 7

8/10/2019 fdi_2013_opt_tool (1)


Solver Categories

There are four general categories of Optimization Toolbox solvers:

MinimizersThis group of solvers attempts to find a local minimum of the objective function near a starting point x0.

They address problems of unconstrained optimization, linear programming, quadratic programming, and

general nonlinear programming.

Multiobjective minimizers

This group of solvers attempts to either minimize the maximum value of a set of functions (fminimax), or

to find a location where a collection of functions is below some prespecified values (fgoalattain).

Least-Squares (curve-fitting) solversThis group of solvers attempts to minimize a sum of squares. This type of problem frequently arises in

fitting a model to data. The solvers address problems of finding nonnegative solutions, bounded or linearly

constrained solutions, and fitting parameterized nonlinear models to data.

Equation solversThis group of solvers attempts to find a solution to a scalar- or vector-valued nonlinear equation f(x) = 0

near a starting point x0. Equation-solving can be considered a form of optimization because it is equivalent

to finding the minimum norm of f(x) near x0.

3 / 3 7

8/10/2019 fdi_2013_opt_tool (1)


8/10/2019 fdi_2013_opt_tool (1)


Classifying a Problem

Identify your objective function as one of five types:LinearQuadraticSum-of-squares (Least squares)Smooth nonlinear

Nonsmooth

Identify your constraints as one of five types:

None (unconstrained)

BoundLinear (including bound)General smoothDiscrete (binary integer)

5 / 3 7

8/10/2019 fdi_2013_opt_tool (1)


Problem classification table

We focus on fmincon

6 / 3 7

8/10/2019 fdi_2013_opt_tool (1)



Classifying Optimization Problems

A Soup Can Example ⇐

Intermezzo



7 / 3 7

( )

8/10/2019 fdi_2013_opt_tool (1)


Soup Can Example (from MathWorks Training Docs)

We are to design a soup can in the shape of a right circular

cylinder.We are to choose values for:1 the diameter (d ),

2 the height (h)

Requirements are:

the volume (πd 2

4 h) must be 333 cm3

the height can be no more than twice the diameter

the cost is proportional to the surface area (πd 2

2 + πdh), and

should be minimized

Since the cost function and the volume constraint are nonlinear,we select fmincon.

8 / 3 7

8/10/2019 fdi_2013_opt_tool (1)


S f i

8/10/2019 fdi_2013_opt_tool (1)


Soup can: cost function

f u n c t i o n [ v a l v a l x ] = c o s t s o u p c a n ( x )% E v a lu at e t he c o s t f u n c t i o n f o r t h e soup −c an e x a mp l e %% x ( 1 ) − d ia me te r o f t he can% x ( 2 ) − h e i g h t o f t he can%% a r ea = 2 ∗( p i ∗d ˆ 2 ) / 4 + p i ∗d ∗h

v a l = p i ∗x (1 )∗ ( x ( 2 ) + x ( 1 ) / 2 ) ;

% Ev a l u a t e t he g r a d i e n t i f n ar g ou t > 1

v a l x = p i ∗ [ x (1)+ x ( 2 ) ; x ( 1 ) ] ;

end

end

10/37

S l i

8/10/2019 fdi_2013_opt_tool (1)


Soup can: volume constraint

f u n c t i o n [ c ceq c x c eq x ]= c o n s o u p c a n ( x , volume )%E v a l ua t e t he c o n s t r a i n t f o r t he soup −c an e x am pl e % x ( 1 ) − d ia me te r o f t he can% x ( 2 ) − h e i g h t o f t he can%

c = [ ] ; % no n o n l i n e a r i n e q u a l i t i e s

c e q = v o l u m e − ( p i / 4 )∗ x ( 2 )∗ x ( 1 ) ˆ 2 ; % v o l u m e = p i ∗d ˆ2 ∗h /4

% compute t h e J a c o b i a n s i f n a rg ou t > 2

c x = [ ] ;c e q x = −( p i / 4 )∗ x (1 )∗ [ 2∗ x ( 2 ) ; x ( 1 ) ] ;

end

end

11/37

S t i t

8/10/2019 fdi_2013_opt_tool (1)


Soup can: set-up script

% S c r i p t t o s e t up soup −c an e x am p le % We a r e t o d e si g n a r i g ht −c y l i n d r i c a l ( c i r c u l a r ) c a n o f a g i v e n v ol u me % and w it h minimum s u r f a c e a r e a ( m a t e r i a l c o s t ) . The h e i g h t c an be no more % th an t w ic e t he d i am et e r

% v o l u m e = p i ∗dˆ2 ∗h /4 % a r e a = 2 ∗( p i ∗d ˆ 2 ) / 4 + p i ∗d ∗h% h \ l e 2 ∗d == > −2 ∗d + h \ l e 0

% I n o ur o p t i m i z a t i o n p ro bl em we h av e % x = [ d ; h ] ;% The s p e c i f i e d v ol um e i s 333 cmˆ3

% We h av e e x t e r n a l f u n c t i o n f i l e s % c o s t s o u p c a n .m% c o n s o u p c a n .m

%% d e f i n e h an dl e t o t he c o n s t r a i n t f u n ct i o n w it h t he s p e c i f i e d volume v a l u e v o l u m e = 3 3 3 ;h c o n = @( x ) c o n s o u p c a n ( x , v ol um e ) ;

% A rr ay s f o r t h e l i n e a r i n e q u a l i t y A = [−2 1 ] ; b = 0 ;

% l o w e r / u p p er b ou nd s l b = [ 4 ; 5 ] ;ub = [ 8 ; 1 5 ] ;

% i n i t i a l g u e s s x0 = [ 6 ; 1 0 ] ;

12/37

ti t l l

8/10/2019 fdi_2013_opt_tool (1)


optimtool: soup can example

13/37

C d Wi d s l

8/10/2019 fdi_2013_opt_tool (1)


Command Window: soup can example

>>

s o u p c a n 2

Max L i n e s e a r c h D i r e c t i o n a l F i r st−o r d e rI t e r F−c o u n t f ( x ) c o n s t r a i n t s t e p l e n g t h d e r i v a t i v e o p t i m a l i t y P r o c e d u r e

0 3 2 4 5 . 0 4 4 5 0 . 2 6 I n f e a s i b l1 6 2 4 7 . 1 3 8 3 5 . 4 1 1 3 . 9 3 1 . 22 9 2 6 5 . 1 1 3 1 . 7 1 3 1 1 7 . 3 2 . 6 83 12 2 6 5 . 9 4 8 0 . 0 5 2 8 5 1 6 . 9 2 0 . 7 9 84 15 2 6 5 . 9 2 0 . 0 6 9 3 9 1 −0.0716 0 . 0 8 9 95 18 2 6 5 . 9 5 6 0 . 0 0 0 1 1 7 4 1 6 . 4 9 0 . 0 0 3 2 66 21 2 6 5 . 9 5 7 4 . 8 7 1 e−08 1 0 . 5 3 6 . 8 8 e−05 H e ss ia n

L o c a l minimum p o s s i b l e . C o n s t r a i n t s s a t i s f i e d .

f mi nc on s to pp ed b ec au se t he p r e d i c t ed c ha nge i n t he o b j e c t i v e f u n c t i o ni s l e s s t h a n t h e s e l e c t e d v al ue o f t he f u n c t i o n t o l e r a n c e and c o n s t r a i n t swe re s a t i s f i e d t o w i t hi n t he s e l e c t e d v a lu e o f t he c o n s t r a i n t t o l e r a n c e .

<s t o p p i n g c r i t e r i a d e t a i l s>

No a c t i v e i n e q u a l i t i e s .>>

14/37


8/10/2019 fdi_2013_opt_tool (1)


Matlab s Optimization Toolbox


A Soup Can Example

Intermezzo ⇐



15/37

fmincon: choice of algorithms

8/10/2019 fdi_2013_opt_tool (1)


fmincon: choice of algorithms

‘trust-region reflective’ requires you to provide a gradient, and

allows only bounds or linear equality constraints, but not both. Within theselimitations, the algorithm handles both large sparse problems and small dense

problems efficiently. It is a large-scale algorithm, and can use special techniques

to save memory usage, such as a Hessian multiply function. For details, see

Trust-Region-Reflective Algorithm.

‘active-set’ can take large steps, which adds speed. The algorithm is effective

on some problems with nonsmooth constraints. It is not a large-scale algorithm.

‘sqp’ satisfies bounds at all iterations. The algorithm can recover from NaN or

Inf results. It is not a large-scale algorithm.

‘Interior-point’ handles large, sparse problems, as well as small dense

problems. The algorithm satisfies bounds at all iterations, and can recover from

NaN or Inf results. It is a large-scale algorithm, and can use special techniques

for large-scale problems.

16/37

Large Scale vs Medium Scale

8/10/2019 fdi_2013_opt_tool (1)


Large-Scale vs Medium-Scale

An optimization algorithm is large scale when it uses linear algebra that doesnot need to store, nor operate on, full matrices. This may be done internally bystoring sparse matrices, and by using sparse linear algebra for computationswhenever possible. Furthermore, the internal algorithms either preserve sparsity,such as a sparse Cholesky decomposition, or do not generate matrices, such asa conjugate gradient method. Large-scale algorithms are accessed by settingthe LargeScale option to on, or setting the Algorithm option appropriately (thisis solver-dependent).

In contrast, medium-scale methods internally create full matrices and use denselinear algebra. If a problem is sufficiently large, full matrices take up asignificant amount of memory, and the dense linear algebra may require a longtime to execute. Medium-scale algorithms are accessed by setting theLargeScale option to off, or setting the Algorithm option appropriately (this issolver-dependent).

Don’t let the name ”large-scale” mislead you; you can use a large-scalealgorithm on a small problem. Furthermore, you do not need to specify any

sparse matrices to use a large-scale algorithm. Choose a medium-scale

algorithm to access extra functionality, such as additional constraint types, or

possibly for better performance.

17/37

fmincon: command line inputs

8/10/2019 fdi_2013_opt_tool (1)


fmincon: command line inputs

x = fmincon(fun, x0, A, b, Aeq, beq, lb, ub, nonlcon,

options)

fun - function handle for the cost function

x0 - initial guess for solution

A, b - matrix, rhs vector for inequality constraints (A x ≤ b )

Aeq, beq - matrix, rhs vector for equality constraints

lb, ub - lower,upper bounds for solution vector

nonlincon - function handle for the nonlinear inequality andequality constraints; [c, ceq] = nonlincon(x)

options - structure of options for the algorithm

18/37

fmincon: additional outputs

8/10/2019 fdi_2013_opt_tool (1)


fmincon: additional outputs

[x,fval,exitflag,output,lambda,grad,hessian]

exitflag1: First-order optimality measure was less than options.TolFun,and maximum constraint violation was less than .TolCon.0: Number of iterations exceeded options.MaxIter or numberof function evaluations exceeded options.FunEv

output -structure of data about performance of the algorithm

lambda - structure of the Lagrange multipliers

grad -gradient of the Lagrangian

Hessian -Hessian of the Lagrangian

19/37


8/10/2019 fdi_2013_opt_tool (1)


Matlab s Optimization Toolbox


A Soup Can Example

Intermezzo

A Trajectory Example ⇐


20/37

Trajectory Example

8/10/2019 fdi_2013_opt_tool (1)


Trajectory Example

We are to launch an object at speed v 0; we seek an initial elevationangle for maximum range. In the classical case with no drag, thebest elevation is π

4 . Suppose we have a simple drag force; b v 2 ?

formulate an initial-value problem for the projectile motion

the initial position and speed are given, the initial elevationangle (γ (0)) is unknown

the final range (x (t f ) to be maximized) occurs when theheight returns to its initial value (final time (t f ) is unknown)

Since the cost function and the final height constraint arenonlinear functions of the unknowns, we select fmincon.

21/37

Trajectory: Setup and solve an IVP

8/10/2019 fdi_2013_opt_tool (1)


Trajectory: Setup and solve an IVP

f u n c t i o n [ r an ge , a l t i t u d e ] = t r a j e c t o r y ( gam 0 , t f , param )% S o l ve an IVP f o r t he b a l l i s t i c t r a j e c t o r y % E v a l u a te t he f i n a l a l t i t u d e and r a ng e

%% g am 0 i s t he i n i t i a l f l i g h t −p at h a n g le ( r a d i a n s )% t f i s t h e f i n a l t i m e ( s )%% r a ng e / a l t i t u d e a r e t he f i n a l v a l u e s %% param i s a d at a s t r u c t u r e % param . b c o ef i s t he d r ag c o e f f i c i e n t % param . g ra v i s t he g r a v i t a t i o n a l a c c e l e r a t i o n (m/ s ̂ 2 )

% p aram . v e l 0 i s t h e i n i t i a l s p e ed (m/ s )

% anonymous f u n c t i o n h a nd l e w it h s p e c i f i e d p a ra m et e rs h r h s = @ ( t , z ) b a l l i s t i c r h s ( t , z , p ar am . b c o e f , p ar am . g r a v ) ;z 0 = [ 0 ; 0 ; param . v e l 0 ; gam 0 ] ; % s e t t he i n i t i a l s t a t e [ ˜ , Z ] = ode23 ( h r h s , [ 0 t f ] , z 0 ) ; % s o l v e t he IVP r a n g e = Z( en d , 1 ) ;a l t i t u d e = Z( en d , 2 ) ;

en d

Note that to evaluate the cost we need the range, and to evaluatethe constraint we need the altitude.Do we really have to solve the IVP twice to evaluate both ?

22/37

Trajectory: the RHS of the ODE system

8/10/2019 fdi_2013_opt_tool (1)


Trajectory: the RHS of the ODE system

f u n c t i o n z d o t = b a l l i s t i c r h s ( ˜ , z , b c o e f , g r a v )% E v al u a t e r h s o f eq . o f mo ti on f o r a b a l l i s t i c o b j e c t % z = [ x , h , v , gamma ] % x − r a n g e % h − a l t i t u d e % v − s p e e d

% gamma − f l i g h t −p at h a n g l e

s i n g = s i n ( z ( 4 ) ) ; c os g = c o s ( z ( 4 ) ) ;v = max ( z ( 3 ) , 0 . 1 ) ; % g u a r d a g a i n s t z er o d i v i s o r z d o t = [ z ( 3)∗ c o s g ; z ( 3 )∗ s i n g ;

−b c o e f ∗ z (3 )∗ z (3 ) − g r a v ∗ s i n g ;−g r a v ∗ c o s g / v ] ;

end

23/37

Matlab: ObjectiveandConstraints

8/10/2019 fdi_2013_opt_tool (1)


j v

f u n c t i o n [ c o st , n o n l i n c o n ] = O b j e c t i v e a n d C o n s t r a i n t s ( p aram )% E n ca p s ul a te s c o st and c o n s t r a i n t f u n c t i o n s f o r f mi nc on% co s t and n o nl i nc o n a r e f u n c t i on h a nd l es % param i s a s t r u c t u r e t h at e nc od es p ar am et er s f o r t he c o s t / c o n s t r a i n t f c n s % I n i t i a l i z e v a r i a b l e s and make them a v a i l a b l e t o t he n e s te d f u n c t i o n s

r a n g e = [ ] ; a l t i t u d e = [ ] ; L a st Z = [ ] ; % i n i t i a l i z e c o s t = @ ob je ct i ve ; n o n l i n c o n = @ co n st ra in ts ;

% Ne st ed f u n c t i o n s f u n c t i o n [ v al , v a l Z ] = o b j e c t i v e ( z )

i f ˜ i s e q u a l ( z , L a st Z ) % u p d at e f o r t h i s v a l u e % S o lv e t he IVP

[ r an ge , a l t i t u d e ] = t r a j e c t o r y ( z ( 1 ) , z ( 2 ) , param ) ;L a s t Z = z ;

en d% E v a lu at e c os t

v a l = −r a n g e ; % mi n im i ze t he n e g a t i v e r an ge v a l Z = [ ] ; % g r a d i e nt n ot computed i n t h i s v e r s i o n

en d%

f u n c t i o n [ c , c eq c Z , c eq Z ] = c o n s t r a i n t s ( z )i f ˜ i s e q u a l ( z , L a st Z ) % u p d at e f o r t h i s v a l u e

% S o lv e t he IVP

[ r an ge , a l t i t u d e ] = t r a j e c t o r y ( z ( 1 ) , z ( 2 ) , param ) ;L a s t Z = z ;

en d% E va lu at e c o n s t r a i n t s

c = [ ] ; % no i n e q u a l i t y c o n s t r a i n t s c e q = a l t i t u d e ;c Z = [ ] ; % J a c o b i a n s n o t c om pu te d c e q Z = [ ] ;

en d

en d24/37

ObjectiveandConstraints: insights

8/10/2019 fdi_2013_opt_tool (1)


j g

Invoking ObjectiveandConstraints defines the functionhandles cost and nonlincon.

Since the variables: param, range, altitude, lastZ aredefined at the high-level, they are available to the nested

functions objective and constraints.

If z = LastZ we solve the IVP and return range and

altitude.

If z == LastZ we use the stored values of range andaltitude.

This approach is useful in cases wherein evaluating the

cost/constraint functions requires an expensive calculation,such as the solution of an ODE/IVP or a PDE/BVP.

Future documentation of the Optimization Toolbox willinclude this description.

25/37

fmincon:trajectory example

8/10/2019 fdi_2013_opt_tool (1)


j y p

% S c r i p t t o s e t p ar am et er s f o r and t he n r un t he max −r an ge t r a j e c t o r y p ro bl em%% param i s a s t r u c t u r e o f d at a f o r t he p r o b l e m% param . b c o ef i s t he d r ag c o e f f i c i e n t

% param . g ra v i s t he g r a v i t a t i o n a l a c c e l e r a t i o n (m/ s ̂ 2 )% p aram . v e l 0 i s t h e i n i t i a l s p e ed (m/ s )

param . b c o e f = 0 . 1 ;param . g ra v = 9 . 8 ;param . v e l 0 = 2 5 . 0 ;

% d e f i n e h an d l e s f o r f u n c ti o n s e v a l u a t in g t he c o s t / c o n t r a i n t s

[ c o st , n o n lc o n ] = O b j e c t i v e a n d C o n s t r a i n t s ( p aram ) ;

% l o w e r / u p p er b ou nd s l b = [ 0 ; 0. 5∗ param . v e l 0 /param . gra v ] ;ub = [ p i / 4 ; 5∗ l b ( 2 ) ] ;

% i n i t i a l g u e s s x 0 = 0 . 5∗( lb+ub ) ;

%% s e t p a r a me t e r s a nd i n v o k e f m in c onOPT = op ti ms et ( ’ fmin con ’ ) ;OPT = o p t i m s e t (OPT , ’ A l g o r i t h m ’ , ’ a c t i v e−s e t ’ , . . .

’ D i sp l a y ’ , ’ i t e r ’ , . . .’ U s e P a r a l l e l ’ , ’ a l w a ys ’ ) ;

% x s t a r = f m in co n ( f un , x0 , A , b , Aeq , beq , l b , ub , n on lc on , o p t i o n s )x s t a r = f m in co n ( c os t , x0 , [ ] , [ ] , [ ] , [ ] , l b , ub , n on lc on , OPT ) ;

26/37


8/10/2019 fdi_2013_opt_tool (1)


p


A Soup Can Example

Intermezzo


2nd Trajectory Example: fsolve ⇐

27/37


8/10/2019 fdi_2013_opt_tool (1)


With the same dynamics as earlier, we now seek an initial elevationangle (γ 0)) and a final time (t f ) so that the trajectory ends at a

specified point in the vertical plane (x f , hf ).

since the IVP solution depends on time, as well as on theinitial elevation angle, we write the range and height functionsas x (t ; γ 0) and h(t ; γ 0), respectively.

we want to find values of t f and γ 0 that lead to zero for thevector-valued function:

f 1(γ 0, t f )

= x (t f , γ 0) − x f

f 2

(γ 0

, t f )

= h(t f , γ 0

) − hf

we use the the function fsolve from the Optimization

Toolbox

28/37

Modified trajectory code

8/10/2019 fdi_2013_opt_tool (1)


This version can return the time/state history [T, Z]

f u n c t i o n [ r e s i d u a l , T , Z ] = t r a j e c t o r y ( gam 0 , t f , param )% S o l ve an IVP f o r t he b a l l i s t i c t r a j e c t o r y

% E v a l u a te t he f i n a l a l t i t u d e and r a ng e %% g am 0 i s t he i n i t i a l f l i g h t −p at h a n g le ( r a d i a n s )% t f i s t h e f i n a l t i m e ( s )%% r a ng e / a l t i t u d e a r e t he f i n a l v a l u e s %% param i s a d at a s t r u c t u r e % param . b c o ef i s t he d r ag c o e f f i c i e n t

% param . g ra v i s t he g r a v i t a t i o n a l a c c e l e r a t i o n (m/ s ̂ 2 )% p aram . v e l 0 i s t h e i n i t i a l s p e ed (m/ s )% param . x f i s t h e s p e c i f i e d t a r g e t r an g e (m)% param . h f i s t h e s p e c i f i e d t a r g e t a l t i t u d e (m)

% anonymous f u n c t i o n h a nd l e w it h s p e c i f i e d p a ra m et e rs h r h s = @ ( t , z ) b a l l i s t i c r h s ( t , z , p ar am . b c o e f , p ar am . g r a v ) ;z 0 = [ 0 ; 0 ; param . v e l 0 ; gam 0 ] ; % s e t t he i n i t i a l s t a t e i f n a r go u t == 1

[ ˜ , Z ] = ode23

( h r h s , [ 0 t f ] , z 0 ) ; % s o l v e t he IVP r e s i d u a l = Z( en d , 1 : 2 ) ’ − [ param . x f ; param . h f ] ;e l s e

[ T , Z ] = ode23 ( h r h s , [ 0 t f ] , z 0 ) ; % s o l v e t he IVP r e s i d u a l = Z( en d , 1 : 2 ) ’ − [ param . x f ; param . h f ] ;

en den d% ’ l o c a l ’ b a l l i s t i c f u n c t i o n g o es h e r e

29/37

fsolve: 2nd trajectory example

8/10/2019 fdi_2013_opt_tool (1)


% S c r i p t t o s e t p ar am et er s f o r and t h en r u n a t r a j e c t o r y t a r ge t p r o b l e m%% param i s a s t r u c t u r e o f d at a f o r t he p r o b l e m% param . b c o ef i s t he d r ag c o e f f i c i e n t

% param . g ra v i s t he g r a v i t a t i o n a l a c c e l e r a t i o n (m/ s ̂ 2 )% p aram . v e l 0 i s t h e i n i t i a l s p e ed (m/ s )% param . x f i s t h e s p e c i f i e d t a r g e t r an g e (m)% param . h f i s t h e s p e c i f i e d t a r g e t a l t i t u d e (m)

param . b c o e f = 0 . 1 ;param . g ra v = 9 . 8 ;param . v e l 0 = 2 5 . 0 ;

param . x f = 8 . 0 ;param . h f = 2 . 0 ;

% d e f i n e h an d l e s f o r f u n c ti o n s e v a l u a t in g t he c o s t / c o n t r a i n t s

f h n d l = @( x ) t r a j e c t o r y ( x ( 1 ) , x ( 2 ) , param ) ;

% i n i t i a l g u e s s x 0 = [ p i / 4 ; 0 .5∗param . ve l 0 /param . gra v ] ;

%% s e t p a ra m et e rs and i n v ok e f s o l v e OPT = o p t i m s e t ( ’ f s o l v e ’ ) ;OPT = o p t i m s et (OPT, ’ D i s p l a y ’ , ’ i t e r ’ , . . .

’ U s e P a r a l l e l ’ , ’ a l w a ys ’ ) ;

% [ x s t a r , f v a l , e x i t f l a g ] = f s o l v e ( FUN , X0 , OPTIONS )[ x s t a r , ˜ , f l a g ] = f s o l v e ( f h n dl , x0 , OPT ) ;

30/37

fsolve: 2nd trajectory example

8/10/2019 fdi_2013_opt_tool (1)


% [ x s t a r , f v a l , e x i t f l a g ] = f s o l v e ( FUN , X0 , OPTIONS )[ x s t a r , ˜ , f l a g ] = f s o l v e ( f h n dl , x0 , OPT ) ;

i f f l a g == 1[ ˜ , T , Z ] = t r a j e c t o r y ( x s t a r ( 1 ) , x s t a r ( 2 ) , param ) ;f i g u r e

p l o t ( Z ( : , 1 ) , Z ( : , 2 ) , ’−−k ’ , ’ L i n eW i dt h ’ , 2 ) ;h o l d on ; g r i d onp l o t ( p a r am . x f , p ar am . h f , ’ r o ’ )x l a b e l ( ’ r a n g e (m) ’ ) ; y l a b e l ( ’ h e i g h t (m) ’ )

e l s ef p r i n t f ( 1 , ’\n f l a g = %02 i \n ’ , f l a g ) ;

en d

31/37

2nd trajectory example: fsolve

8/10/2019 fdi_2013_opt_tool (1)


Note that as in the zero-drag case, the problem has two solutions

Low trajectory High trajectory

32/37

THE END

8/10/2019 fdi_2013_opt_tool (1)


Please complete the evaluation formhttp://www.fdi.vt.edu/training/evals/

Thanks

33/37

Backup - underlying ideas - problem w/o constraints

8/10/2019 fdi_2013_opt_tool (1)


Problem P 0: Find x ∗ ∈ IRn to minimize a smooth function

f : IRn → IR.

We assume that f is twice continuously differentiable in theneighborhood of a solution.

If x ∗ a minimizer for P 0, then x ∗ is a stationary point for f , sothat (∇f )x ∗ = 0 ∈ IR

n, furthermore the Hessian of f , is

positive semi-definite, ∇2

f x ∗ ≥ 0.Applying Newton’s method to ∇f = 0 we get the updatep k = −

∇2f

−1

x k (∇f )x k

, x k +1 = x k + p k

Algorithms for P 0 generate estimates for∇2f

based on

computes changes in (∇f )

The update is commonly generalized to x k +1 = x k + αp k ,where α > 0 is a step-size.

Trust-region methods minimize a quadratic approximation tof near x k subject to a step size (trust-region radius).

34/37

8/10/2019 fdi_2013_opt_tool (1)


Backup- underlying ideas - problem w equality constraints

8/10/2019 fdi_2013_opt_tool (1)


Problem P c: Find x ∗ ∈ IRn to minimize a smooth function

f : IRn → IR; subject to g (x ) = 0 ∈ IRm where

g : IRn → IRm

We assume that f , g are twice continuously differentiable inthe neighborhood of a solution.

If x ∗

is a minimizer for P c and the Jacobian J = ∇g has fullrank at x ∗ then there exists a vector λ̂ ∈ IR

m such that x ∗ is astationary point for the Lagrange function

L(x ) = f (x ) + λ̂, g (x ). Furthermore, x ∗ is a local minimizerfor L in the null space of J (x ∗).

The latter condition implies that the projected Hessian of L ispositive semi-definite Z T

∇2L

x ∗

Z ≥ 0 where the columnsof Z span the null-space of J (x ∗).

36/37

Backup- underlying ideas - problem w inequality constraints

8/10/2019 fdi_2013_opt_tool (1)


Problem P i - the constraints = k + 1, ..., m are inequalities,

g ≤ 0.

Karush-Kuhn-Tucker theory implies that λ ≥ 0(NB - in some formulations λ ≤ 0.)

Many algorithms are based on an active-set strategy. Someset A ⊂ {k + 1, ..., m} of inequalities are treated as equalitiesin a version of problem P c

At each (major) iteration the set A is adjusted:1 if g > 0 for some ∈ Ac , then add to the active-set2 If λ < 0 for some ∈ A, then remove from the active-set

37/37

fdi_2013_opt_tool (1)

Documents