presentazione di powerpoint - uniroma1.itiacoviel/materiale/lecture_5_2019_sito.pdfpart of the...

Optimal Control

Prof. Daniela Iacoviello

Lecture 5

24/10/2019 Pagina 2

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM:YOU MUST STUDY ON THE BOOKS

Part of the slides has been taken from the References indicated below

Pagina 224/10/2019Prof. D.Iacoviello - Optimal Control

24/10/2019 Pagina 3

References

B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989

E.R.Pinch, Optimal control and the calculus of variations, Oxford science publications,

1993

C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993

A.Calogero, Notes on optimal control theory, 2017

L. Evans, An introduction to mathematical optimal control theory, 1983

H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972

D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004

D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise

Introduction", Princeton University Press, 2011

How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare:

Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.

……………………………………..

Pagina 324/10/2019Prof.Daniela Iacoviello- Optimal Control

24/10/2019 Pagina 4

Course outline

• Introduction to optimal control

• Calculus of variations

• Calculus of variations and optimal control

• The maximum principle

• The Hamilton Jacobi Bellman equation

• The linear quadratic regulator

• The minimum time problem

• The linear quadratic gaussian problem

Pagina 424/10/2019Prof.Daniela Iacoviello- Optimal Control

• Hamilton-Jacobi- Bellman equation

Prof. D.Iacoviello - Optimal Control 524/10/2019

William Rowan Hamilton

(4 August 1805 – 2 September 1865)

He was an Irish physicist, astronomer, and mathematician, who

made important contributions to classical mechanics,

optics and algebra.

His studies of mechanical and optical systems led

him to discover new mathematical concepts and techniques.

His greatest contribution is perhaps the reformulation of

Newtonian mechanics, now called Hamiltonian mechanics.

This work has proven central to the modern study of classical

field theories such as electromagnetism and to the development

of quantum mechanics. In mathematics, he is perhaps best

known as the inventor of quaternions.

.


Jacobi (Posdam 1804- Berlin 1851)

He was still only 12 years old yet he had reached the necessary

standard to enter University in 1817

Around 1825 Jacobi changed from the Jewish faith to

become a Christian which now made university

teaching possible for him.

By the academic year 1825-26 he was teaching at the University

of Berlin.

Jacobi carried out important research in partial differential

equations of the first order and applied them to the differential

equations of dynamics.

He also worked on determinants and studied the functional

determinant now called the Jacobian.


Richard Bellman (1920- 1984, USA)

He was interested in dynamic programming, with

applications in biology and medicine.


Principle of optimality


The Hamilton-Jacobi equation

It is an approach in some sense alternative to the Minimum

Principle of Pontryagin, combined with the Euler Lagrange

equation.

Actually the Hamilton – Jacobi equation has so far rarely

proved useful except for linear regulator problems, to which

it seems particularly well suited.



The Hamilton-Jacobi (H-J) equation is satisfied by the optimal

performance index under suitable differentiability and

continuity assumptions.

If a solution to the H-J equation has certain differentiability

properties, then this solution is the desired performance index.



- Such a solution need not exist

- Not every optimal performance index satisfies the Hamilton –

Jacobi equation

The HJ equation represents only a sufficient condition on the

optimal performance index



Minimizing over [t, tf] is equivalent to minimizing over [t, t1]

and [t1, tf]

t t1 tf

x(t)

xx1(t1)

x2(t1)

x3(t1)

x1(tf)

x2(tf)

x3(tf)



Principle of optimality:

OR:

( )

( )

( )

++

= )((),(),(min),(),(min),(

1

1

1

1 ,,

*f

t

tttu

t

tttu

txGduxLduxLttxJ

f

f

( )

( ) ( )

+= 11

*

,

* ),(),(),(min),(

1

1

ttxJduxLttxJ

t

tttu



Principle of optimality:

The optimal cost for trajectories commencing at t and

finishing at tf is incurred by minimizing the sum of the

cost in transiting to x1(t1) and the optimal cost from there

onwards that is:

and

( )

( ) ( )

+= 11

*

,

* ),(),(),(min),(

1

1

ttxJduxLttxJ

t

tttu

( )1

),(),(

t

t

duxL ( )ttxJ ),(*



A control policy optimal over the interval [t, tf] is

optimal over all subintervals [t1, tf]

t t1 tf

x(t)

xx1(t1)

x2(t1)

x3(t1)

x1(tf)

x2(tf)

x3(tf)


The Hamilton-Jacobi equation has so far rarely proved

useful except for linear regulator problems



The Hamilton-Jacobi equationAssume the Hamiltonian regular: there exists a unique minimum

with respect to u(t).

The Hamilton-Jacobi equation:


0),(

,,),(

),(),(),(

=

+

t

x

txVt

x

txVtxutxH

t

txVTT

0,,),(

),(),(),(

,),(

,,),(

),(),(),(

=

+

+

ttx

txVtxutxf

x

txV

tx

txVt

x

txVtxutxL

t

txV

TT

TT


Problem:

Consider the system:

Find the optimal control

that minimizes the cost:

( ) giventxtuxfx )(,,, 0=

fttttu ,),( 0*

( ) ( ) ( ))(),(),(),(),(

0

f

t

t

txGduxLtutxJ

f

+=


Define the Hamiltonian:

Assume:

➢ f, L and G are continuously differentiable in all

arguments

➢ The Hamiltonian regular: there exists a unique minimum

with respect to u(t),

Prof. D.Iacoviello - Optimal Control 24

( ) ( ) ( )tuxftuxLtuxH T ,,,,,,, +=

24/10/2019

Derivation of the Hamilton-Jacobi equation

Let’s consider the system:

with the cost index:

Notation:

Initial instant

initial state

freeorfixedttxfixedxuxfx ff ),(,),,( 0=

+= ft

tftxGdtttutxLJ

0

))(()),(),((minmin

+=

f

f

t

tf

ttttu

txGdtttutxLxtJ0

0

))(()),(),((min),(

,)(

00


++=

++=

+=

f

f

f

f

f

f

t

tf

t

tttt

tu

t

tf

t

tttt

tu

t

tf

ttttu

txGdtttutxLdtttutxL

txGdtttutxLdtttutxL

txGdtttutxLxtJ

1

1

0

0

1

1

0

0

0

0

))(()),(),(()),(),((min

))(()),(),(()),(),((min

))(()),(),((min),(

,)(

,)(

,)(

00


( ) ),,,(,,,),()),(),((min

))(()),(),(()),(),((min

],[00110111*

,)(

,)(

10

1

0

0

1

1

0

0

ttf

t

tttt

tu

t

tf

t

tttt

tu

uxttxxtttxtJdtttutxL

txGdtttutxLdtttutxL

f

f

f

=

+=

++=


Idea: dttt += 01

,)(,()),(),((min

))(()),(),((min),(

*

],[

,)(

*

+++=

++=

+

+dttxdttJdtttutxL

txGdtttutxLxtJ

dtt

tdttt

f

t

ttttu

f

f


smalldt

],[)()( dttttuu +

dtttutxLduxLdtt

t)),(),(()),(),((

+

dtd

Jdx

x

JtxtJdttxdttJ

+

+++

**** ))(,())(,(

+

++=

+++= +

+

dtt

xtJdx

x

xtJxtJdtttutxL

dttxdttJdtttutxLxtJdtt

tdttt

),(),(),()),(),((min

)(,()),(),((min),(

***

*

],[

*


+=

− dx

x

xtJdtttutxLdt

t

xtJ

tu

),()),(),((min

),( *

)(

*

Divide for dt and consider 0

lim→t

+=

− ),(

),()),(),((min

),( **

uxfx

xtJttutxL

t

xtJ


J*(x(t),t) is a solution of the Hamilton-Jacobi (H-J) equation:

with boundary conditions:

-J*(x(t),t) continuous with respect to its arguments

-the first derivatives of J*(x(t),t) continuous with respect to its

arguments

-

-

−

−=

tttx

Jtxutxf

x

J

ttx

JtxutxL

t

J

,,,),(),('

,,),(),(

**

**

( ) ( ))(),(*fff txGttxJ =


( ) ( )

= tttx

x

Jtxuttxu

T

o ,),(),(),(*

24/10/2019

is the optimal performance index for

( )( )

( )tutxJttxJfttu

),(),(min),(,

* =

( ) ( ) ( ))(),(),(),(),( f

t

t

txGduxLtutxJ

f

+=


and the control:

Is the optimal control at time t for the class problems with cost index

( ) ( )

= tttx

x

Jtxuttxuo ,),(),(),(

*

( ) ( ) ( ))(),(),(),(),( f

t

txGduxLuxJ

f

+=

24/10/2019


Example 2 (from Anderson)

Consider the system:

With performance index:

Applying the H-J equation we have:

ux =

( ) 0,2

1)(),0( 0

422

0

=

++= twithdtxxuuxJ

ft

t


The minimizing control is:

And we have:

With boundary condition:

+++−=

u

x

Jxxu

t

J

tu

*422

)(

*

2

1min

x

Ju

−=

*

2

1

42

2**

2

1

4

1xx

x

J

t

J−−

=

( ) 0),( =ff ttxJ

In actual fact, it is rarely possible to solve a Hamilton-

Jacobi equation



Main results

The Hamilton-Jacobi (H-J) equation is satisfied by the

optimal performance index under suitable differentiability

and continuity assumptions.

If a solution to the H-J equation has certain differentiability

properties, then this solution is the desired performance

index.

Note that the H-J equation represents only a sufficient

condition on the optimal performance index


presentazione di powerpoint - uniroma1.itiacoviel/materiale/lecture_5_2019_sito.pdfpart of the...

Documents