introduction to the mathematical theory of control, lecture 5motta/lectures valona...

logoslides

Introduction to the Mathematical Theory of Control,Lecture 5

Monica Motta

Dipartimento di MatematicaUniversità di Padova

Valona, September 15, 2017

M. Motta (Padua Un.) Control Theory Valona, September 15, 2017 1 / 19

Table of contents

1 Necessary optimality conditions: Pontryagin Maximum Principle (PMP)

2 Examples on the use of the PMP

3 Proof of the Pontryagin Maximum Principle

4 Some sufficient optimality conditions


PMP without final constraints

Given T > 0, an initial state x , and a cost function : Rn ! R weconsider the maximization problem

(P)

8>><

>>:

maxu2U

(x(T , u))

over the pairs (x , u)(·) verifying

x(t) = f (t , x(t), u(t)) u 2 U , x(0) = x .

We shall assume:

(HP

) The set ⌦ ⇢ Rn is open, f is continuous in ⌦⇥ U and C

1 in x . Thepayoff : Rn ! R is differentiable.

Notice that here we are not making any assumption on the set U ofadmissible control values. In particular, we may well have U = Rm.(We deal with optimization problems without endpoint constraints, since this is a case where the key

ideas can be more clearly described)M. Motta (Padua Un.) Control Theory Valona, September 15, 2017 3 / 19

Theorem 1 (PMP without endpoint constraints).Assume (H

P

). Let u

⇤(·) be a bounded, admissible optimal control for

problem (P), whose corresponding trajectory x

⇤(·) is optimal. Denote

p : [0,T ] ! Rn

the solution of the adjoint equation

p(t) = �p(t)D

x

f (t , x⇤(t), u⇤(t)), p(T ) = D (x⇤(T )). (1)

Then the maximality condition

p(t) · f (t , x⇤(t), u⇤(t)) = maxw2U

p(t) · f (t , x⇤(t),w), (2)

holds for a.e. t 2 [0,T ].


In coordinates, the adjoint equation can be rewritten as

p

i

(t) = �nX

j=1

p

j

(t)@f

j

@x

i

(t , x⇤(t), u⇤(t)), p

i

(T ) =@

@x

i

(x⇤(T )) (3)

(i = 1, . . . , n), and the maximality condition as

nX

i=1

p

i

(t) f

i

(t , x⇤(t), u⇤(t)) = maxw2U

(nX

i=1

p

i

(t) f

i

(t , x⇤(t),w)

), (4)

for a.e. t 2 [0,T ].


Geometric meaning of the maximality condition

’At each time ⌧ , among the possible attainable speedsx(⌧) 2 {f (⌧, x⇤(⌧),w) : w 2 U} , we should choose the one thatmaximizes the scalar product p(⌧) · x(⌧)’.


The Pontryagin Maximum Principle motivates a practical method forfinding optimal solutions.

1 Define the function u = u(t , x , u) 2 U, in terms of the maximalitycondition:

p · f (t , x , u(t , x , p)) = maxw2U

p · f (t , x ,w)

2 solve the system of 2n differential equations in the variables x , p:(

x = f (t , x , u(t , x , p))

p = �D

x

f (t , x , u(t , x , p))

with boundary conditions

x(0) = x , p(T ) = D (x(T )).


In general, this method encounters two main difficulties:

1 The map (t , x , p) 7! u(t , x , p) may be multivalued and/ordiscontinuous.

2 The above equations in (x , p)(·) do not constitute a Cauchy problemon R2n, but a (usually harder) two–point boundary value problem.

In particular cases, however, the equations for p and x can beuncoupled, and a solution is more easily found.


Examples

Example 2 (Linear pendulum with external force).Let q be the position of a linearized pendulum controlled by an externalforce u, with magnitude constraint

|u(t)| 1 8t .

Assume that the motion is determined by the equations

q(t) + q(t) = u(t), q(0) = q(0) = 0.

We wish to maximize the displacement q(T ) at a fixed terminal time T .If x1 := q, x2 := q, the optimization problem can be formulated as

(P)

8>><

>>:

maxu

x1(T , u)

over the pairs (x , u)(·) verifying u(t) 2 [�1, 1] and

(x1, x2) = (x2,�x1 + u), (x1, x2)(0) = (0, 0).


Example 3.To appreciate the effect of the map u = u(t , x , p) being multivalued,consider the problem

(P)

8>><

>>:

maxu

x3(T , u)


(x1, x2, x3) = (u,�x1, x2 � x

21 ), (x1, x2, x3)(0) = (0, 0, 0).


Example 4.To see that the conditions of the PMP are not sufficient for optimality,consider the problem

(P)

8>><

>>:

maxu

x2(T , u)


(x1, x2) = (u, x21 ), (x1, x2)(0) = (0, 0),

and notice that the constant control u

⇤ ⌘ 0 yields the constant trajectoryx

⇤ ⌘ 0 and they satisfy the conditions of the PMP (verify!). Howeveru

⇤ ⌘ 0 is NOT optimal.


Basic steps of the proof of the PMPTo understand the derivation of the Pontryagin Maximum Principle (PMP), it is best to assume first theoptimal control u

⇤(·) is continuous. In this case the maximality condition holds for all t 2 [0, T ].

1 Fix any time ⌧ > 0 and any value w 2 U. For any " > 0, consider a perturbed control function

u"(t) =

(w if t 2 [⌧ � ", ⌧ ]

u

⇤(t) otherwise

called a needle variation of u.2 Let x"(·) := x(·, u"). Evaluate for any t � ⌧ :

v(t) :=dx"

d"(t)

��"=0+

. Observe that v(⌧) = f (⌧, x

⇤(⌧),w)� f (⌧, x

⇤(⌧), u

⇤(⌧)).

3 Since u

⇤(·) is optimal, (x⇤(T )) � (x"(T )) for any " > 0. Hence, differentiating w.r.t. ",

0 � lim"!0+

(x"(T ))� (x⇤(T ))

"= D (x⇤(T )) ·

dx"

d"(T )

��"=0+

= D (x⇤(T )) · v(T )

4 Prove that the the scalar product p(t) · v(t) is constant, so that the condition at T can be"transported" at ⌧ and the maximality condition follows:

0 � D (x⇤(T ))·v(T ) = p(T )·v(T )=p(⌧)·v(⌧) = p(⌧)·[f (⌧, x

⇤(⌧),w)�f (⌧, x

⇤(⌧), u

⇤(⌧))].


Some pictures:

Needle variation of u

Perturbed trajectories and variation v


Tools:

Theorem 5 (Differentiability respect to initial conditions).

Let g : R⇥ Rn ! Rn

be a function, measurable in t , C

1in x and such that

there exists constants C, L such that

|g(t , x)| C, |g(t , x)� g(t , y)| L|x � y | 8(t , x , y). (5)

Let x(·) := x(·, t0, x0) be the solution of

x(t) = g(t , x(t)), x(t0) = x0. (6)

For a vector v0 2 Rn, call v(·) the solution of the linear Cauchy problem

v(t) = D

x

g(t , x(t))v(t), v(t0) = v0.

Then for any t > t0,

lim"!0+

x(t , t0, x0 + "v0)� x(t)

"= v(t).


Set y"(·) := x(·) + " v(·). The picture describes the first order variationof a solution of a ODE, as the initial point x0 is changed to x0 + "v0.


Adjoint systemsWe call adjoint system of the linear system

x(t) = A(t) x(t) (7)

the differential equationp(t) = �p(t)A(t), (8)

that is,

(p1(t), . . . , p

n

(t)) = � (p1(t), . . . , p

n

(t))

0

B@

a11(t) . . . a1n

(t)...

...a

n1(t) . . . a

nn

(t)

1

CA

Theorem 6 (Property of the adjoint system).Let x(·), p(·) two solutions of (7), (8) respectively, defined on the same

time interval [a, b]. Then their scalar product p(t) · x(t) is constant.

Proof. This is verified by the direct computation

d

dt

(x · p) = x · p + x · p = p · A(t)x � pA(t) · x = 0.


For the general case, of possibly discontinuous optimal control u

⇤(·), weapply to t 7! h(t) = f (t , x⇤(t), u⇤(t)) the following

Theorem 7 (Lebesgue Differentiation Theorem).Let h : [a, b] ! Rn

be integrable. Then, almost every ⌧ 2 [a, b] is a

Lebesgue point, i.e.

lim"!0+

12"

Z ⌧+"

⌧�"|h(⌧)� h(s)| ds = 0.


Some sufficient optimality conditions

The simplest sufficient optimality conditions can be obtained combiningthe conditions for the existence of optimal controls with the PMP.

Theorem 8 (Existence + PMP).Assume (H) and let be continuously differentiable. Assume that an

optimal solution for problem (P) exists.

If there is a finite number of admissible controls u1, . . . , uk

that satisfy the

PMP, then, among them, the one which yields the lowest value of the cost

is optimal.

Less trivial results can be obtained combining convexity assumptions withthe PMP.

Let me mention a different approach to derive sufficient optimalityconditions, NOT based on the PMP, known as Dynamic Programming.


Thank you for your attention!

WEB page: http://www.math.unipd.it/⇠ motta/

E-Mail: [email protected]


LECTURE 5: the PMP, proofgiovedì 14 settembre 2017 16:01

Nuova sezione 1 Pagina 1

introduction to the mathematical theory of control, lecture 5motta/lectures valona...

Documents