optimalcontrol_1
DESCRIPTION
aircraft teori in control systemTRANSCRIPT
![Page 1: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/1.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
INTRODUCTION TO OPTIMAL CONTROL
SYSTEMS
Harry G. Kwatny
Department of Mechanical Engineering & MechanicsDrexel University
OPTIMAL CONTROL
![Page 2: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/2.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
OUTLINEINTRODUCTION
PreliminariesBibliography
THE OPTIMAL CONTROL PROBLEMDefinitionExamples
OPTIMIZATION BASICSLocal ExtremaConstraints
VARIATIONAL CALCULUSClassical Variational CalculusFree Terminal TimeSystems with Constraints
OPTIMAL CONTROL
![Page 3: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/3.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
PRELIMINARIES
WHAT IS OPTIMAL CONTROL?
I Optimal control is an approach to control systems design thatseeks the best possible control with respect to a performancemetric.
I The theory of optimal control began to develop in the WW IIyears. The main result of this period was the Wiener-Kolmogorovtheory that addresses linear SISO systems with Gaussian noise.
I A more general theory began to emerge in the 1950’s and 60’sI In 1957 Bellman published his book on Dynamic ProgrammingI In 1960 Kalman published his multivariable generalization of
Wiener-KolmogorovI In 1962 Pontryagin et al published the maximal principleI In 1965 Isaacs published his book on differential games
OPTIMAL CONTROL
![Page 4: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/4.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
PRELIMINARIES
COURSE CONTENT
I Introduction & Preliminary ExamplesI Optimization ConceptsI Reachability, Controllability & Time Optimal ControlI The Maximum PrincipleI Principle of Optimality, Dynamic programming, & the HJB
EquationI Game Theory
OPTIMAL CONTROL
![Page 5: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/5.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
BIBLIOGRAPHY
BIBLIOGRAPHY
I E. B. Lee and L. Markus, Foundations of Optimal ControlTheory. New York: John Wiley and Sons, Inc., 1967.
I A. E. Bryson and Y.-C. Ho, Applied Optimal Control.Waltham: Blaisdell, 1969.
I S. J. Citron, Elements of Optimal Control. New York: Holt,Rinehart and Winston, Inc., 1969.
I D. E. Kirk, Optimal Control Theory: An Introduction.Englewood Cliffs, NJ: Prentice-Hall, 1970. (now availablefrom Dover)
OPTIMAL CONTROL
![Page 6: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/6.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
DEFINITION
PROBLEM DEFINITION
We will define the basic optimal control problem:I Given system dynamics
x = f (x, u) , x ∈ X ⊂ Rn, u ∈ U ⊂ Rm
I Find a control u (t) , t ∈ [0,T] that steers the system froman initial state x (0) = x0 to a target set G and minimizes thecost
J (u (·)) = gT (x (T)) +
∫ T
0g (x (t) , u (t)) dt
REMARKgT is called the terminal cost and g is the running cost. The terminaltime T can be fixed or free. The target set can be fixed or moving.
OPTIMAL CONTROL
![Page 7: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/7.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
DEFINITION
OPEN LOOP VS. CLOSED LOOP
I If we are concerned with a single specified initial state x0, thenwe might seek the optimal control control u (t), u : R→ Rm thatsteers the system from the initial state to the target. This is anopen loop control.
I On the other hand, we might seek the optimal control as afunction of the state u (x), u : Rn → Rm. This is a closed loopcontrol; sometimes called a synthesis.
I The open loop control is sometimes easier to compute, and thecomputations are sometimes performed online – a methodknown as model predictive control.
I The closed loop control has the important advantage that it isrobust with respect to model uncertainty, and that once the(sometimes difficult) computations are performed off-line, thecontrol is easily implemented online.
OPTIMAL CONTROL
![Page 8: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/8.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A MINIMUM TIME PROBLEM
Consider steering a unit mass, with bounded applied controlforce, from an arbitrary initial position and velocity to rest at theorigin in minimum time. Specifically,
x = vv = u, |u| ≤ 1
The cost function is
J =
∫ T
0dt ≡ T
REMARKThis is an example of a problem with a control constraint.
OPTIMAL CONTROL
![Page 9: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/9.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A MINIMUM FUEL PROBLEMConsider the decent of a moon lander.
h = vv = −g + u
mm = −ku
The thrust u is used to steer the system to h = 0, v = 0. Inaddition we wish to minimize the fuel used during landing, i.e.
J =
∫ t
0kudt
Furthermore, u is constrained, 0 ≤ u ≤ c, and the stateconstraint h ≥ 0 must be respected.
REMARKThis problem has both control and state constraints.
OPTIMAL CONTROL
![Page 10: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/10.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A LINEAR REGULATOR PROBLEM
Consider a system with linear dynamics
x = Ax + Bu
We seek a feedback control that steers the system from anarbitrary initial state x0 towards the origin in such a way as tominimize the cost
J = xT (T) QTx (T) +1
2T
∫ T
0
{xT (t) Qx (t) + uT (t) Ru (t)
}dt
The final time T is considered fixed.
OPTIMAL CONTROL
![Page 11: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/11.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
EXAMPLES
EXAMPLE – A ROBUST SERVO PROBLEMConsider a system with dynamics
x = Ax + B1w + B2uz = C1x + D11w + D12uy = C2x + D21w + D22u
where w is an external disturbance. The goal is to find an output (y)feedback synthesis such that the performance variables (processerrors) z remain close to zero. Note that w (t) can be characterized inseveral ways, stochastic (the H2 problem)
J = E[∫ ∞−∞
zT (t) z (t) dt]
or deterministic (the H∞ problem)
J = max‖w‖2=1
∫ ∞−∞
zT(t)z(t)dt
OPTIMAL CONTROL
![Page 12: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/12.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
LOCAL EXTREMA
DEFINITIONS
Consider the scalar function
f (x) , x ∈ Rn
which is defined and smooth on a domain D ⊂ Rn. We furtherassume that the region D is defined by a scalar inequalityψ (x) ≤ 0, i.e.,
intD = {x ∈ Rn |ψ (x) < 0}∂D = {x ∈ Rn |ψ (x) = 0}
OPTIMAL CONTROL
![Page 13: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/13.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
LOCAL EXTREMA
DEFINITIONS
DEFINITION (LOCAL MINIMA & MAXIMA)An interior point x∗ ∈ intD is a local minimum if there exists aneighborhood U of x∗ such that
f (x) ≥ f (x∗) ∀x ∈ U
It is a local maximum if
f (x) ≤ f (x∗) ∀x ∈ U
Similarly, for a point x∗ ∈ ∂D, we use a neighborhood U of x∗
within ∂D. With this modification boundary local minima andmaxima are defined as above.
OPTIMAL CONTROL
![Page 14: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/14.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
LOCAL EXTREMA
OPTIMAL INTERIOR POINTS
I Necessary conditions. A point x∗ ∈ intD is an extremalpoint (minimum or maximum) only if
∂f∂x
(x∗) = 0
I Sufficient conditions. x∗ is a minimum if
∂2f∂x2 (x∗) > 0
a maximum if∂2f∂x2 (x∗) < 0
OPTIMAL CONTROL
![Page 15: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/15.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
OPTIMIZATION WITH CONSTRAINTS – NECESSARY
CONDITIONSWe need to find extremal points of f (x), with x ∈ ∂D. i.e., findextremal points of f (x) subject to the constraint ψ (x) = 0.
Consider a more general problem where there are m constraints, i.e.,ψ : Rn → Rm.
Let λ be an m-dimensional constant vector (called Lagrangemultipliers) and define the function
H (x, λ)∆= f (x) + λTψ (x)
Then x∗ is an extremal point only if
∂H (x∗, λ)
∂x= 0,
∂H (x∗, λ)
∂λ≡ ψ (x) = 0
Note there are n + m equations in n + m unknowns x, λ
OPTIMAL CONTROL
![Page 16: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/16.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
SIGNIFICANCE OF THE LAGRANGE MULTIPLIERConsider extremal points of f (x1, x2) subject to the single constraintψ (x1, x2) = 0. At an extremal point (x∗1 , x
∗2 ) we must have
df (x∗1 , x∗2 ) =
∂f (x∗1 , x∗2 )
∂x1dx1 +
∂f (x∗1 , x∗2 )
∂x2dx2 = 0 (1)
but dx1 and dx2 are not independent. They satisfy
dψ (x∗1 , x∗2 ) =
∂ψ (x∗1 , x∗2 )
∂x1dx1 +
∂ψ (x∗1 , x∗2 )
∂x2dx2 = 0 (2)
From (1) and (2) it must be that∂f (x∗1 , x
∗2 )/∂x1
∂ψ (x∗1 , x∗2 )/∂x1
=∂f (x∗1 , x
∗2 )/∂x2
∂ψ (x∗1 , x∗2 )/∂x2
∆= −λ
Accordingly, (1) and (2) yield∂f (x∗1 , x
∗2 )
∂x1+ λ
∂ψ (x∗1 , x∗2 )
∂x1= 0,
∂f (x∗1 , x∗2 )
∂x2+ λ
∂ψ (x∗1 , x∗2 )
∂x2= 0
REMARKNote that these are the form of the previous slide.
OPTIMAL CONTROL
![Page 17: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/17.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
OPTIMIZATION WITH CONSTRAINTS – SUFFICIENT
CONDITIONS
df =∂H∂x
dx +12
dxT ∂2H∂x2 dx− λTdψ + h.o.t.
butdψ =
∂ψ
∂xdx = 0⇒ dx ∈ ker
∂ψ
∂x⇒ dx = Ψdα
Ψ = span ker∂ψ (x)
∂xThus, for x∗ extremal
df (x∗) =12
dαTΨT ∂2H (x∗)∂x2 Ψdα+ h.o.t.
ΨT ∂2H (x∗)∂x2 Ψ > 0⇒ min
ΨT ∂2H (x∗)∂x2 Ψ < 0⇒ max
OPTIMAL CONTROL
![Page 18: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/18.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CONSTRAINTS
EXAMPLE
f (x1, x2) = x1
(x2
1 + 2x22 − 1
), ψ (x1, x2) = x2
1 + x22 − 1
Interior:(x1, x2, f ) = (0,−0.707, 0) ∨ (0, 0.707, 0) ∨ (−0.577, 0, 0.385) ∨ (0.577, 0.− 0.385)
Boundary:
(x1, x2, λ, f ) = (−0.577,−0.8165, 1.155,−0.385) ∨ (−0.577, 0.8165, 1.155,−0.385)∨ (0.577,−0.8165,−1.155, 0.385) ∨ (0.577, 0.8165,−1.155, 0.385)∨ (−1, 0, 1, 0) ∨ (1, 0,−1, 0)
OPTIMAL CONTROL
![Page 19: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/19.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
OPTIMIZING A TIME TRAJECTORY
I We are interested in steering a controllable system along atrajectory that is optimal in some sense.
I Three methods are commonly used to address suchproblems:
I The ‘calculus of variations’I The Pontryagin ‘maximal Principle’I The ‘principle of optimality’ and dynamic programming
I The calculus of variations was first invented to characterizethe dynamical behavior of physical systems governed by aconservation law.
OPTIMAL CONTROL
![Page 20: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/20.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS: LAGRANGIAN SYSTEMSA Lagrangian System is characterized as follows:
I The system is define in terms of a vector of configurationcoordinates, q, associated with velocities q.
I The system has kinetic energy T (q, q) = qTM (q) q/2, andpotential energy V (q) from which we define the Lagrangian
L (q, q) = T (q, q)− V (q)
I The system moves along a trajectory q (t), between initialand final times t1, t2 in such a way as to minimize theintegral
J (q (t)) =
∫ t2
t1L (q (t) , q (t)) dt
OPTIMAL CONTROL
![Page 21: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/21.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
EXAMPLES OF LAGRANGIAN SYSTEMS
T = 12 qTM (q) q
M (q) =(`2
1 (m1 + m2) + `22m2 + 2`1`2m2 cos θ2 `2m2 (`2 + `1 cos θ2)
`2m2 (`2 + `1 cos θ2) `22m2
)V (q) = m1g (g`1 (m1 + m2) sin θ1 + g`2m2 sin (θ1 + θ2))
m1
m2
�1
� 2
1θ
2θ
OPTIMAL CONTROL
![Page 22: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/22.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS –1: NECESSARY
CONDITIONSI A real-valued, continuously differentiable function q (t) on the
interval [t1, t2] will be called admissible.I Let q∗ (t) be an optimal admissible trajectory and q (t, ε) a not
necessarily optimal trajectory, with
q (t, ε) = q∗ (t) + εη (t)
where ε > 0 is a small parameter and η (t) is arbitrary.I Then
J (q (t, ε)) =
∫ t2
t1L (q∗ (t) + εη (t) , q∗ (t) + εη (t)) dt
I An extremal of J is obtained from
δJ (q (t)) =∂J (q (t, ε))
∂ε
∣∣∣∣ε=0
= 0
OPTIMAL CONTROL
![Page 23: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/23.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS – 2
δJ (q (t)) =
∫ t2
t1
(∂L (q∗ (t) , q∗ (t))
∂q∗η (t) +
∂L (q∗ (t) , q∗ (t))q∗
η (t))
dt
δJ ⇒∫∂L∂q
dη +
∫∂L∂q
dt
We can apply the ‘integration by parts’ formula∫udv = uv−
∫vdu
to the first term to obtain
δJ (q (t)) =∫ t2
t1
(− d
dt∂L(q∗(t),q∗(t))
∂q∗ + ∂L(q∗(t),q∗(t))q∗
)η (t) dt
+ ∂L(q∗(t),q∗(t))∂q∗ η (t)
∣∣∣t2t1
OPTIMAL CONTROL
![Page 24: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/24.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
CLASSICAL VARIATIONAL CALCULUS
CALCULUS OF VARIATIONS – 3Now, set δJ (q (t)) = 0 to obtain:
I the Euler-Lagrange equations
ddt∂L (q∗ (t) , q∗ (t))
∂q∗− ∂L (q∗ (t) , q∗ (t))
q∗= 0
I the transversality (boundary) conditions
∂L (q∗ (t1) , q∗ (t1))
∂q∗δq (t1) = 0,
∂L (q∗ (t2) , q∗ (t2))
∂q∗δq (t2) = 0
REMARKThese results allow us to treat problems in which the initial and terminal times are fixedand individual components of q(t1) and q(t2) are fixed or free. Other cases of interestinclude: 1) the terminal time is free, and 2) the terminal time is related to the terminalconfiguration, e.g., by a relation ϕ (q (t2) , t2) = 0.
OPTIMAL CONTROL
![Page 25: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/25.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
FREE TERMINAL TIME, 1Consider the case of fixed initial state and free terminal time. Let q∗(t) be theoptimal trajectory with optimal terminal time t∗2 . The perturbed trajectoryterminates at time t∗2 + δt2. Its end state is q∗ (t∗2 ) + εη (t∗2 + ετ). Theperturbed cost is
J (q∗, δq, δt) =∫ t∗2 +δt
t1
L (q∗ (t) + δq (t) , q∗ (t) + δq (t)) dt
From this we obtain:
δJ (q∗ (t)) =∫ t∗2
t1
(− d
dt∂L(q∗(t),q∗(t))
∂q∗ +∂L(q∗(t),q∗(t))
q∗
)δq (t) dt
+∂L(q∗(t∗2 ),q
∗(t∗2 ))∂q∗ δq (t∗2 ) + L (q∗ (t∗2 ) , q
∗ (t∗2 )) δt
Now, we want to allow both the final time and the end point to vary. Theactual end state is:
δq2∆= δq (t∗2 + δt) = δq (t∗2 ) + q∗ (t∗2 ) δt
OPTIMAL CONTROL
![Page 26: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/26.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
FREE TERMINAL TIME, 2
δJ (q∗ (t)) =∫ t∗2
t1
(− d
dt∂L(q∗(t),q∗(t))
∂q∗ +∂L(q∗(t),q∗(t))
q∗
)δq (t) dt
+∂L(q∗(t∗2 ),q
∗(t∗2 ))∂q∗ δq2 +
(L (q∗ (t∗2 ) , q
∗ (t∗2 ))−∂L(q∗(t∗2 ),q
∗(t∗2 ))∂q∗ q∗ (t∗2 )
)δt
Thus, we have have the Euler-Lagrange Equations, as before, but thetransversality conditions become:
I the previous conditions:∂L (q∗ (t1) , q∗ (t1))
∂q∗δq (t1) = 0,
∂L (q∗ (t∗2 ) , q∗ (t∗2 ))
∂q∗δq (t∗2 ) = 0
I plus, additional conditions:∂L (q∗ (t∗2 ) , q
∗ (t∗2 ))∂q∗
δq (t∗2 ) = 0(L (q∗ (t∗2 ) , q
∗ (t∗2 ))−∂L (q∗ (t∗2 ) , q
∗ (t∗2 ))∂q∗
q∗ (t∗2 ))δt = 0
OPTIMAL CONTROL
![Page 27: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/27.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
FREE TERMINAL TIME, 3
Ordinarily, the initial state and time are fixed so that the transversalityconditions require
δq (t1) = 0
For free terminal time δt is arbitrary. Thus from the transversality conditions:
1. If the terminal state is fixed, δq (t2) = 0 the terminal condition is:
L (q∗ (t∗2 ) , q∗ (t∗2 ))−
∂L (q∗ (t∗2 ) , q∗ (t∗2 ))
∂q∗q∗ (t∗2 ) = 0
2. If the terminal state is free, δq2 arbitrary the terminal condition is:
∂L (q∗ (t∗2 ) , q∗ (t∗2 ))
∂q∗= 0, L (q∗ (t∗2 ) , q
∗ (t∗2 )) = 0
OPTIMAL CONTROL
![Page 28: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/28.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1: FREE ENDPOINT, FIXED TERMINAL TIMEFirst note that in the elementary variational calculus, we couldsimply replace q by x, and add the definition x = u, so that thecost function becomes
J (x (t)) =
∫ t2
t1L (u (t) , x (t)) dt
Hence, we have a simple control problem. Now suppose x ∈ R,t1 = 0, t2 = 1, and L =
(x2 + u2
)/2:
J (x (t)) =
∫ 1
0
12(x2 + u2) dt
The Euler-Lagrange equations become simplyddt
(∂L∂u
)− ∂L∂x
= 0⇒ u− x = 0
OPTIMAL CONTROL
![Page 29: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/29.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1, CONT’DThus, we have the equations[
xu
]=
[0 11 0
] [xu
]
x (0) = x0,∂L (x (1) , u (1))
∂u= 0⇒ u (1) = 0
Consequently,
[x (t)u (t)
]= e
0 11 0
t [ab
]⇒[
x (t)u (t)
]=
[a cosh (t) + b sinh (t)b cosh (t) + a sinh (t)
]
a = x0, b =x0 − e2x0
1 + e2
OPTIMAL CONTROL
![Page 30: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/30.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1: FIXED ENDPOINT, FIXED TERMINAL TIME
Suppose we consider a fixed end point, say x (t2) = 0 with theterminal time still fixed at t2 = 1. Then the Euler-Lagrangeequations remain the same, but the boundary conditionschange to:
x (0) = x0, x (1) = 0
Thus, we compute
a = x0, b = −x0 + e2x0
−1 + e2
OPTIMAL CONTROL
![Page 31: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/31.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1: FIXED ENDPOINT, FREE TERMINAL TIME
Suppose we consider a fixed end point, say x (t2) = xf with theterminal time t2 free. Then the Euler-Lagrange equationsremain the same, but the boundary conditions change to:
x (0) = x0, x (t2) = xf , (L− Luu)|t2 = 0
From this we compute
a = x0, a2 = b2, a cosh (t2) + b sinh (t2) = xf
OPTIMAL CONTROL
![Page 32: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/32.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
FREE TERMINAL TIME
EXAMPLE 1, CONT’D
The figures below show the optimal control and statetrajectories from the initial state x0 = 1,xf = 0, with t2 = 1 for thefixed time case. For the free time case xf = 0.01.
Free terminal state, Fixedtime:
0.2 0.4 0.6 0.8 1.0t
-0.5
0.5
1.0
x,u
Fixed terminal state,Fixed Time:
0.2 0.4 0.6 0.8 1.0t
-1.0
-0.5
0.5
1.0
x,u
Fixed terminal state,Free time:
1 2 3 4 5t
-1.0
-0.5
0.5
1.0
x,u
REMARKIn the free time case, with xf = 0.01, the final time is t2 = 4.60517. With xf = 0 the finaltime is t2 =∞. Note that the case of free terminal state and free terminal time (notshown) is trivial with t2 = 0.
OPTIMAL CONTROL
![Page 33: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/33.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
VARIATIONAL CALCULUS WITH DIFFERENTIAL
CONSTRAINTS
Systems with ‘nonintegrable’ differential , i.e., nonholonomic,constraints have been treated by variational methods. Asbefore, we seek extremals of the functional:
J (q (t)) =
∫ t2
t1L (q (t) , q (t) , t) dt
But now subject to the constraints:
ϕ (q (t) , q (t) , t) = 0
where ϕ : Rn × Rn × R→ Rm.
OPTIMAL CONTROL
![Page 34: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/34.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
DIFFERENTIAL CONSTRAINTS, CONT’D
The constraints are enforce for all time, so introduce the mLagrange multipliers λ (t) and consider the modified functional
J (q (t)) =
∫ t2
t1L (q (t) , q (t) , t) + λT (t)ϕ (q (t) , q (t) , t) dt
We allow variations in both q and λ and t2, Taking the variationand integrating by parts yields
δJ =∫ t2
t1
({− d
dt
[Lq + λTϕq
]+[Lq + λTϕq
]}δq (t) + ϕTδλ
)dt
+[Lq + λTϕq
]t2δq2 +
([L + λTϕq
]−[Lq + λTϕq
]q)
t2δt2
OPTIMAL CONTROL
![Page 35: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/35.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
NECESSARY CONDITIONS WITH CONSTRAINTSAgain we have the Euler-Lagrange equations, now in the form:
ddt
[Lq + λTϕq
]−[Lq + λTϕq
]= 0
The differential constraints:
ϕ (q, q, t) = 0
and the boundary conditions[Lq + λTϕq
]t2δq2 = 0
With free terminal time, we also have([L + λTϕq
]−[Lq + λTϕq
]q)
t2= 0
OPTIMAL CONTROL
![Page 36: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/36.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLEOnce again consider the problem
x = u, J (x (t)) =∫ t2
0
12
(x2 + u2
)dt
But now, add the constraint
ϕ (x, x) = x2 − 1 ≡ u2 − 1 = 0→ u = ±1
The Euler-Lagrange equations now become
ddt
[Lu + λTϕu
]−[Lx + λTϕx
]= 0⇒ λ =
x2u
REMARKI Note that u = ±1→ u = 0 almost everywhere,I Also,
∫ t0 u2dt = t
OPTIMAL CONTROL
![Page 37: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/37.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE CONTINUED
We now consider three cases:I fixed terminal time, fixed end pointI fixed terminal time, free end pointI free terminal time fixed end point
OPTIMAL CONTROL
![Page 38: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/38.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FIXED TERMINAL TIME, FIXED END POINT
Steer from x (0) = x0 to x (1) = x1.
I system x = u
I Euler equation λ = x2u
I constraint u = ±1
I boundary conditions x (0) = x0, x (1) = x1
Assume single switch at time T
x0 + kT − k(1− T) = x1 ∧ (k = ±1) ∧ 0 ≤ T ≤ 1⇒ −1 + x1 ≤ x0 ≤ 1 + x1 ∧ (k = ±1) ∧ T = k−x0+x1
2k
OPTIMAL CONTROL
![Page 39: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/39.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FIXED TERMINAL TIME, FREE END POINT
Steer from x (0) = x0.
I system x = u
I Euler equation λ = x2u
I constraint u = ±1
I initial condition x (0) = x0
I terminal condition x (1) = x1
I terminal condition[Lu + λTϕu
]t2= 0⇒ λ (t2) = − 1
2
The necessary conditions are satisfied with
(k = ±1) ∧(−3
4≤ λ0 ≤ −
12
)∧(
T =12−√
1− 2λ0√2
)
OPTIMAL CONTROL
![Page 40: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/40.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FIXED TERMINAL TIME, FREE END POINT–2
Only the case λ0 = − 12 ,T = 1
2 is optimal.
λ0 = −12
:
0.0 0.2 0.4 0.6 0.8 1.0-1.0
-0.5
0.0
0.5
t
x,Λ,J
λ0 = −3
4
:
0.0 0.2 0.4 0.6 0.8 1.0-1.0
-0.5
0.0
0.5
t
x,Λ,J
OPTIMAL CONTROL
![Page 41: OptimalControl_1](https://reader037.vdocument.in/reader037/viewer/2022110100/55cf96ea550346d0338e9d90/html5/thumbnails/41.jpg)
INTRODUCTION THE OPTIMAL CONTROL PROBLEM OPTIMIZATION BASICS VARIATIONAL CALCULUS
SYSTEMS WITH CONSTRAINTS
EXAMPLE: FREE TERMINAL TIME, FIXED END POINT
Steer from x (0) = x0 to x (t2) = 0I system x = uI Euler equation λ = x
2uI constraint u = ±1I initial condition x (0) = x0
I terminal condition x (t2) = 0I terminal condition([
L + λTϕq]−[Lu + λTϕu
]u)
t2= 1
2 − 1− 2λ (t2) = 0⇒ λ (t2) = − 14
(x0 ≥ 0) ∧ (k = −1) ∧ (t2 = x0) ∧(λ0 =
14(1 + x2
0
))∨
(x0 ≤ 0) ∧ (k = +1) ∧ (t2 = −x0) ∧(λ0 =
14(1 + x2
0
))
OPTIMAL CONTROL