pseudo spectral methods
TRANSCRIPT
A Primer on Pseudospectral Methods for
Solving Optimal Control Problems
Anil V. Rao
Gainesville, FL 32607
12 – 15 March 2012
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Preliminaries
This Primer consists of material that has been taken exclusively from the
following documents available in the public domain:
• The Following Textbooks on Optimal Control:
⊲ Kirk, D. E., Optimal Control Theory: An Introduction, Dover Publications,
Mineola, New York, 2004.
⊲ Athans, M. A. and Falb, P. E., Optimal Control: An Introduction to the
Theory and Its Applications, Dover Publications, Mineola, New York, 2006.
⊲ Bryson, A. E. and Ho, Y-C, Applied Optimal Control, Hemisphere Publishing,
New York, 1975.
⊲ Dr. Anil V. Rao’s Lecture Notes from University of Florida Course EML
6934. URL: http://vdol.mae.ufl.edu.
⊲ Various Journal and Conference Articles on Pseudospectral Methods
Co-Authored by Dr. Anil V. Rao. URL: http://vdol.mae.ufl.edu.
⊲ Benson, D. A., A Gauss Pseudospectral Transcription for Optimal Control,
PhD Thesis, Department of Aeronautics and Astronautics, MIT, November
2004.
Copyright © 2006–2012 Anil Vithala Rao Page 2
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Huntington, G. T., Advancement and Analysis of a Gauss Pseudospectral
Transcription for Optimal Control, PhD Thesis, Department of Aeronautics
and Astronautics, MIT, May 2007.
⊲ Darby, C. L., hp Pseudospectral Methods for Solving Nonlinear Optimal
Control Problems, PhD Thesis, Department of Mechanical and Aerospace
Engineering, University of Florida, April 2011.
⊲ Garg, D., Advancements in Global Pseudospectral Methods for Optimal
Control, PhD Thesis, Department of Mechanical and Aerospace Engineering,
University of Florida, June 2011.
Copyright © 2006–2012 Anil Vithala Rao Page 3
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Course Schedule
• Day 1
⊲ Part I: Calculus of Variations (1.5 Hour)
⊲ Part II: Optimal Control Theory (1.5 Hour)
⊲ Part III: Nonlinear Optimization (1.5 Hours)
⊲ Part IV: Numerical Methods for Optimal Control (1.5 Hours)
• Day 2
⊲ Part V: Numerical Integration and Interplation (1 Hour)
⊲ Part VI: Foundation of Pseudospectral Methods (1 Hours)
⊲ Part VII: Implementation of a Pseudospectral Method (2 Hours)
⊲ Part VIII: Wrap-Up (1 Hour)
Copyright © 2006–2012 Anil Vithala Rao Page 4
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Ingredients for Solving an Optimal Control Problem
Numerical Methods forSolving Differential Equations
ConstrainedNonlinear Optimization
Systems of NonlinearAlgebraic Equations
DirectMethods
IndirectMethods
• Indirect Methods
⊲ First-Order Optimality Conditions of Continuous Problem Derived
⊲ Need to Solve Systems of Nonlinear Equations
⊲ Becomes a Root-Finding Problem
• Direct Methods
⊲ Optimal Control Problem Approximated in an Appropriate Manner
⊲ Becomes a Nonlinear Programming Problem
⊲ Problem Solved Using Techniques from Nonlinear Optimization
• Both Indirect and Direct Methods: Need to Solve Differential Equations
Copyright © 2006–2012 Anil Vithala Rao Page 5
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part I: Calculus of Variations
Copyright © 2006–2012 Anil Vithala Rao Page 6
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Functions and Functionals
Definition 1
A function f(q) is a rule or correspondence that assigns to each element q ∈ D
an element f(q) ∈ R. The quantities D and R are called the domain and range
of the function, respectively.
Definition 2
A functional J(x) is a rule or correspondence that assigns to each function
x ∈ Ω an element J(x) ∈ R. The quantity Ω is the class of functions over which
the functional is defined and R is the set of real numbers.
Linearity of Functions
A function is said to be linear if the following property is satisfied ∀ p,q ∈ D
and ∀ α, β ∈ R:
f(αp+ βq) = αf(p) + βf(q)
Copyright © 2006–2012 Anil Vithala Rao Page 7
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Linearity of Functionals
A functional is said to be linear if the following property is satisfied ∀ x,y ∈ Ω
and ∀ α, β ∈ R:
J(αx+ βy) = αJ(x) + βJ(y)
Norms
The norm in Rn is a rule or correspondence that assigns to each point q ∈ D a
real number. The norm of q, denoted ‖q‖, satisfies the following properties
1. ‖q‖ ≥ 0, ∀ q ∈ D (q 6= 0)
2. ‖q‖ = 0 ⇔ q = 0
3. ‖αq‖ = |α|‖q‖, ∀ α ∈ R
4. ‖p+ q‖ ≤ ‖p‖+ ‖q‖, ∀ p,q ∈ D
Copyright © 2006–2012 Anil Vithala Rao Page 8
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Norm of a Function
The norm of a function is a rule or correspondence that assigns to each
function x ∈ Ω (where x is defined on t ∈ [t0, tf ]) a real number. The norm of
x, denoted ‖x‖, satisfies the following properties
1. ‖x‖ ≥ 0, ∀ x ∈ Ω (x 6= 0)
2. ‖x‖ = 0 ⇔ x ≡ 0 on t ∈ [t0, tf ]
3. ‖αx‖ = |α|‖x‖, ∀ α ∈ R
4. ‖x+ y‖ ≤ ‖x‖+ ‖y‖, ∀ x,y ∈ D
Increment of a Function
Let f be a function and let q,∆q ∈ D. Then the increment of the function
from q to q+∆q, denoted ∆f , is defined as
∆f = f(q+∆q)− f(q)
It is noted that ∆f depends upon both q and ∆q, i.e., ∆f = ∆f(q,∆q)
Copyright © 2006–2012 Anil Vithala Rao Page 9
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Increment of a Functional
Let J be a functional and let x, δx ∈ Ω. Then the increment of the functional J
from x to x+ δx, denoted ∆J , is defined as
∆J = J(x+ δx)− J(x)
It is noted that δJ depends upon both x and δx, i.e., ∆J = ∆J(x, δx)
Differential of a Function
The general form for the increment of a function is given as
∆f(q,∆q) = df(q,∆q) + g(q,∆q)‖∆q‖
The quantity df(q,∆q) is called the differential of the function. The
differential of a function is a linear operator on ∆q, i.e.,
df(q, αp+ βr) = αdf(q,p) + βdf(q, r)
Variation of a Functional
The general form for the increment of a functional is given as
∆J(x, δx) = δJ(x, δx) + g(x, δx)‖δx‖
Copyright © 2006–2012 Anil Vithala Rao Page 10
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
The quantity δJ(x, δx) is called the variation of the functional. The variation
of a functional is a linear operator on δx, i.e.,
δJ(x, αy + βz) = αδJ(x,y) + βδJ(x, z)
Finally, the quantity δx is called the variation of the function x.
Copyright © 2006–2012 Anil Vithala Rao Page 11
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Minima and Maxima of Functions and Functionals
Definition 3
A function f with domain D is said to have a local extremum at the point q∗ if
∃ ǫ > 0 such that ∀ q ∈ D such that ‖q− q∗‖ < ǫ, the increment ∆f has the
same sign. If
∆f = f(q)− f(q∗) ≥ 0
then f(q∗) is said to be a local minimum of f . If
∆f = f(q)− f(q∗) ≤ 0
then f(q∗) is said to be a local maximum of f . If either condition is satisfied for
arbitrarily large ǫ, then f(q∗) is said to be a global minimum or global
maximum.
Definition 4
A functional J with domain Ω is said to have a local extremum at the function
x∗ if ∃ ǫ > 0 such that ∀ x ∈ Ω such that ‖x− x∗‖ < ǫ, the increment ∆J has
the same sign. If
∆J = J(x)− J(x∗) ≥ 0
Copyright © 2006–2012 Anil Vithala Rao Page 12
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
then J(x∗) is said to be a local minimum of J . If
∆J = J(x)− J(x∗) ≤ 0
then J(x∗) is said to be a local maximum of J . If either condition is satisfied
for arbitrarily large ǫ, then J(x∗) is said to be a global minimum or global
maximum.
Copyright © 2006–2012 Anil Vithala Rao Page 13
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Basic Problem
• Solve Basic Calculus of Variations Problem Using Variations
• Specifically, Will Take the First Variation of J , denoted δJ
• Will Set δJ = 0
δJ = δ
∫ tf
t0
L[x(t), x(t), t]dt
• Question: Which Quantities Vary (and Which Do Not Vary)?
⊲ Quantities x(t) and x(t) Vary
⊲ However, t Does Not Vary Because It is a Dummy Variable of
Integration
• Computation of δJ
δJ = δ
∫ tf
t0
L[x(t), x(t), t]dt
• Because t0 and tf are Fixed, We Have
δJ =
∫ tf
t0
δL[x(t), x(t), t]dt
Copyright © 2006–2012 Anil Vithala Rao Page 14
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Also, Because x(t) and x(t) Vary on t ∈ [t0, tf ], We Have
δL =∂L
∂xδx+
∂L
∂xδx
• We Then Obtain
δJ =
∫ tf
t0
[
∂L
∂xδx+
∂L
∂xδx
]
dt
• Applying Integration By Parts on Second Term,
∫ tf
t0
∂L
∂xδxdt =
[
∂L
∂xδx
]tf
t0
−
∫ tf
t0
d
dt
(
∂L
∂x
)
δxdt
• Now Utilize the Following Information:
⊲ Initial and Terminal State is Fixed −→ δx0 = δxf = 0
⊲ Initial and Terminal Time is Fixed −→ δt0 = δtf = 0
⊲ Furthermore,
δx0 = δx(t0) + x(t0)δt0
δxf = δx(tf ) + x(t0)δtf
Copyright © 2006–2012 Anil Vithala Rao Page 15
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Therefore, δx(t0) = δx(tf ) = 0 Which Implies That
[
∂L
∂xδx
]tf
t0
=
(
∂L
∂x
)
tf
δx(tf )−
(
∂L
∂x
)
t0
δx(t0) = 0
• Consequently, We Obtain
δJ =
∫ tf
t0
[
∂L
∂xδx−
d
dt
(
∂L
∂x
)
δx
]
dt
• Last Equation Can Be Rewritten As
δJ =
∫ tf
t0
[
∂L
∂x−d
dt
(
∂L
∂x
)]
δxdt
• Because δJ Must be Zero for All Variations,
∂L
∂x−d
dt
(
∂L
∂x
)
= 0 −→ Euler-Lagrange Equation
Copyright © 2006–2012 Anil Vithala Rao Page 16
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Calculus of Variations Example
• Find an Extremal for the Functional
J [x(t)] =
∫ tf
t0
[
x2(t)− x2(t)]
dt, x(0) = 0, x(π/2) = 1
• Solution: Euler-Lagrange Equation Given as
∂L∂x
− d
dt
(
∂L∂x
)
= −2x− 2x = 0 =⇒ x+ x = 0
• General Solution of Differential Equation:
x(t) = c1 cos t+ c2 sin t
• Applying Boundary Conditions, We Obtain c0 = 0 and c1 = 1. Therefore,
x∗(t) = sin t
Copyright © 2006–2012 Anil Vithala Rao Page 17
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
More General Calculus of Variations Problems
• Starting Point for Functional Optimization
• More General Problems: t0, tf , x(t0), and x(tf ) May Be Free
• δx0 and δxf : Variation in Initial and Final value of x
• δx(t0) and δx(tf ): δx Evaluated at Initial and Terminal time
• Specifically, δx0 = δx(t0) + x(t0)δt0 and δxf = δx(tf ) + x(tf )δtf
Copyright © 2006–2012 Anil Vithala Rao Page 18
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Free Final Time Functional Optimization Problem
• Consider Following Problem. Minimize
J [x(t)] ≡ J =
∫ tf
t0
L[x(t), x(t), t]dt
• Boundary Conditions:
t0 = fixed , tf = free
x(t0) = fixed , x(tf ) = fixed
Copyright © 2006–2012 Anil Vithala Rao Page 19
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Solution of Free Final Time COV Problem
• Taking δJ Gives
δJ = δ
∫ tf
t0
L[x(t), x(t), t]dt
• Because tf is Free −→ δtf 6= 0. Therefore,
δJ =∂J
∂tfδtf +
∫ tf
t0
δL[x(t), x(t), t]dt
• Applying the Fundamental Theorem of Calculus, We Have
∂J
∂tf= L[x(tf ), x(tf ), tf ] =⇒ δJ = L[x(tf ), x(tf ), tf ]δtf +
∫ tf
t0
δL[x(t), x(t), t]dt
• Already Have Term Inside Integral from Earlier. Therefore,
δJ = L[x(tf ), x(tf ), tf ]δtf +
(
∂L∂x
)
tf
δx(tf )−(
∂L∂x
)
t0
δx(t0)
+
∫ tf
t0
[
∂L∂x
− d
dt
(
∂L∂x
)]
δxdt
Copyright © 2006–2012 Anil Vithala Rao Page 20
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Now t0 is Fixed −→ δt0 = 0. Therefore,
δJ = L[x(tf ), x(tf ), tf ]δtf +
(
∂L∂x
)
tf
δx(tf )
+
∫ tf
t0
[
∂L∂x
− d
dt
(
∂L∂x
)]
δxdt
• Now the Final Value of x is Fixed. Therefore, δxf = 0 Which Implies That
δxf = δx(tf ) + x(tf )δtf = 0 =⇒ δx(tf ) = −x(tf )δtf
• Therefore,
δJ = L[x(tf ), x(tf ), tf ]δtf −(
∂L∂x
)
tf
x(tf )δtf
+
∫ tf
t0
[
∂L∂x
− d
dt
(
∂L∂x
)]
δxdt
Copyright © 2006–2012 Anil Vithala Rao Page 21
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Combining Terms, We Obtain
δJ =
L[x(tf ), x(tf ), tf ]−(
∂L∂x
)
tf
x(tf )
δtf
+
∫ tf
t0
[
∂L∂x
− d
dt
(
∂L∂x
)]
δxdt
• Now, Because δx and δtf are Independent, We have
L[x(tf ), x(tf ), tf ]−(
∂L∂x
)
tf
x(tf ) = 0
∂L∂x
− d
dt
(
∂L∂x
)
= 0
Copyright © 2006–2012 Anil Vithala Rao Page 22
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example with Free Final Time
• Minimize
J [x(t)] =
∫ tf
1
[
2x(t) +1
2x2(t)
]
dt, x(1) = 4, x(tf ) = 4. tf > 1
• Solution:
⊲ Euler-Lagrange Equation: x = 2 =⇒ x(t) = t2 + c1t+ c2
⊲ Boundary Condition 2x(tf )−12 x
2(tf ) = 0
⊲ Applying Boundary Condition Gives
x(1) = 4 = 1 + c1 + c2 =⇒ c1 + c2 = 3
2x(tf )− 12x2(tf ) = 2(t2f + c1tf + c2) + 2tf + c1 = 0
⊲ Final Solution
x∗(t) = t2 − 6t+ 9
tf = 5
Copyright © 2006–2012 Anil Vithala Rao Page 23
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Free Final Endpoint Functional Optimization Problem
• Consider Following Problem. Minimize
J [x(t)] ≡ J =
∫ tf
t0
L[x(t), x(t), t]dt
• Boundary Conditions
t0 = fixed , tf = fixed
x(t0) = fixed , x(tf ) = free
Copyright © 2006–2012 Anil Vithala Rao Page 24
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Minimization of Functionals of Vector Functions
• In General, Functionals May be Functions of Vector Functions
• Functional of a Vector Function: Let x(t) ∈ Rn,
x(t) =
x1(t)
x2(t)
...
xn(t)
• Functional of x(t)
J [x(t)] =
∫ tf
t0
L[x(t), x(t), t]dt
• Extremization of Functional of Vector Function
δJ = δ
∫ tf
t0
L[x(t), x(t), t]dt = 0
• Solved in a Manner Analogous to Function Optimization
Copyright © 2006–2012 Anil Vithala Rao Page 25
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Assume That t0, tf , x(t0), and x(tf ) Are Fixed
=⇒ δJ =
∫ tf
t0
δL[x(t), x(t), t]dt = 0
• Consequently,
δJ =
∫ tf
t0
[
(
∂L∂x
)T
δx+
(
∂L∂x
)T
δx
]
dt
• Integrating Second Term of Integrand By Parts Gives
δJ =
[
(
∂L∂x
)T
δx
]tf
t0
+
∫ tf
t0
[
(
∂L∂x
)T
− d
dt
(
∂L∂x
)T]
δxdt
• Applying Boundary Conditions ⇒ δx(t0) = δx(tf ) = 0. Therefore,
∂L∂x
− d
dt
(
∂L∂x
)
= 0 =⇒ Vector Euler-Lagrange Equation
• In Scalar Form∂L∂xi
− d
dt
(
∂L∂xi
)
= 0, (i = 1, . . . , n)
• Extremization of More General Functionals Proceeds in Analogous Manner
Copyright © 2006–2012 Anil Vithala Rao Page 26
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Summary
• Calculus of Variations
⊲ Foundation for Functional Optimization
⊲ Deals with Continuous-Time (Infinite-Dimensional) Problem
⊲ Candidate Optimal Solutions are Called Extremals
• Determination of an Extremal Solution
⊲ Compute First Variation of Cost Functional
⊲ Set All Independent Terms in Variation to Zero
• Importance of Calculus of Variations
⊲ Foundation for Optimal Control Theory
⊲ Optimal Control Problem: Application of Variational Calculus
Copyright © 2006–2012 Anil Vithala Rao Page 27
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part II: Optimal Control Theory
Copyright © 2006–2012 Anil Vithala Rao Page 28
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
General Optimal Control Problem
• Determine x(t), u(t), and p that Minimize the Cost Functional
J = Φ(x(tf ), tf ,p) +
∫ tf
t0
L[x(t),u(t),p, t]dt
• Subject to the Dynamic Constraint
x = f(x,u,p, t)
• The Boundary Conditions
φ(x(t0), t0,x(tf ), tf ,p) = 0
• And the Path Constraints
Cmin ≤ C(x(t),u(t),p) ≤ Cmax
Copyright © 2006–2012 Anil Vithala Rao Page 29
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Key Features of Optimal Control Problem
• Functional Minimization Problem
⊲ x(t) and u(t) are Functions of Time
⊲ J is a Function of Functions x(t) & u(t) −→ J is a Functional
• Optimal Control Problems are Constrained
• Constraints: Differential Equations, Boundary Conditions, & Path
• Need to Solve a Constrained Functional Minimization Problem
• Broad Classes of Optimal Control Problems
⊲ Free and Fixed Terminal Times
⊲ Free Terminal State (Initial-Value Problem)
⊲ Constrained Terminal State (Boundary-Value Problem)
⊲ Path Constrained and Path Unconstrained
Copyright © 2006–2012 Anil Vithala Rao Page 30
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Conventions Used in Derivation
• Let f : Rn −→ R. The Gradient and Hessian of f are Defined As
∂f
∂x=
[
∂f
∂x1
∂f
∂x2· · · ∂f
∂xn
]
∂2f
∂x2=
∂2f
∂x21
∂2f
∂x2x1· · · ∂2f
∂xnx1∂2f
∂x1x2
∂2f
∂x22· · · ∂2f
∂xnx1...
.... . .
...
∂2f
∂x1xn
∂2f
∂x2xn· · · ∂2f
∂x2n
• Using Conventions,We Have
∂2f
∂x2=
∂
∂x
[
∂f
∂x
]T
• Let f : Rn −→ Rm. Then the Jacobian of f(x) is an m× n matrix, i.e.,
∂f
∂x=
[
∂f
∂x1
∂f
∂x2· · · ∂f
∂xn
]
∈ Rm×n
Copyright © 2006–2012 Anil Vithala Rao Page 31
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Optimal Control Problem without Path Constraints
• Construct Augmented Cost Functional
Ja = Φ− νTφ+
∫ tf
t0
[
L+ λT(f − x)]
dt
where
λ(t) = Costate
ν = Lagrange Multiplier of Boundary Condition
• Make Following Assumptions (For Now)
⊲ Initial and Terminal Times are Free
⊲ Control is Unconstrained
⊲ No Path Constraints
Copyright © 2006–2012 Anil Vithala Rao Page 32
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
First-Order Optimality Conditions
• Augmented Cost Functional
Ja = Φ− νTφ+
∫ tf
t0
[
L+ λT(f − x)]
dt
• Terminology
⊲ ν: Lagrange Multiplier of Boundary Condition
⊲ λ: Costate or Adjoint of Differential Equation
• Define
H = L+ λTf = H(x,λ,u)
• The Quantity H is called the Hamiltonian
• In Terms of the Hamiltonian We Have
Ja = Φ− νTφ+
∫ tf
t0
[
H− λTx]
dt
• First Variation of Augmented Cost Functional
δJa = δΦ− δ
νTφ+
∫ tf
t0
[
H− λTx]
dt
Copyright © 2006–2012 Anil Vithala Rao Page 33
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
which gives
δJa = δΦ− δνTφ− νTδφ+ δ
∫ tf
t0
[
H− λTx]
dt
• Now We Have
δΦ =∂Φ
∂x(t0)δx(t0) +
[
∂Φ
∂x(t0)x(t0) +
∂Φ
∂t0
]
δt0
+∂Φ
∂x(tf )δx(tf ) +
[
∂Φ
∂x(tf )x(tf ) +
∂Φ
∂tf
]
δtf
• Combining Terms, We Have
δΦ =∂Φ
∂x(t0)[δx(t0) + x(t0)δt0] +
∂Φ
∂t0δt0
+∂Φ
∂x(tf )[δx(tf ) + x(tf )δtf ] +
∂Φ
∂tfδtf
Copyright © 2006–2012 Anil Vithala Rao Page 34
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Terminology
δx0, δxf = Variations in Initial and Terminal State
δx(t0), δx(tf ) = Variations in State at Initial
and Terminal Times
• Note That δx(t0) 6= δx0 and δx(tf ) 6= δxf
• Relationships Between δx0, δxf and δx(t0), δx(tf )
δx0 = δx(t0) + x(t0)δt0
δxf = δx(tf ) + x(tf )δtf
• In Terms of δx0 and δxf , We Obtain
δΦ =∂Φ
∂x(t0)δx0 +
∂Φ
∂t0δt0
+∂Φ
∂x(tf )δxf +
∂Φ
∂tfδtf
Copyright © 2006–2012 Anil Vithala Rao Page 35
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• In a Similar Manner, We Have for φ That
δφ =∂φ
∂x(t0)δx0 +
∂φ
∂t0δt0
+∂φ
∂x(tf )δxf +
∂φ
∂tfδtf
• Next, We Have
δ
∫ tf
t0
[
H− λTx]
dt =
∫ tf
t0
[
δH− δλTx− λTδx]
dt
+[
H− λTx]
tfδtf
−[
H− λTx]
t0δt0
• Where
δH =
[
∂H
∂x
]
δx+
[
∂H
∂λ
]
δλ+
[
∂H
∂u
]
δu
Copyright © 2006–2012 Anil Vithala Rao Page 36
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
And, via Integration by Parts, We Have∫ tf
t0
λTδxdt =
[
λTδx
]tf
t0
−∫ tf
t0
λT
δxdt
• Important Note:
⊲ Terms Outside Integral are evaluated at t0 and tf
⊲ Consequently, We Have δx(t0) and δx(tf )
• Therefore,
δ
∫ tf
t0
[
H− λTx]
dt = λT(t0)δx(t0)− λT(tf )δx(tf )
+[
H(tf )− λT(tf )x(tf )]
δtf −[
H(t0)− λT(t0)x(t0)]
δt0
+
∫ tf
t0
[
∂H∂x
]
δx+
[
∂H∂λ
]
δλ+
[
∂H∂u
]
δu
dt
−∫ tf
t0
δλTx+ λ
T
δx
dt
Copyright © 2006–2012 Anil Vithala Rao Page 37
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Using the Earlier Results for δx(t0) and δx(tf ),
δ
∫ tf
t0
[
H− λTx]
dt
= λT(t0) [δx0 − x(t0)δt0]− λT(tf ) [δxf − x(tf )δtf ]
+[
H(tf )− λT(tf )x(tf )]
δtf −[
H(t0)− λT(t0)x(t0)]
δt0
+
∫ tf
t0
[
∂H∂x
]
δx+
[
∂H∂λ
]
δλ+
[
∂H∂u
]
δu
dt
−∫ tf
t0
δλTx+ λ
T
δx
dt
• Rearranging Gives
δ
∫ tf
t0
[
H− λTx]
dt = λT(t0)δx0 − λT(tf )δxf
+H(tf )δtf −H(t0)δt0
+
∫ tf
t0
[(
∂H∂x
+ λT
)
δx+
(
∂H∂λ
− xT
)
δλ
+∂H∂u
δu
]
dt
Copyright © 2006–2012 Anil Vithala Rao Page 38
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Terms Outside Integral
⊲ Terms involving δν:
δν = 0 or φ(x(t0), t0,x(tf ), tf ) = 0
⊲ Terms involving δx0 and δxf :
δx0 = 0 or λ(t0) = −[
∂Φ
∂x(t0)
]T
+
[
∂φ
∂x(t0)
]T
ν
δxf = 0 or λ(tf ) =
[
∂Φ
∂x(tf )
]T
−[
∂φ
∂x(tf )
]T
ν
⊲ Terms involving δt0 and δtf :
δt0 = 0 or H(t0) =∂Φ
∂t0− νT ∂φ
∂t0
δtf = 0 or H(tf ) = − ∂Φ
∂tf+ νT ∂φ
∂tf
Copyright © 2006–2012 Anil Vithala Rao Page 39
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Terms Inside Integral (i.e., δx, δu, and δλ):
∂H∂x
+ λT
= 0
∂H∂λ
− xT = 0
∂H∂u
= 0
• Rearranging These Last Three Equations, We Obtain
x =
[
∂H∂λ
]T
= f(x(t),u(t), t) ⇐⇒ State Dynamics
λ = −[
∂H∂x
]T
⇐⇒ Costate Dynamics
[
∂H∂u
]T
= 0 ⇐⇒ Condition for Optimal Control
Copyright © 2006–2012 Anil Vithala Rao Page 40
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Necessary Conditions and Extremals
• Double-Boxed Terms Are Necessary Conditions
• Let (x∗(t),λ∗(t),u∗(t)) Satisfy Necessary Conditions
⊲ (x∗(t),λ∗(t)) is an extremal trajectory
⊲ u∗(t) is an extremal control
• Important Notes About Extremal Solutions
⊲ Only candidate optimal solutions
⊲ Each Extremal May Be Either
Minimum
Maximum
Saddle
Copyright © 2006–2012 Anil Vithala Rao Page 41
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Understanding Structure of Optimality Conditions
• Dynamics of Augmented System
x = Hλ(x,λ)
λ = −Hx(x,λ)
• Vector Field (Hλ,−Hx) ∈ R2n is a Hamiltonian System
• Time Derivative of Hamiltonian
dHdt
=∂H∂t
+
[
∂H∂x
]Tdx
dt+
[
∂H∂λ
]Tdλ
dt
=∂H∂t
+
[
∂H∂x
]T [
∂H∂λ
]
−[
∂H∂λ
]T [
∂H∂x
]
=∂H∂t
+HT
xHλ −HT
λHx
But
HT
xHλ −HT
λHx = 0
Therefore,dHdt
=∂H∂t
Copyright © 2006–2012 Anil Vithala Rao Page 42
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Important Properties of Hamiltonian
⊲ Total Time Derivative = Partial Derivative With Respect to Time
⊲ If Hamiltonian not an explicit function of time
=⇒ ∂H∂t
= 0 ⇒ dHdt
= 0 ⇒ H = constant
⊲ If Hamiltonian not an explicit function of time and tf is free
H(tf ) = − ∂Φ
∂tf+ νT ∂φ
∂tf= 0 ⇒ H = 0
Copyright © 2006–2012 Anil Vithala Rao Page 43
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example
• Cost Functional
J =1
2
∫ tf
0
(qx2 + ru2)dt
where x(t) ∈ R, u ∈ R and q > 0 and r > 0 are constants
• Dynamic Constraints
x = ax+ bu
where a and b are constants
• Boundary Conditions
x(0) = x0
x(tf ) = xf
where x0 and xf are given values
Copyright © 2006–2012 Anil Vithala Rao Page 44
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Solution to Example
• Form the Hamiltonian
H = L+ λf =1
2qx2 +
1
2ru2 + λ(ax+ bu)
• State and Costate Dynamics
x = Hλ = ax+ bu
λ = −Hx = −qx− aλ
• Optimal Control
Hu = ru+ λb = 0 ⇒ u∗ = − brλ
• Substitution of Optimal Control into State Dynamics
x = ax− b2
rλ
• State-Costate Dynamics Including Condition for Optimal Control
x
λ
=
a − b2
r
−q −a
x
λ
Copyright © 2006–2012 Anil Vithala Rao Page 45
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Transversality Conditions
φ =
x(0)− x0
x(tf )− xf
ν =
ν1
ν2
Therefore,
∂φ
∂x(0)=
1
0
∂φ
∂x(tf )=
0
1
We Then Obtain
λ(0) = ν1 = Unknown
λ(tf ) = ν2 = Unknown
• Note: Neither (λ(0), λ(tf )) nor (ν1, ν2) is Known
=⇒ Determining λ⇐⇒ Determining ν
Copyright © 2006–2012 Anil Vithala Rao Page 46
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Hamiltonian Boundary-Value Problem for Example
• State-Costate Dynamics and Boundary Conditions
x
λ
=
a − b2
r
−q −a
x
λ
,
x(0) = x0
x(tf ) = xf
• λ(0) and λ(tf ) are unknown
• General Form of Dynamics: Linear Time-Invariant (LTI) System
p = Ap
• General Solution of First-Order LTI System
p(t) = c1eµ1tw1 + c2e
µ2tw2
where
p =
x
λ
µ1, µ2 = Eigenvalues of A
w1,w2 = Eigenvectors of A
Copyright © 2006–2012 Anil Vithala Rao Page 47
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Determination of Solution to HBVP
• Eigenvalues of System Matrix A
det [µI−A] = det
µ− a b2
r
q µ+ a
= µ2 − a2 − qb2
r= 0
• Assuming That q > 0 and r > 0, We Obtain
µ = ±√
a2 +qb2
r
• Note: Eigenvalues Have Same Magnitude But Opposite Sign
• Let ω =√
a2 + qb2
r =⇒ µ = ±ω
• Eigenvectors of A
⊲ Eigenvector for µ = ω
ω − a b2
r
q aω + a
w11
w12
=
0
0
Copyright © 2006–2012 Anil Vithala Rao Page 48
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
From first equation we have
(ω − a)w11 +b2
rw12 = 0
Let w11 = −1/(ω − a). We then have
w12 =r
b2
This gives
w1 =
−1/(ω − a)
r/b2
⊲ Eigenvector for µ = −ω
−ω − a b2
r
q −ω + a
w21
w22
=
0
0
From the first equation we have
−(ω + a)w21 +b2
rw22 = 0
Let w21 = 1/(ω + a). We then obtain
w22 =r
b2
Copyright © 2006–2012 Anil Vithala Rao Page 49
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Which implies that
w2 =
1/(ω + a)
r/b2
• Solving for Constants c1 and c2
⊲ At t = 0 we have
x(0) = x0 = c1w11 + c2w21 = − c1ω − a
+c2
ω + a
⊲ At t = tf we have
x(tf ) = xf = c1eωtfw11 + c2e
−ωtfw21 = − c1eωtf
ω − a+c2e
−ωtf
ω + a
⊲ Solving last two equations for c1 and c2 gives
c1
c2
= 1
eωtf −e
−ωtf
e−ωtf (ω − a) −(ω − a)
eωtf (ω + a) −(ω + a)
x0
xf
• Solution for a = b = q = r = 1, t0 = 0, x(t0) = 1, tf = (1, 5, 10, 15, 20), and
x(tf ) = 2
Copyright © 2006–2012 Anil Vithala Rao Page 50
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
0 5 10 15 200
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
x(t)
tf = 1
tf = 5
tf = 10
tf = 15
tf = 20
Copyright © 2006–2012 Anil Vithala Rao Page 51
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
0 5 10 15 20−1
−0.5
0
0.5
1
1.5
2
2.5
t
λ(t)
tf = 1
tf = 5
tf = 10
tf = 15
tf = 20
• Observations of Solution Structure
⊲ For small tf , solution
Exhibits no decay
Moves essentially directly from initial to terminal condition
Copyright © 2006–2012 Anil Vithala Rao Page 52
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ For large tf , solution
Appears to have a three-segment structure
First segment: decay from initial condition to zero
Second segment: remains very close to zero
Third segment: rapid growth to meet terminal condition
⊲ Key points
Both state and costate have decaying and growing behavior
Growth and decay show up as tf increases
Copyright © 2006–2012 Anil Vithala Rao Page 53
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example: Orbit-Raising Problem
Problem: determine the thrust direction as a function of time that transfers a
spacecraft from Earth orbit to Mars orbit in minimum time (Reference: Bryson
and Ho, Applied Optimal Control, Hemisphere Publishing, New York, 1975).
Copyright © 2006–2012 Anil Vithala Rao Page 54
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Dynamics in First-Order Form
r = u
u =v2
r− µ
r2+T
msinα
v = −uvr
+T
mcosα
m = − T
g0Isp
• Astronomical System of Units (AU) Chosen Such That µ = 1 and Distance
of Sun to Earth is Unity
• Boundary Conditions
⊲ Initial conditions: circular orbit of size 1 AU (i.e., Earth orbit)
r(0) = r0 = 1 AU
u(0) = 0
v(0) =√
µ/r0 = 1 AU/TU
m(0) = 1
Copyright © 2006–2012 Anil Vithala Rao Page 55
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Terminal conditions: circular orbit of size 1.5 AU (i.e., Mars orbit)
r(tf ) = rf = 1.5 AU
u(tf ) = 0
v(tf ) =√
µ/rf ≈ 0.816496581 AU/TU
⊲ Objective Functional: Minimize Time
J =
Mayer Form⇓
tf =
Lagrange Form⇓
∫ tf
0
dt
⊲ We will use Mayer form of cost functional
• Hamiltonian (Observe that L ≡ 0 and Φ = tf )
H = L+ λTf = λr r + λuu+ λv v + λmm
= λru+ λu
[
v2
r− µ
r2+T
msinα
]
+ λv
[
−uvr
+T
mcosα
]
+ λm
[
− T
g0Isp
]
Copyright © 2006–2012 Anil Vithala Rao Page 56
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Costate Equations
λr = −∂H∂r
= λu
[
v2
r2− 2µ
r3
]
− λvuv
r2
λu = −∂H∂u
= −λr + λvv
r
λv = −∂H∂v
= −λu2v
r+ λv
u
r
λm = −∂H∂m
= λu
[
T
m2sinα
]
+ λv
[
T
m2cosα
]
• Condition for Optimal Control
∂H∂α
= 0 =⇒ λuT
mcosα− λv
T
msinα = 0
Solving for α Gives
α∗ = tan−1(λu, λv)
Note: A four-quadrant inverse tangent is used because α ∈ [−π, π]
• Transversality Conditions
Copyright © 2006–2012 Anil Vithala Rao Page 57
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ The boundary conditions can be written in the general form
φ(x(t0), t0,x(tf ), tf ) =
r(t0)− r0
u(t0)
v(t0)−√
µ/r0
m(t0)−m0
r(tf )− rf
u(tf )
v(tf )−√
µ/rf
= 0
Copyright © 2006–2012 Anil Vithala Rao Page 58
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Let νT = (νr0, νu0, νv0, νm0, νrf , νuf , νvf )
∂φ
∂x(t0)=[
∂φ
∂r(t0)∂φ
∂u(t0)∂φ
∂v(t0)∂φ
∂m(t0)
]
=
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
0 0 0 0
0 0 0 0
0 0 0 0
Copyright © 2006–2012 Anil Vithala Rao Page 59
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
∂φ
∂x(tf )=[
∂φ
∂r(tf )∂φ
∂u(tf )∂φ
∂v(tf )∂φ
∂m(tf )
]
=
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
1 0 0 0
0 1 0 0
0 0 1 0
⊲ Initial transversality conditions
λ(0) = − ∂Φ
∂x(t0)+
[
∂φ
∂x(t0)
]T
ν ⇒
λr(0)
λu(0)
λv(0)
λm(0)
=
νr0
νu0
νv0
νm0
Copyright © 2006–2012 Anil Vithala Rao Page 60
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Terminal transversality conditions
λ(tf ) =∂Φ
∂x(t0)−[
∂φ
∂x(tf )
]T
ν ⇒
λr(tf )
λu(tf )
λv(tf )
λm(tf )
=
−νrf−νuf−νvf0
• Notes
⊲ Only one of the boundary costates is known
⊲ Seven unknown boundary conditions
• One Last Important Point: ∂H/∂α = 0 is Only a Necessary Condition
⊲ Sufficient condition: ∂2H/∂α2 > 0
∂2H∂α2
= −λuT
msinα− λv
T
mcosα > 0
=⇒ −λu sinα− λv cosα > 0 =⇒ tanα > −λv
λu
⊲ Therefore,λu
λv
> −λv
λu
=⇒ λ2u + λ2
v > 0
Copyright © 2006–2012 Anil Vithala Rao Page 61
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Hamiltonian Boundary-Value Problem for Orbit-Transfer Problem
• Dynamics (α∗ = tan−1(λu, λv)):
r = u , u =v2
r− µ
r2+T
msinα∗
v = −uvr
+T
mcosα∗ , m = − T
g0Isp
λr = λu
[
v2
r2− 2µ
r3
]
− λvuv
r2, λu = −λr + λv
v
r
λv = −λu2v
r+ λv
u
r, λm = λu
[
T
m2sinα∗
]
+ λv
[
T
m2cosα∗
]
• Boundary Conditions:
r(t0) = r0 , u(t0) = 0 , v(t0) =√
µ/r0 , m(t0) = m0,
r(tf ) = rf , u(tf ) = 0 , v(tf ) =√
µ/rf , λm(tf ) = 0
• Notes:
⊲ Initial and Terminal Conditions are Incomplete
⊲ Half of the initial conditions and terminal conditions are specified
Copyright © 2006–2012 Anil Vithala Rao Page 62
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
The Minimum Principle of Pontryagin
• In General, the Optimal Control Satisfies the Condition
J(u)− J(u∗) ≥ 0
for all u sufficiently close to u∗
• Therefore J(u) = J(u∗) + δJ(u∗, δu)
• Then, from Variation of Augmented Cost Functional
δJ(u∗, δu) =
∫ tf
t0
[
∂H∂u
]
u∗
δudt
• Term in Integrand Given As[
∂H∂u
]
u∗
δu = H(x∗,u∗ + δu,λ∗)−H(x∗,u∗,λ∗) + higher order terms
• Consequently, to First Order
δJ(u∗, δu) =
∫ tf
t0
[H(x∗,u∗ + δu,λ∗)−H(x∗,u∗,λ∗)] dt ≥ 0
Copyright © 2006–2012 Anil Vithala Rao Page 63
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Because δJ(u∗, δu) > for All Admissible Variations in u,
H(x∗,u∗,λ∗) ≤ H(x∗,u∗ + δu,λ∗)
• Leads to Following Condition for Optimal Control
u∗ = argmin
u∈UH(x∗,u,λ∗), U = Admissible Control Set
• Possibilities for Determining Optimal Control
⊲ Control Lies Interior to U
∂H∂u
= 0 =⇒ Strong Form of Minimum Principle
⊲ Control Lies on Boundary of U
One-sided differential → must use weak form of minimum principle
u∗ = argmin
u∈UH(x∗,u,λ∗) ⇐⇒ Weak Form of Minimum Principle
⊲ Example: Hamiltonian linear in control
H = L(x) + λTf(x) + λT
u
∂H/∂u provides no information about u
Must use weak form of minimum principle
Copyright © 2006–2012 Anil Vithala Rao Page 64
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example: Minimum-Time Double Integrator
• Consider the Following Optimal Control Problem. Minimize
J = tf =
∫ tf
0
dt
Subject to
x1 = x2
x2 = u
with
x1(0) = x10 , x1(tf ) = 0
x2(0) = x20 , x2(tf ) = 0
and
|u| ≤ 1 ⇐⇒ −1 ≤ u ≤ 1
• Problem in Words
Determine the optimal trajectory and control that steers the system from
an initial nonzero state to the origin in minimum time.
Copyright © 2006–2012 Anil Vithala Rao Page 65
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Solution of Double Integrator Minimum Time Example
• Hamiltonian (Using Lagrange Form of Problem)
H = 1 + λ1x2 + λ2u
• Optimal Control
⊲ ∂H/∂u gives no information
⊲ Need to apply general form of minimum principle
u∗ = arg minu∈[−1,1]
H = −sgn λ2 = ±1
where sgn(·) is the sign or signum function
• Costate Equations
λ1 = −∂H/∂x1 = 0 =⇒ λ1(t) = constant = λ10
λ2 = −∂H/∂x2 = −λ1 =⇒ λ2(t) = −λ10t+ λ20
• Observation: λ2(t) is a Straight Line
• Four Possibilities for λ2(t) (Depending on Sign of λ10)
Copyright © 2006–2012 Anil Vithala Rao Page 66
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ λ2(t) is always positive
⊲ λ2(t) is always negative
⊲ λ2(t) switches from positive to negative
⊲ λ2(t) switches from negative to positive
Copyright © 2006–2012 Anil Vithala Rao Page 67
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
0 0.5 1 1.5−2
−1
0
1
2
0 0.5 1 1.5−2
−1
0
1
2
0 0.5 1 1.5−2
−1
0
1
2
0 0.5 1 1.5−2
−1
0
1
2
λ1(t)
λ1(t)
λ1(t)λ1(t)
λ2(t)
λ2(t)
λ2(t)
λ2(t)
u(t)
u(t)
u(t)u(t)
tt
tt
λ1 ≤ 0, λ2 ≥ 0 λ1 ≥ 0, λ2 ≤ 0
λ1 > 0, λ2 > 0 λ1 < 0, λ2 < 0
• Four Possible Optimal Controls: −1 , +1 , −1,+1 , +1,−11
Copyright © 2006–2012 Anil Vithala Rao Page 68
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Suppose we let s = ±1. Then we have
x2(t) = st+ x20
x1(t) = 12st2 + x20t+ x10
• We can then eliminate t from above equations to obtain
x1 = x10 +12sx22 − 1
2sx220
t = s(x2 − x20)
Copyright © 2006–2012 Anil Vithala Rao Page 69
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
−3 −2 −1 0 1 2 3−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
x1
x2
s = −1
s = 1
• Important Notes
⊲ Control must steer initial state to origin
Copyright © 2006–2012 Anil Vithala Rao Page 70
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ As a result, the extremal solution must include one of the parabolas
that pass through the origin
• Structure of Optimal Solution
⊲ Case 1: Follow thick green curve into origin: u = −1
⊲ Case 2: Follow thick blue curve into origin: u = +1
⊲ Case 3: start on a green curve and switch to thick blue curve: u = −1
followed by u = +1
⊲ Case 4: start on a blue curve and switch to thick green curve: u = 1
followed by u = −1
• Form of optimal control; bang-bang
⊲ u∗ is never on the interior of admissible control set
⊲ u∗ is always at its minimum or maximum value
• Physical Example: Liquid Rocket Thruster: T ∈ [0, Tmax]
Copyright © 2006–2012 Anil Vithala Rao Page 71
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Singular Arcs
• Recall the Minimum Principle of Pontryagin
H(x∗,u∗,λ∗) ≤ H(x∗,u,λ∗)
• Suppose Now that There Exists an Interval [t1, t2] Such That
⊲ u∗(t) cannot be determined from Minimum Principle
⊲ Then [t1, t2] is called a singular interval or singular arc
• In General, Singular Intervals are Difficult to Detect
• Certain Classes of Problems: Singular Problems May Be Solvable
⊲ Hamiltonian not an explicit function of time −→ H∗ is constant
⊲ Can use conditions on H∗ to assist with singular arc detection
Copyright © 2006–2012 Anil Vithala Rao Page 72
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Determination of Singular Arc Conditions
• Consider a Problem where H∗ is constant
• Let [t1, t2] be an Interval Such That
⊲ Optimal control cannot be determined from necessary conditions
⊲ Auxiliary conditions must be used to determine u∗(t) on [t1, t2]
• Approach for Determining Singular Arc Conditions
⊲ Differentiate H with respect to time q times until control appears
⊲ Because H∗ is constant, we know that
d(q)H
dt(q)= 0
⊲ Furthermore, dH(q)/dt(q) is an explicit function of control
⊲ Thus, dH(q)/dt(q) can be used to solve for control on [t1, t2]
Copyright © 2006–2012 Anil Vithala Rao Page 73
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Singular Arc Example: Goddard Rocket Problem
• Maximize h(tf ) Subject to
⊲ Dynamics
h(t) = v
v(t) = T (t)−D(v,h)m − g
m(t) = −T (t)c
⊲ Boundary Conditions
h(0) = 0
v(0) = 0
m(0) = m0 = given
h(tf ) = Free
v(tf ) = Free
m(tf ) = mf = Given
⊲ Control constraint: 0 ≤ T (t) ≤ Tmax
Copyright © 2006–2012 Anil Vithala Rao Page 74
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Goddard Rocket Problem (Continued)
• Hamiltonian
H = λhv + λv
(
T −D
m− g
)
− λmT
c
= λhv +
(
λvm−λmc
)
T − λv
(
D
m+ g
)
• Key Issue
⊲ Optimal control may not be bang-bang
⊲ Reason: depends upon sign of λv/m− λm/c
• Possibilities
T ∗(t) =
0 , λv/m− λm/c > 0
0 ≤ T ∗(t) ≤ Tmax , λv/m− λm/c = 0
Tmax , λv/m− λm/c < 0
• Suppose that λv/m− λm/c = 0 =⇒ Singular Arc
Copyright © 2006–2012 Anil Vithala Rao Page 75
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Goddard Rocket Problem (Continued)
• H Does Not Depend Explicitly on t =⇒ H∗ is constant
• All Time Derivatives of H∗ are Zero
• In This Case, on Singular Arc We Have
λvm−λmc
= 0 =⇒ cλv −mλm = 0
• Can Let I = λv/m− λm/c
• Differentiate I with Respect to t Until Control Appears
• This Problem: Control Appears in I and We Know I∗ = 0
• The Condition I∗ = 0
⊲ Enables determining the control on the singular arc
⊲ Gives a “feedback law” for optimal control on singular arc
Copyright © 2006–2012 Anil Vithala Rao Page 76
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Summary
• Optimal Control Theory Founded on Calculus of Variations
• Using This Approach, Must Formulate
⊲ First-Order Optimality Conditions of Continuous Problem
⊲ Leads to a Hamiltonian Boundary-Value Problem
• Determination of Optimal Control
⊲ If Control is Interior to Feasible Control Set: Hu = 0
⊲ If Control Lies on Boundary of Control Set
u∗ = argminu∈UH(x∗,u∗,λ∗, t∗)
• Even in Numerical Methods, Optimal Control Theory Important
⊲ Provides Systematic Means of Formulating Problem
⊲ Enables Analysis of Solution
• Later: Will Discuss Solution Methods for HBVP
Copyright © 2006–2012 Anil Vithala Rao Page 77
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part III: Nonlinear Optimization
Copyright © 2006–2012 Anil Vithala Rao Page 78
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Preliminaries
• A scalar is any real number, i.e., x is a scalar if x ∈ R where R is the set of
real numbers
• A column vector x lies in n-dimensional real space, i.e., x is a column
vector if x ∈ Rn and is defined as
x =
x1
x2...
xn
• A row vector lies in n-dimensional real space, i.e., y is a row vector if
y ∈ Rn and is defined as
y = [ y1 y2 · · · yn ] ≡ (y1, y2, . . . , yn)
Copyright © 2006–2012 Anil Vithala Rao Page 79
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• A matrix A ∈ Rm×n is defined as
A =
a11 a12 · · · a1n
a21 a22 · · · a2n...
.... . .
...
am1 am2 · · · amn
• The Transpose of a Matrix A ∈ Rm×n, Denoted AT , is Defined as
AT =
a11 a12 · · · a1m
a21 a22 · · · a2m...
.... . .
...
an1 am2 · · · anm
• Column Vectors and Row Vectors related by a transpose, i.e.,
x is a column vector ⇐⇒ xT is a row vector
• Conventions Used for Vector Functions
⊲ The notation Rm → R
n means mapping from Rm to R
n.
Copyright © 2006–2012 Anil Vithala Rao Page 80
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Let F : Rn → R. Then the gradient of F is a column vector and is
defined as
G(x) ≡ ∂F
∂x=[
∂F∂x1
∂F∂x2
· · · ∂F∂xn
]
⊲ Let f : Rn → Rm be a function that maps vectors from R
n to Rm. Then
h(x) is a column vector defined as
f(x) =
f1(x)
f2(x)
...
fm(x)
where fi(x) ≡ fi(x1, . . . , xn), i = 1, . . . ,m.
⊲ The Jacobian matrix or Jacobian of f(x), defined as J(x) ∈ Rm×n,
J(x) ≡ ∂f
∂x=[
∂f∂x1
∂f∂x2
· · · ∂f∂xn
]
Copyright © 2006–2012 Anil Vithala Rao Page 81
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
where ∂f/∂xj , (j = 1, . . . , n) are m-dimensional column vectors, i.e.,
∂f
∂xj=
∂f1∂xj
∂f2∂xj
...
∂fn∂xj
• In terms of f1(x), . . . , fm(x) and x1, . . . , xn the Jacobian can be written as
J(x) =
∂f1∂x1
∂f1∂x2
· · · ∂f1∂xn
∂f2∂x1
∂f2∂x2
· · · ∂f2∂xn
......
. . ....
∂fm∂x1
∂fm∂x2
· · · ∂fm∂xn
∈ Rm×n
• Using These Conventions, for the special case of a function F : Rn → R,
the gradient and Jacobian of F are defined, respectively, as
G(x) = ∂F∂x
≡ ∇xF
J(x) = ∂G∂x
≡ ∂2F∂x2 ≡ H(x) = ∇2
xF
Copyright © 2006–2012 Anil Vithala Rao Page 82
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
where the function H(x) has a special name called the Hessian of F . The
Hessian of F is written explicitly in terms of the gradient as
H(x) =[
∂G∂x1
∂G∂x2
· · · ∂G∂xn
]
or, alternatively, in terms of F as
H(x) =
∂2F
∂x21
∂2F∂x2x1
· · · ∂2F∂xnx1
∂2F∂x1x2
∂2F
∂x22
· · · ∂2F∂xnx2
......
. . ....
∂2F∂x1xn
∂2F∂x2xn
· · · ∂2F∂x2
n
∈ Rn×n
It is noted that the Hessian is symmetric.
• Important Notes: Using the above conventions, the Hessian is derivative
of the transpose of the gradient, i.e.,
H(x) =∂
∂x
[
GT(x)
]
• The Hessian matrix H(x) is symmetric, i.e., H = HT
• Abbreviated Notation for Gradient, Jacobian, and Hessian
Copyright © 2006–2012 Anil Vithala Rao Page 83
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Gradient of a Scalar Function F (x):
G(x) ≡ ∂F
∂x≡ Fx
⊲ Hessian of a Scalar Function F (x):
H(x) ≡ ∂2F
∂x2≡ Fxx
⊲ Jacobian of a Vector Function f(x):
J(x) ≡ ∂f
∂x≡ fx
Copyright © 2006–2012 Anil Vithala Rao Page 84
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Unconstrained Optimization
• Let F : Rn → R. Then x∗ is called a strong minimum of F if there exists
δ ∈ R such that
F (x∗) < F (x∗ +∆x)
for all 0 < ‖∆x‖ < δ. Similarly, x∗ is called a weak minimum of F if x∗
is not a strong minimum and there exists δ ∈ R such that
F (x∗) ≤ F (x∗ +∆x)
0 < ‖∆x‖ < δ.
• Global vs. Local Minimum
⊲ x∗ is said to be a global minimum of F if δ =∞.
⊲ x∗ is said to be a local minimum of F if δ <∞.
In general, a scalar function F (x) will have many local minima of which
one of them may be a global minimum.
Copyright © 2006–2012 Anil Vithala Rao Page 85
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Necessary & Sufficient Conditions for a Local Minimum
• Assume F (x) is Twice Differentiable in a Neighborhood of x∗, We Have
F (x∗) ≈ F (x∗ +∆x) +
[
∂F
∂x
]
x∗
∆x+1
2∆x
T
[
∂2F
∂x2
]
x∗
∆x
= F (x∗ +∆x) +GT(x∗)∆x+
1
2∆x
TH(x)∆x
• x∗ is said to be a stationary point of F (x) if
G(x∗) = 0
• Necessary Conditions
⊲ G(x∗) = 0, i.e., x∗ must be a stationary point of F
⊲ H(x∗) ≥ 0, (i.e., H(x∗) is positive semi-definite)
• Sufficient Conditions
⊲ G(x∗) = 0, i.e., x∗ must be a stationary point of F
⊲ H(x∗) > 0, (i.e., H(x∗) > 0, is positive definite)
Copyright © 2006–2012 Anil Vithala Rao Page 86
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Solution of Unconstrained Optimization Problems
• What Needs to Be Done
⊲ Determine stationary points from G(x): give candidate local minima
⊲ Check definiteness of H(x) at all stationary points
⊲ Accept all stationary points where H is either PD or PSD
⊲ Reject all stationary points where H is neither PD nor PSD
• Above Mathematical Approach Nice in Theory
• General Problem
⊲ No analytic solution
⊲ Need to solve numerically
⊲ Numerical methods are generally iterative
• Categories of numerical methods for unconstrained optimization
⊲ Univariate (functions of a single variable) problems
⊲ Multivariate (functions of many variables) problems
Copyright © 2006–2012 Anil Vithala Rao Page 87
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Univariate Optimization
• Consider the following problem where x ∈ R:
minx∈R
F (x)
• Necessary & sufficient conditions for a minimum are
F ′(x) =dF
dx= 0 , F ′′(x) =
d2F
dx2> 0
• Complicated F : Need to Employ an Iterative Method
• First Step: Bracket the Minimum
⊲ Know values of F at x1, x2, and x3
⊲ Condition: F (x2) < F (x1) and F (x2) < F (x3)
Copyright © 2006–2012 Anil Vithala Rao Page 88
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
xx1 x2 x3
F (x)
⊲ Take an Additional Point x4
xx1 x2 x3x4
F (x)
• ThenF (x4) < F (x2) ⇒ New Interval: (x1, x4, x2)
F (x4) > F (x2) ⇒ New Interval: (x4, x2, x3)
• Question: How Do We Choose x4?
Copyright © 2006–2012 Anil Vithala Rao Page 89
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Commonly Used Methods
⊲ Golden section
⊲ Polynomial approximation
Copyright © 2006–2012 Anil Vithala Rao Page 90
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Golden Section Search
• Let L = x3 − x1 and τ = x2−x1
x3−x2, i.e.,
τ =x2 − x1x3 − x2
=⇒
x2 − x1 = Lτ1+τ
x3 − x2 = L1+τ
1
xx1 x2 x3
L
L1+τ
Lτ1+τ
τ
• Goal: Maximize Reduction in Bracket at Each Step
• Choose x4 To Be in Larger Interval
Copyright © 2006–2012 Anil Vithala Rao Page 91
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
1
xx1 x2 x3x4
L
L1+τ
Lτ1+τ
τ
• Want New Interval to Have Same Ratio as Original Interval
1
1
xx1 x2 x3x4
L
L1+τ
Lτ1+τ
τ
τ
Lτ2
(1+τ)2Lτ
(1+τ)2
Copyright © 2006–2012 Anil Vithala Rao Page 92
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Therefore,
x2 − x4 =1
1 + τ
Lτ
1 + τ=
Lτ
(1 + τ)2
• Want x2 − x1 to equal x3 − x4 Which Implies That
x2 − x1 x3 − x4 =Lτ
1 + τ
• Using the constraint that x2 − x1 = x3 − x4, we obtain
L = x3 − x1 = x4 − x1 + x3 − x4 =Lτ2
(1 + τ)2+
Lτ
1 + τ
• Rearranging, we obtain
(1 + τ)2 = τ(1 + τ) + τ2 =⇒ τ2 − τ − 1 = 0
• Roots of Polynomial:
τ1,2 =1±
√5
2
• Because τ Must Be Greater Than Unity (and Positive),
τ =1 +
√5
2= Golden Ratio
Copyright © 2006–2012 Anil Vithala Rao Page 93
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Algorithm 1 (Golden Section)
1. Given Interval (x1, x2, x3) Where F (x2) < F (x1) and F (x2) < F (x3)
2. Choose x4 To Be in Large Part of Interval Such That Ratio of Large Part
of Interval is τ
3. Find F (x4)
4. If F (x4) > F (x2), New Interval is (x1, x2, x4), Otherwise New Interval is
(x2, x4, x3)
5. Assign New Interval =⇒ (x1, x2, x3)
6. Tolerance met? Yes ⇒ Stop; Otherwise ⇒ Step 2
Copyright © 2006–2012 Anil Vithala Rao Page 94
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Polynomial Line Search
• Key Attributes of Golden Section
⊲ No assumptions were made about function
⊲ Very robust, but has slow (linear) convergence
⊲ Would like a method with faster convergence
• Faster Convergence: Quadratic Approximation to F
• Approach: Assume F (x) is Well-Behaved
⊲ Start with three points x1, x2, and x3
⊲ Fit quadratic through these points, i.e.,
p(x) = a(x− b)2 + c
⊲ x4 obtained where p(x) is minimized (i.e., x = b)
⊲ After some algebra, it turns out that
b =1
2
F (x1)(x23 − x22) + F (x2)(x
21 − x23) + F (x3)(x
22 − x21)
F (x1)(x3 − x2) + F (x2)(x3 − x1) + F (x3)(x2 − x1)
• Benefits of Quadratic Line Search
Copyright © 2006–2012 Anil Vithala Rao Page 95
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Much faster convergence than golden section
⊲ Near minimum: superlinear convergence
⊲ Caution: no such thing as a free lunch
Quadratic line search can fail
May need to use a less aggressive approach
• Polynomial Line Search Can Be of Any Order
⊲ For N th-degree approximation, need N + 1 points
⊲ Generally only need to go as high as a cubic
• Algorithm for Quadratic Line Search
Algorithm 2 (Quadratic Line Search)
1. Given Interval (x1, x2, x3) Where F (x2) < F (x1) and F (x2) < F (x3)
2. Approximate F by a Quadratic p(x) Using x1, x2, and x3
3. Determine x4 from Minimum of p(x)
4. If F (x4) > F (x2), New Interval is (x1, x2, x4), Otherwise New Interval is
(x2, x4, x3)
Copyright © 2006–2012 Anil Vithala Rao Page 96
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
5. Assign New Interval =⇒ (x1, x2, x3)
6. Tolerance met? Yes ⇒ Stop; Otherwise ⇒ Step 2
Copyright © 2006–2012 Anil Vithala Rao Page 97
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Multivariate Unconstrained Optimization
• Supply an initial guess of x∗, i.e., x0
• Perform an update at each step k as follows:
xk+1 = xk + αkDkpk
where pk is the search direction, αk is the step length and is
determined via a line search, and Dk ∈ Rn×n is a positive definite matrix.
Algorithm 3 (Unconstrained Minimization)
1. Set k = 0 and choose initial guess, xk = x
2. Test for convergence: G(xk) = 0 and H(xk) > 0 ?
3. If answer to step (2) is “yes”, then quit
4. Otherwise, compute
(a) search direction, pk
(b) step length, αk
(c) Positive definite matrix, Dk
Copyright © 2006–2012 Anil Vithala Rao Page 98
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
5. Update xk as
xk+1 = xk + αkDkpk
6. Return to step (2)
Copyright © 2006–2012 Anil Vithala Rao Page 99
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Methods for Determination of pk, αk, and Dk
• Determination of pk and Dk
⊲ First-order methods
Use gradient information
Common method: steepest descent
⊲ Second-order methods
Use both gradient and Hessian information
Common method: Newton method
⊲ Quasi-Newton methods
Use an approximation to Hessian
Common methods: DFP and BFGS
• Methods for Determination of Step Length, αk
⊲ Step Length Determined from Univariate Optimization Methods
Exact Line Search
Inexact Line Search
Copyright © 2006–2012 Anil Vithala Rao Page 100
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Method of Steepest Descent
• Want a Search Direction that Decreases Value of F at Each Step
• Perform a Taylor Series Expansion About xk
F (xk+1) ≈ F (xk) +GT(xk)(xk+1 − xk)
= F (xk) +GT
k(αkDkpk)
This last result implies that
xk+1 − xk = αkDkpk
• Now assume that αk > 0 and is chosen such that the function F decreases
in moving from xk to xk+1, i.e., F (xk+1) < F (xk). Then it must be the
case that
GT
kDkpk < 0
• Suppose We Choose pk = −Gk and DK = I. Then
xk+1 − xk = −αkGk =⇒ xk+1 = xk − αkGk
• The Direction pk = −Gk is Called the Direction of Steepest Descent
Copyright © 2006–2012 Anil Vithala Rao Page 101
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Newton Method
• Suppose at Each Step we Choose xk+1 Such That
G(xk+1) = 0
• Then
G(xk+1) ≈ G(xk) +
[
∂G
∂x
]
xk
(xk+1 − xk) = 0
• But We Know That[
∂G
∂x
]
xk
= H(xk)
• Solving for xk+1, we obtain the Newton Method
xk+1 = xk −H−1(xk)G(xk)
• Therefore, for a Newton Method,
Dk = H−1(xk) ; pk = G(xk) ; αk = 1
Copyright © 2006–2012 Anil Vithala Rao Page 102
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Comparison Between Steepest Descent & Newton Iteration
• Steepest Descent
⊲ Simple Iterative Approach: Uses Only First-Derivatives
⊲ Slow (Linear) Convergence, But Very Reliable
• Newton Method
⊲ More Sophisticated: Uses Second-Derivatives
⊲ Much Faster Convergence (Quadratic)
⊲ Not As Reliable
H May Not Remain Positive Definite
Known to Diverge Depending Upon Initial Guess
• Want an Iterative Procedure with Fast Convergence and Reliability
• Common Balanced Approach: Quasi-Newton Methods
Copyright © 2006–2012 Anil Vithala Rao Page 103
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Quasi-Newton Methods
• Definition of a Quasi-Newton Method
xk+1 = xk − αkDkGk
where Dk is an approximation to the inverse of the Hessian at each step
• Key Idea: Approximate Convexity (i.e., Second Derivative)
⊲ Does Not Require Computation the Hessian of F
⊲ Requires Only Information Functions and Gradients
• Construct Quasi-Newton Approximation to H
⊲ Consider two successive iterates xk and xk+1
⊲ We know that
G(xk+1) ≈ G(xk) +Hk(xk+1 − xk)
=⇒ G(xk+1)−G(xk) ≈ Hk(xk+1 − xk)
⇐⇒ Gk+1 −Gk ≈ Hk(xk+1 − xk)
Copyright © 2006–2012 Anil Vithala Rao Page 104
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Suppose now that we let
qk = Gk+1 −Gk
⊲ Then, using the fact that pk = xk+1 − xk, we have
pT
kHkpk ≈ (Gk+1 −Gk)Tpk = q
T
kpk
⊲ It is noted that pT
kHkpk is an approximation to the curvature of F
along pk.
⊲ Approximation to Hessian Constructed Using Information from
Previous Iterations
• Suppose Now That We Have Bk, where Bk is an Approximation to Hk
⊲ At Step k + 1, Bk Updated Using Update Matrix Uk As
Bk+1 = Bk + Uk
⊲ Bk+1 must satisfy the Quasi-Newton condition
Bk+1pk = qk
Copyright © 2006–2012 Anil Vithala Rao Page 105
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Assume now that Uk = vwT. Then
Bk+1pk = (Bk + vwT)pk = qk
=⇒ v(wTpk) = qk −Bkpk
⊲ Can Be Shown That
Bk+1 = Bk +1
wTpk
(qk −Bkpk)wT
• Well-Known Quasi-Newton Update Algorithms
⊲ Davidon-Fletcher-Powell (DFP)
Bk+1 = Bk − BkpkpT
kBk
pT
kBkpk
+qkq
T
k
qT
kpk
+(
pT
kBkpT
k
)
ykyT
k
where
yk =qk
qT
kpk
− Bkpk
pT
kBkqk
⊲ Broyden-Fletcher-Goldfarb-Shanno (BFGS)
Bk+1 = Bk − BkpkpT
kBK
pT
kBkpk
+qkq
T
k
qT
kpk
Copyright © 2006–2012 Anil Vithala Rao Page 106
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Using the Quasi-Newton Approximation to the Inverse of the Hessian (Bk),
xk+1 = xk + αkBkpk
• Important Points About Quasi-Newton Updates
⊲ B0 Positive Definite −→ Bk Remains Positive Definite
⊲ B0 Generally Taken As Identity Matrix
Copyright © 2006–2012 Anil Vithala Rao Page 107
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Line Search
• What is the Best Value of αk?
• Obtain αk from Solution of Minimization Problem
minαk≥0
F (xk + αkpk)
• Solution of Line Search Minimization Problem
⊲ Optimal value of αk: exact line search
⊲ Suboptimal value of αk: inexact line search
⊲ Reasons to employ inexact line search
Exact line searches are inefficient and add little to increase rate of
convergence
Inexact line searches are fast and efficient
• Commonly Used Inexact Line Search Methods
⊲ Golden Section
⊲ Bisection
⊲ Polynomial Approximations
Copyright © 2006–2012 Anil Vithala Rao Page 108
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Important Points About Line Search
⊲ A line search is a univariate optimization problem
⊲ Line searches satisfy Goldstein-Armijo condition:
0 ≤ −κ1αkF′(0) ≤ F (0)− F (αk) ≤ −κ2αkF
′(0)
Copyright © 2006–2012 Anil Vithala Rao Page 109
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Trust Region Methods
• Pose the Following Constrained Optimization Problem:
minp
GT
kp+1
2pTHkp subject to p
Tp < ∆
• Idea: ∆ is a “Trust-Radius”
• Property: for λ ∈ R The Solution p of
(HK + λI)p = −Gk
Solves the problem if either
λ = 0 and pTp ≤ ∆
or
λ ≥ 0 and pTp = ∆
• Properties of Trust Region Methods
⊲ ∆ Sufficiently large −→ Newton method
⊲ ∆→ 0 and ‖p‖ → 0 −→ p Approaches Steepest Descent Direction
Copyright © 2006–2012 Anil Vithala Rao Page 110
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example of Unconstrained Optimization
• Quartic Unconstrained Minimization Problem:
minx1,x2
F (x1, x2)
where
F (x1, x2) = 100(x1 − x22)2 + (1− x1)
2
• Necessary Conditions for Optimality
∂F∂x1
= 200(x1 − x22)− 2(1− x1) = 0
∂F∂x2
= −400x2(x1 − x22) = 0
• Stationary Points
(x1, x2) = (101, 0)
(x1, x2) = (1, 1)
• Hessian of F
H =
200 −400x2
−400x2 1200x22
Copyright © 2006–2012 Anil Vithala Rao Page 111
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Check Definiteness of H at Each Stationary Point
H(x1 = 101, x2 = 0) =
200 0
0 0
≥ 0
H(x1 = 1, x2 = 1) =
200 −400
−400 1200
> 0
=⇒ Optimal solution is (x1, x2) = (1, 1)
Copyright © 2006–2012 Anil Vithala Rao Page 112
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Solution of Example Via Iteration
• Use Various Iterative Approaches to Solve Example
• MATLAB: can use fminunc (Code on Next Slides)
⊲ Three different Hessian updates: steepest descent, DFP, and BFGS
⊲ Need to set tolerances on x and F : Choose 10−5
• Results
Copyright © 2006–2012 Anil Vithala Rao Page 113
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
0 100 200 300 400 5000
0.5
1
1.5
2
2.5
3
3.5
Iterations
L2−
Nor
m E
rror
Steepest DescentDFPBFGS
Copyright © 2006–2012 Anil Vithala Rao Page 114
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
FMINUNC Code for Quartic Example Problem
global xpath iters;
xpath = zeros(10000,2);
iters = 0;
xlow = -Inf*ones(2,1);
xupp = Inf*ones(2,1);
opt = optimset(’fminunc’);
opt = optimset(opt,’Hessupdate’,’steepdesc’,’gradobj’,’on’,’Display’, ...
’iter’,’LargeScale’,’off’,’InitialHessType’,’identity’, ...
’tolfun’,1e-5’,’tolx’,1e-5,’MaxFunEvals’,100000,’MaxIter’, ...
10000,’OutputFcn’,@quartic_outfun);
x0 = [-1.9 2];
xout_gradsearch = fminunc(@quartic_usrfun,x0,opt);
xpath_gradsearch = xpath(1:iters,:);
xpath = zeros(10000,2);
iters = 0;
opt = optimset(’fminunc’);
opt = optimset(opt,’Hessupdate’,’bfgs’,’gradobj’,’on’,’Display’, ...
’iter’,’LargeScale’,’off’,’InitialHessType’,’identity’, ...
’tolfun’,1e-5,’tolx’,1e-5,’MaxFunEvals’,100000,’OutputFcn’,@ ...
quartic_outfun);
xout_bfgssearch = fminunc(@quartic_usrfun,x0,opt);
xpath_bfgssearch = xpath(1:iters,:);
xpath = zeros(10000,2);
iters = 0;
opt = optimset(’fminunc’);
opt = optimset(opt,’Hessupdate’,’dfp’,’gradobj’,’on’,’Display’, ...
’iter’,’LargeScale’,’off’,’InitialHessType’,’identity’, ...
’tolfun’,1e-5,’tolx’,1e-5,’MaxFunEvals’,100000,’OutputFcn’,@ ...
quartic_outfun);
xout_dfpsearch = fminunc(@quartic_usrfun,x0,opt);
Copyright © 2006–2012 Anil Vithala Rao Page 115
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
xpath_dfpsearch = xpath(1:iters,:);
x_ex = [1 1];
x_exact = [ones(length(xpath_gradsearch),1) ones(length(xpath_gradsearch),1)];
err_grad = xpath_gradsearch-x_exact;
err_grad = sqrt(dot(err_grad,err_grad,2));
x_exact = [ones(length(xpath_bfgssearch),1) ones(length(xpath_bfgssearch),1)];
err_bfgs = xpath_bfgssearch-x_exact;
err_bfgs = sqrt(dot(err_bfgs,err_bfgs,2));
x_exact = [ones(length(xpath_dfpsearch),1) ones(length(xpath_dfpsearch),1)];
err_dfp = xpath_dfpsearch-x_exact;
err_dfp = sqrt(dot(err_dfp,err_dfp,2));
figure(1);
hh1=plot(xpath_gradsearch(:,1),xpath_gradsearch(:,2),’o’);
hold on;
hh2=plot(x_ex(1),x_ex(2),’d’);
set(hh2,’MarkerSize’,12,’MarkerFaceColor’,’Black’);
print -depsc2 quartic_iterations.eps;
figure(2);
hh1=plot(xpath_bfgssearch(:,1),xpath_bfgssearch(:,2),’x’);
hold on;
hh2=plot(x_ex(1),x_ex(2),’d’);
set(hh2,’MarkerSize’,12,’MarkerFaceColor’,’Black’);
figure(3);
hh1=plot(xpath_dfpsearch(:,1),xpath_dfpsearch(:,2),’x’);
hold on;
hh2=plot(x_ex(1),x_ex(2),’d’);
set(hh2,’MarkerSize’,12,’MarkerFaceColor’,’Black’);
figure(4);
iters_grad =length(err_grad);
iters_bfgs =length(err_bfgs);
iters_dfp =length(err_dfp);
n_grad = 1:iters_grad;
Copyright © 2006–2012 Anil Vithala Rao Page 116
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
n_bfgs = 1:iters_bfgs;
n_dfp = 1;iters_dfp;
hh=plot(n_grad,err_grad,’-x’,n_bfgs,err_bfgs,’-o’,n_dfp,err_dfp,’-^’);
set(hh,’LineWidth’,1.5);
grid on;
xl=xlabel(’Iterations’);
yl=ylabel(’L_2-Norm Error’);
ll=legend(’Steepest Descent’,’DFP’,’BFGS’);
set(xl,’FontName’,’Times’,’FontSize’,16);
set(yl,’FontName’,’Times’,’FontSize’,16);
set(ll,’FontName’,’Times’,’FontSize’,16);
% Quartic Objective Function
function [F,G] = quartic_usrfun(x);
if size(x,1)==2,
x=x’;
end;
F = 100*(x(:,2)-x(:,1).^2).^2+(1-x(:,1)).^2;
G = [100*(4*x(1)^3-4*x(1)*x(2))+2*x(1)-2; 100*(2*x(2)-2*x(1)^2)];
return;
% Quartic Output Function
function stop = quartic_outfun(x,optimValues,state,varargin)
global xpath iters;
stop=[];
persistent history
persistent searchdir
hold on
switch state
case ’init’
history = []; searchdir = [];
case ’iter’
% Concatenate current point and objective function value
% with history. x must be a row vector.
history = [history;[x optimValues.fval]];
% Concatenate current search direction with searchdir.
Copyright © 2006–2012 Anil Vithala Rao Page 117
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
searchdir = [searchdir; optimValues.searchdirection’];
plot(x(1),x(2),’o’);
% Label points with iteration number.
text(x(1)+.15,x(2),num2str(optimValues.iteration));
iters = iters+1;
xpath(iters,:) = x;
case ’done’
assignin(’base’,’hist’, history);
assignin(’base’,’search’, searchdir);
otherwise
end
hold off
return;
Copyright © 2006–2012 Anil Vithala Rao Page 118
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Constrained Optimization
• Most Optimization Problems are Constrained
• Different Types of Constraints
⊲ Equality: constraints are active at optimal solution
⊲ Inequality: constraints may be either active or inactive at optimal
solution
• General form of constrained optimization problem:
minxF (x)
s.t.
g(x) = 0
h(x) ≤ 0
where g : Rn → Rm and h : Rn → R
p
• Above Optimization Problem is Called a Nonlinear Programming Problem
(NLP)
Copyright © 2006–2012 Anil Vithala Rao Page 119
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Following Discussion
⊲ Nonlinear equality-constrained problem (NEP)
⊲ Nonlinear inequality-constrained problem (NIP)
Copyright © 2006–2012 Anil Vithala Rao Page 120
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Equality Constraints
• Nonlinear Equality-Constrained Problem (NEP)
minxF (x)
s.t.
g(x) = 0
• Lagrangian Function
L(x,λ) = F (x) +
m∑
i=1
λigi(x)
• Define the following quantities:
⊲ Jacobian of the constraints as
A(x) =∂g
∂x∈ R
m×n
⊲ Hessian of the constraints, defined as
Hi(x) =∂2gi∂x2
∈ Rn×n, (i = 1, . . . ,m)
Copyright © 2006–2012 Anil Vithala Rao Page 121
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
where Ai(x) is the ith column of A(x)
⊲ A matrix Z(x) such that
Z(x∗) ∈ Rn×m =⇒ Z
T ∈ Rm×n
where the columns of Z(x∗) form a basis for the set of vectors
orthogonal to the rows of A(x∗)
⊲ The Hessian of Lagrangian: W(x,λ) = H(x)−∑m
i=1 λiHi ∈ Rn×n
Theorem 1 (Necessary Conditions for NEP)
1. g(x∗) = 0
2. ZT(x∗)G(x∗) = 0⇐⇒ G(x∗) = AT(x∗)λ∗
3. ZT(x∗)W(x∗,λ∗)Z(x∗) ≥ 0 (i.e., PSD)
Theorem 2 (Sufficient Conditions for NEP)
1. g(x∗) = 0
2. ZT(x∗)G(x∗) = 0⇐⇒ G(x∗) = AT(x∗)λ∗
3. ZT(x∗)W(x∗,λ∗)Z(x∗) > 0
Copyright © 2006–2012 Anil Vithala Rao Page 122
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Inequality Constraints
• Nonlinear Inequality-Constrained Problem (NIP)
minxF (x)
s.t.
g(x) = 0
h(x) ≤ 0
• Lagrangian Function
L(x,λ) = F (x) +
m∑
i=1
λigi(x) +
p∑
i=1
µihi(x)
• Define the following quantities at optimal solution:
⊲ q = number of active constraints
⊲ a(x∗) = vector of all active constraints
⊲ ν = vector of Lagrange multipliers corresponding to a(x∗)
• Furthermore, define the following quantities:
Copyright © 2006–2012 Anil Vithala Rao Page 123
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Jacobian of the active constraints as
A(x) =∂a
∂x∈ R
q×n
⊲ Hessian of the active constraints,
Hi(x) =∂2ai∂x2
∈ Rn×n, (i = 1, . . . , q)
where Ai(x) is the ith column of A(x)
⊲ A matrix Z(x) such that
Z(x∗) ∈ Rn×q =⇒ Z
T ∈ Rq×n
where the columns of Z(x∗) form a basis for the set of vectors orthogonal to
the rows of A(x∗)
⊲ The Hessian of the Lagrangian Function
W(x,λ) = H(x)−m∑
i=1
λiHi ∈ Rn×n
• Associate the Following Two Sets with the Inequality Constraints
⊲ A(x) = j | hj(x) = 0
⊲ A′(x) = j | hj(x) < 0
Copyright © 2006–2012 Anil Vithala Rao Page 124
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Theorem 3 (Necessary Conditions for NIP)
1. g(x∗) = 0 and h(x∗) ≤ 0
2. ZT(x∗)G(x∗) = 0⇔ G(x∗) = AT(x∗)λ∗
3. µ∗i ≥ 0
4. ZT(x∗W(x∗,λ∗)Z(x∗) ≥ 0
Theorem 4 (Sufficient Conditions for NIP)
1. g(x∗) = 0 and h(x∗) ≤ 0
2. ZT(x∗)G(x∗) = 0⇔ G(x∗) = AT(x∗)λ∗
3. µ∗i ≥ 0
4. ZT(x∗W(x∗,λ∗)Z(x∗) > 0
Copyright © 2006–2012 Anil Vithala Rao Page 125
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Numerical Solution of General NLP
• General NLP Must Be Solved Numerically
• Commonly Used Approaches for Solving NLPs
⊲ Sequential Quadratic Programming (SQP)
Approximate Lagrangian as a quadratic
Linearize Constraints
Make progress to optimal solution based on a merit function
⊲ Interior Point (Barrier)
Equality Constraints Enforced Using Quadratic Penalty Term
Inequality Constraints Managed via a Barrier Function
Sequence of Problems Solved Using Homotopy Parameter
• Notation
⊲ x: Optimization Variables
⊲ λ: Lagrange Multipliers of equality Constraints
⊲ µ: Lagrange Multipliers of inequality Constraints
Copyright © 2006–2012 Anil Vithala Rao Page 126
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Sequential Quadratic Programming (SQP)
• Choose an initial guess (x.λ,µ)
• Approximate L by a Quadratic, i.e.,
L(x,λ,µ) ≈ gTp+
1
2pTHp
• Linearize Constraints[
∂g
∂x
]
xk
pk = gk
[
∂h
∂x
]
xk
pk ≤ hk
• Equations Form a Quadratic Program (QP)
• QP Solved to Determine
⊲ Search direction, p
⊲ Estimates of active constraints and Lagrange multipliers
• Globalization Strategy: Use a Merit Function
⊲ Used to maintain progress toward optimal solution
Copyright © 2006–2012 Anil Vithala Rao Page 127
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ At optimal solution, merit function = cost function
⊲ Types of merit functions
Penalty functions
Barrier functions
Copyright © 2006–2012 Anil Vithala Rao Page 128
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Penalty Functions
• Let
c(x) =
g(x)
h(x)
• Penalty Function:
P (x, ρ) = F (x) +ρ
2cT(x)c(x)
where ρ is a penalty parameter.
Algorithm 4 (Penalty Function Method)
1. If xk satisfies the optimality conditions, then terminate.
2. Using the current iterate xk, solve the unconstrained subproblem
minx∈Rn
P (x, ρk)
3. Set ρk+1 > ρk, increment counter k → k + 1, and return to step 1
• Can be shown that
limρ→∞
x∗(ρ) = x
∗
Copyright © 2006–2012 Anil Vithala Rao Page 129
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Penalty Function Method (Continued)
• Issues with Standard Penalty Function
⊲ As ρ→∞ Unconstrained Problem Becomes Difficult to Solve
⊲ Need to Overcome Ill-Conditioning
• Augmented Lagrangian Penalty Function
P (x) = L(x, λ, µ) +ρ
2cT(x)c(x)
= F (x)− λTc(x) +
ρ
2cT(x)c(x)
• Improvements of Augmented Lagrangian Penalty Function
⊲ Avoids Ill-Conditioning of Penalty Function
⊲ Converges for Finite Value of ρ
⊲ However, Choosing “reasonable” Value of ρ is Difficult
Copyright © 2006–2012 Anil Vithala Rao Page 130
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Barrier Function Method
• Quadratic Penalty Functions
⊲ Generate Infeasible Iterates
⊲ Inappropriate When Necessary to Maintain Feasibility
⊲ Solution: Use a Feasible-Point Method
⊲ Barrier Method is Such an Approach
• Idea of a Barrier Function
⊲ Choose a Function that Prevents Infeasibilities
⊲ Common to Choose a Logarithmic Barrier Function, i.e.,
B(x, r) = F (x)− r
m∑
i=1
ln ci(x)
where r is the Barrier parameter
• Can Be Shown That
limr→0
x∗(r) = x
∗
Copyright © 2006–2012 Anil Vithala Rao Page 131
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Well-Known NLP Solvers
• NPSOL
⊲ Dense NLP Solver
⊲ Good for Problem Up To A Few Hundred Variables
• SNOPT
⊲ Sparse NLP Solver: Uses SQP Quasi-Newton Method
⊲ Capable of Solving Problems with Thousands of Variables & Constraints
• IPOPT
⊲ Also a Sparse NLP Solver: Uses a Barrier Method
⊲ Capable of Using Either Quasi-Newton or Newton Method
⊲ Capable of Solving Problems with Thousands of Variables & Constraints
• Will Show Examples Using SNOPT and IPOPT
Copyright © 2006–2012 Anil Vithala Rao Page 132
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example Using SNOPT
• Consider the Following Optimization Problem
minimize F (x) = (x1 − x2)2 + (x2 − x3)
3 + (x3 − x4)4 + (x4 − x5)
4
subject to
x3 + x4 + x5 ≥ 3
x1 + x22 + x33 = 3
x1 + x2 ≥ 1
x2 − x23 + x4 = 1
x1x5 = 1
• Constraint Jacobian and Objective Function Sparsity Pattern
Copyright © 2006–2012 Anil Vithala Rao Page 133
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
1 2 3 4 5
1 0 0 1 1 1
2 1 1 1 0 0
3 1 1 0 0 0
4 0 1 1 1 0
5 1 0 0 0 1
6 1 1 1 1 1
• Observations
⊲ 12 Out of 30 Derivatives Are Zero
⊲ Percentage of Nonzeros = 18/30 = 60%
• Can Take Advantage of Sparsity in SNOPT
• Problem Solved More Efficiently Using Sparse Linear Algebra
Copyright © 2006–2012 Anil Vithala Rao Page 134
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Code for SNOPT Example
• Main Function Codefunction [x,F,xmul,Fmul,INFO] = hsmain()
snprint(’hsmain.out’);
hsmain.spc = which(’hsmain.spc’);
snspec (hsmain.spc);
snseti(’Major Iteration limit’, 250);
[x,xlow,xupp,xmul,xstate,Flow,Fupp,Fmul,Fstate,ObjAdd,ObjRow, ...
A,iAfun,jAvar,iGfun,jGvar] = snoptExampleData;
[x,F,xmul,Fmul,INFO]= snsolve( x, xlow, xupp, xmul, xstate, ...
Flow, Fupp, Fmul, Fstate, ...
ObjAdd, ObjRow, A, iAfun, jAvar,...
iGfun, jGvar, ’snoptExampleFun’);
snprint off; % Closes the file and empties the print buffer
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [x,xlow,xupp,xmul,xstate,Flow,Fupp,Fmul,Fstate,ObjAdd,ObjRow, ...
A,iAfun,jAvar,iGfun,jGvar] = snoptexampledata()
neF = 6; % Number of Functions (Including Objective Function)
n = 5; % Number of Variables
Obj = 6; % Objective Function Row
ObjRow = 6; % Objective Function Row
% Assign the Sparsity Pattern for the Non-Constant Elements
% Column
Copyright © 2006–2012 Anil Vithala Rao Page 135
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
% | 1 2 3 4 5
% +----------------------
% 1 |
% 2 | 6 7
% 3 |
% 4 | 8
% 5 | 9 10
% row 6 | 1 2 3 4 5 Objective row
%
G = [ Obj, 1;
Obj, 2;
Obj, 3;
Obj, 4;
Obj, 5;
2, 2;
2, 3;
4, 3;
5, 1;
5, 5 ];
iGfun = G(:,1); jGvar = G(:,2);
% Assign the Sparsity Pattern for the Constant Derivatives
% Column
% | 1 2 3 4 5
% +----------------------
% 1 | 4 5 6
% 2 | 1
% 3 | 7 8
% 4 | 2 3
% 5 |
% row 6 | Objective row
%
A = [ 2, 1, 1;
4, 2, 1;
4, 4, 1;
1, 3, 1;
Copyright © 2006–2012 Anil Vithala Rao Page 136
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
1, 4, 1;
1, 5, 1;
3, 1, 1;
3, 2, 1 ];
iAfun = A(:,1); jAvar = A(:,2); A = A(:,3);
ObjAdd = 0;
% Initial Guess of Solution
x = [ 2;
sqrt(2) - 1 ;
sqrt(2) - 1 ;
2;
0.5 ];
% Set Lower and Upper Bounda on Variables and Constraints
xlow = -Inf*ones(n,1);
xupp = Inf*ones(n,1);
xstate = zeros(n,1);
xmul = zeros(n,1);
Flow = -Inf*ones(neF,1);
Fupp = Inf*ones(neF,1);
Flow(1) = 3;
Flow(2) = 3;
Fupp(2) = 3;
Flow(3) = 1;
Flow(4) = 1;
Fupp(4) = 1;
Flow(5) = 1;
Fupp(5) = 1;
Fmul = zeros(neF,1);
Fstate = zeros(neF,1);
Copyright © 2006–2012 Anil Vithala Rao Page 137
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Code to Compute Functionsfunction [F,G] = snoptExampleFun(x)
% Computes the Functions for the SNOPT Example
% Part 1: Define the Nonlinear Part of Functions and Derivatives for SNOPT Example
F = [ 0;
x(2)*x(2) + x(3)*x(3)*x(3);
0;
-x(3)*x(3);
x(1)*x(5);
(x(1)-x(2))^2 + (x(2)-x(3))^3 + (x(3)-x(4))^4 + (x(4)-x(5))^4 ];
% Part 2: Compute Derivatives of Functions
Obj = 6; % Objective Function is the 6th Row
G = [ Obj, 1, 2*(x(1)-x(2));
Obj, 2, 3*(x(2)-x(3))^2 - 2*(x(1)-x(2));
Obj, 3, 4*(x(3)-x(4))^3 - 3*(x(2)-x(3))^2;
Obj, 4, 4*(x(4)-x(5))^3 - 4*(x(3)-x(4))^3;
Obj, 5, -4*(x(4)-x(5))^3;
2, 2, 2* x(2);
2, 3, 3* x(3)^2;
4, 3, -2* x(3);
5, 1, x(5);
5, 5, x(1) ];
G = G(:,3);
Copyright © 2006–2012 Anil Vithala Rao Page 138
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example Using IPOPT
• Consider the Following Optimization Problem
minimize x3 + x1x4
3∑
i=1
xi
subject to the constraints
xi ≥ 1, (i = 1, . . . , 4)
xi ≤ 5, (i = 1, . . . , 4)
x1x2x3x4 ≥ 25∑4
i=1 x2i = 40
• To Use IPOPT in Newton Mode, Must Supply
⊲ Objective Function, Gradient, Constraints, and Jacobian
⊲ Hessian of NLP Lagrangian
⊲ Sparsity Pattern for Jacobian and Hessian
• This Example: Will Show How to Supply These Functions in IPOPT
Copyright © 2006–2012 Anil Vithala Rao Page 139
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Code for IPOPT Example
function [x, info] = exampleipopt
% Test IPOPT Problem Using Matlab Interface
% Reference: Hock and Schittowski (1981), Test Examples for
% Nonlinear Programming Codes. Lecture Nodes in Economics and
% Mathematical Systems, Vol. 187, Springer-Verlag.
x0 = [1 5 5 1]; % Initial Guess of Solution
xlb = [1 1 1 1]; % Lower Bound on Decision Variables
xub = [5 5 5 5]; % Upper Bound on Decision Variables
clb = [25 40]; % Lower Bound on Constraints
cub = [inf 40]; % Upper Bound on Constraints
options.lb = xlb;
options.ub = xub;
options.cl = clb;
options.cu = cub;
% Initialize the dual point.
options.zl = [1 1 1 1];
options.zu = [1 1 1 1];
options.lambda = [1 1];
% Set the IPOPT options.
options.ipopt.mu_strategy = ’adaptive’;
options.ipopt.tol = 1e-7;
% The callback functions.
funcs.objective = @objective;
funcs.constraints = @constraints;
funcs.gradient = @gradient;
funcs.jacobian = @jacobian;
funcs.jacobianstructure = @jacstructure;
funcs.hessian = @hessian;
funcs.hessianstructure = @hessstructure;
Copyright © 2006–2012 Anil Vithala Rao Page 140
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
% Run IPOPT.
[x info] = ipopt(x0,funcs,options);
% ----------------------------------------------------------------------
function f = objective(x)
f = x(1)*x(4)*sum(x(1:3)) + x(3);
% ----------------------------------------------------------------------
function g = gradient (x)
g = [ x(1)*x(4) + x(4)*sum(x(1:3))
x(1)*x(4)
x(1)*x(4) + 1
x(1)*sum(x(1:3)) ];
% ----------------------------------------------------------------------
function c = constraints(x)
c = [prod(x);sum(x.^2)];
% ----------------------------------------------------------------------
function J = jacobian(x)
J = sparse([prod(x)./x; 2*x]);
% ----------------------------------------------------------------------
function S = jacstructure
S = sparse(ones(2,4));
% ----------------------------------------------------------------------
function H = hessian (x, sigma, lambda)
H = sigma*[ 2*x(4) 0 0 0;
x(4) 0 0 0;
x(4) 0 0 0;
2*x(1)+x(2)+x(3) x(1) x(1) 0 ];
Copyright © 2006–2012 Anil Vithala Rao Page 141
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
H = H + lambda(1)*[ 0 0 0 0;
x(3)*x(4) 0 0 0;
x(2)*x(4) x(1)*x(4) 0 0;
x(2)*x(3) x(1)*x(3) x(1)*x(2) 0 ];
H = H + lambda(2)*diag([2 2 2 2]);
H = sparse(H);
function HS = hessstructure
HS = sparse(tril(ones(4)));
Copyright © 2006–2012 Anil Vithala Rao Page 142
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Summary
• Nonlinear Optimization
⊲ Foundation for Function Optimization
⊲ Deals with Discrete (Finite-Dimensional) Problem
⊲ Formulated as a Nonlinear Programming Problem (NLP)
⊲ Candidate Optimal Solutions are Called Extrema
• Determination of an Extreme Solution
⊲ Compute First Derivative of Augmented Cost Function
⊲ Set All Independent Terms in Variation to Zero
• In Practice, NLPs Solved Numerically via Iteration
⊲ SQP Methods
⊲ Barrier Methods
⊲ Trust-Region Methods
• Previously Developed Sophisticated NLP Software Has Been Described
Copyright © 2006–2012 Anil Vithala Rao Page 143
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part IV: Numerical Methods for Optimal Control
Copyright © 2006–2012 Anil Vithala Rao Page 144
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Overview
• Parts I and II of Course
⊲ Optimal Control Theory
⊲ Nonlinear Optimization
• This Part of Course
⊲ We Develop Numerical Methods Using Parts I and II
⊲ Numerical Methods Consist of Three Parts
Solution of Differential Equations
Application of Methods for Root-Finding
Application of Nonlinear Optimization
• Categories of Numerical Methods for Optimal Control
⊲ Indirect Methods: Combine Differential Equations with Root-Finding
⊲ Direct Methods: Combine Differential Equations with Nonlinear
Optimization
Copyright © 2006–2012 Anil Vithala Rao Page 145
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Overview of Indirect Methods
• Indirect Methods
⊲ Use Optimal Control Theory
⊲ Develop First-Order Optimality Conditions from Calculus of Variations
(Part I of Course)
⊲ Attempt to Solve Hamiltonian Boundary-Value Problem (HBVP)
⊲ Recall State-Costate Dynamics
(x, λ) = (Hλ,−Hx)
⊲ Boundary Conditions on State and Costate
φ(x(t0, t0,x(tf ), tf ) = 0
λ(t0) = − ∂Φ
∂x(t0)+
[
∂φ
∂x(t0)
]T
ν
λ(tf ) =∂Φ
∂x(tf )−[
∂φ
∂x(tf )
]T
ν
Copyright © 2006–2012 Anil Vithala Rao Page 146
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Overview of Direct Methods
• Calculus of Variation is Not Used =⇒ Optimality conditions are not derived
• Instead
⊲ State and/or Control Parametrized Using Function Approximation
⊲ Dynamics are Integrated Either Explicitly or Implicitly (Will Explain)
⊲ Cost functional Replaced with Integral Approximation (Quadrature)
⊲ Infinite-Dimensional Optimal Control Problem =⇒ Nonlinear
Programming Problem (NLP)
⊲ NLP solved by well-known methods
Copyright © 2006–2012 Anil Vithala Rao Page 147
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Commonly Used Indirect Methods
• Shooting Method
⊲ Guess Made of Unknown Initial and/or Terminal Conditions
⊲ State-Costate Dynamics Integrated from t0 to tf
⊲ Error in Initial and Terminal Boundary Conditions Assessed
⊲ Iterated Until All Boundary Conditions are Satisfied
• Multiple-Shooting Method
⊲ Time Interval Divided Into Segments
⊲ Guess Made of Unknown Initial Conditions at Start of Each Segment
⊲ State-Costate Dynamics Integrated on Each Segment
⊲ Errors at Each Segment Interface and Boundary Conditions Assessed
⊲ Iterated Until All Boundary Conditions are Satisfied
Copyright © 2006–2012 Anil Vithala Rao Page 148
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Indirect Shooting Method
• Consider the Following HBVP
p = G(p), p = (x,λ), G(p) = (Hλ,−Hx)
with boundary conditions
φ(p(t0), t0,p(tf ), tf ) = 0
• Observation: Do Not Have complete Set of Initial and/or Terminal Conditions
• Iterative Procedure (Indirect Shooting)
⊲ Guess Made of of p(t0) = p0 and, possibly, t0 and tf
⊲ Integrate p = G(p) From t0 to tf
⊲ At tf , have pf from Integration
⊲ Compute φ(p(t0), t0, p(tf ), tf ) and Assess Proximity to Zero
⊲ Iterate on Unknown Components of p(t0), t0, & tf Until ‖φ‖ < ǫ
• Observe That Error in φ Depends Upon p0; Can Solve Using Newton method
Copyright © 2006–2012 Anil Vithala Rao Page 149
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Schematic of Indirect Shooting Method
Target Bulls-Eye
Cannon
Copyright © 2006–2012 Anil Vithala Rao Page 150
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Appeal of Indirect Shooting Method
• Shooting is a Simple Root Finding Problem
• Number of Variables In Root-Finding Problem is Small
⊲ Only Number of Unknown Initial Conditions
⊲ Low-Dimensional Problem −→ Requires Little Memory on a Computer
• Why Not Always Use Indirect Shooting?
⊲ Hamiltonian Systems are Unstable in Either Direction of Time
⊲ Error in Unknown Initial Conditions Grows When Integrating t0 to tf
⊲ Even for Simple Problem, Indirect Shooting Can Break Down
• Next Slides: Example of Difficulties Using Indirect Shooting
Copyright © 2006–2012 Anil Vithala Rao Page 151
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Hamiltonian Boundary-Value Problem for Simple LQ Problem
• State-Costate Dynamics and Boundary Conditions
x
λ
=
a − b2
r
−q −a
x
λ
,
x(0) = x0
x(tf ) = xf
• General Solution of First-Order LTI System
p(t) = c1eµ1tw1 + c2e
µ2tw2
where
p =
x
λ
µ1, µ2 = Eigenvalues of A
w1,w2 = Eigenvectors of A
• Eigenvalues of System Matrix A
det [µI−A] = det
µ− a b2
r
q µ+ a
= µ2 − a2 − qb2
r= 0
Copyright © 2006–2012 Anil Vithala Rao Page 152
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Assuming That q > 0 and r > 0, We Obtain
µ = ±√
a2 +qb2
r−→ Real Eigenvalues with Opposite Sign
• Consider Solution for a = b = q = r = 1, t0 = 0, x(t0) = 1,
tf = (1, 5, 10, 15, 20), and x(tf ) = 2
0 5 10 15 200
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
t
x(t)
Fint=1Fint=5Fint=10Fint=15Fint=20
Copyright © 2006–2012 Anil Vithala Rao Page 153
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
0 5 10 15 20−1
−0.5
0
0.5
1
1.5
2
2.5
t
lam
bda(
t)
Fint=1Fint=5Fint=10Fint=15Fint=20
• Key points
⊲ Both state and costate have decaying and growing behavior
⊲ Growth and decay show up as tf increases
Copyright © 2006–2012 Anil Vithala Rao Page 154
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Solution of LQ Problem Using Indirect Shooting
• Two missing boundary conditions: λ(0) and λ(tf )
• Can develop iteration as follows:
⊲ Integrate state-costate dynamics either
From t0 to tf =⇒ need to iterate on λ(0)
From tf to t0=⇒ need to iterate on λ(tf )
⊲ Issue
A has both a positive and negative eigenvalue
State-costate dynamics are unstable in either direction of time
• Numerical Problems Arise When tf is Large
⊲ Growing Component of Dynamics Due to Eigenvalue at µ = +ω
⊲ Let ∆λ0 be Error in Guess of λ(0)
⊲ Then Error in Initial Condition Given as
∆p(0) =
0
∆λ0
Copyright © 2006–2012 Anil Vithala Rao Page 155
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Resulting Error at tf∆p(tf ) = eAtf∆p(0)
⊲ Now eAtf has a Mode That Grows Like eωt
⊲ Sufficiently large tf −→ eωtf will Result in Overflow on a Computer
⊲ Note: Overflow will Eventually occur tf Sufficiently Large, Even for
extremely small ∆λ(0)
⊲ Numerical Solution of a HBVP Can be Extremely Difficult
• Overcoming Difficulties of Indirect Shooting: Indirect
Multiple-Shooting Method
Copyright © 2006–2012 Anil Vithala Rao Page 156
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example: Orbit-Transfer Problem
• Hamiltonian Boundary-Value Problem
r = u
u =v2
r− µ
r2+T
msinα∗
v = −uvr
+T
mcosα∗
m = − T
g0Isp
λr = λu
[
v2
r2− 2µ
r3
]
− λvuv
r2
λu = −λr + λvv
r
λv = −λu2v
r+ λv
u
r
λm = λu
[
T
m2sinα∗
]
+ λv
[
T
m2cosα∗
]
Copyright © 2006–2012 Anil Vithala Rao Page 157
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
r(t0) = r0 , u(t0) = 0
v(t0) =√
µ/r0 , m(t0) = m0
r(tf ) = rf , u(tf ) = 0
v(tf ) =√
µ/rf , λm(tf ) = 0
• Notes:
⊲ Initial and Terminal Conditions are Incomplete
⊲ Half of the initial conditions and terminal conditions are specified
• Solved Numerically in MATLAB Using Indirect Shooting Method
Copyright © 2006–2012 Anil Vithala Rao Page 158
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Code for Orbit Transfer (Uses SNOPT)
% Minimum-Time Orbit Transfer Example
% Solves Hamiltonian Boundary-Value Problem
% Using SNOPT with Feasible Point Option
clear snoptcmex
clear all
close all
clc
warning off
mu = 1;
r0 = 1;
u0 = 0;
v0 = 1;
m0 = 1;
rf = 1.5;
uf = 0;
vf = sqrt(mu/rf);
lam0 = [-1; -1; 1; 1];
tf0 = 2;
guess = [lam0; tf0];
global xpath iters;
iters = 0;
xlow = -50*ones(5,1);
xupp = 50*ones(5,1);
Flow = [-Inf; zeros(4,1)];
Fupp = [+Inf; zeros(4,1)];
snset(’Feasible Point’);
userfun = ’orbitObjandCons’;
snscreen on
[xout,F,inform,xmul,Fmul] = snopt(guess,xlow,xupp,Flow,Fupp,userfun);
p0 = [r0; u0; v0; m0; xout(1:4)];
pf = [rf; uf; vf; xout(1:4); 0];
tspan = linspace(0,xout(5),40);
Copyright © 2006–2012 Anil Vithala Rao Page 159
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
odeopts = odeset(’RelTol’,1e-12,’AbsTol’,1e-12);
[tout,pout]=ode15s(@orbitode_bvp,tspan,p0,odeopts);
alpha = unwrap(atan2(pout(:,6),pout(:,7)))*180/pi;
orbitIndirectTraj = [tout pout alpha];
save orbitIndirectTraj.mat orbitIndirectTraj;
figure(1);
pp = plot(tout,pout(:,1),’-o’,tout,pout(:,2),’-d’,tout,pout(:,3),’-s’,tout,pout(:,4),’-v’);
set(pp,’LineWidth’,1.5,’MarkerSize’,6);
xl = xlabel(’Time’);
yl = ylabel(’States’);
ll = legend(’r’,’u’,’v’,’m’,’Location’,’NorthWest’);
set(xl,’FontSize’,12);
set(yl,’FontSize’,12);
set(ll,’FontSize’,16);
set(gca,’FontName’,’Times’,’FontSize’,12);
grid on;
print -depsc2 orbitRaisingIndirectShootingStates.eps
figure(2);
pp = plot(tout,pout(:,5),’-o’,tout,pout(:,6),’-d’,tout,pout(:,7),’-s’,tout,pout(:,8),’-v’);
set(pp,’LineWidth’,1.5,’MarkerSize’,6);
xl = xlabel(’Time’);
yl = ylabel(’Costates’);
ll = legend(’lamr’,’lamu’,’lamv’,’lamm’,’Location’,’NorthWest’);
set(xl,’FontSize’,12);
set(yl,’FontSize’,12);
set(ll,’FontSize’,16);
set(gca,’FontName’,’Times’,’FontSize’,12);
grid on;
print -depsc2 orbitRaisingIndirectShootingCostates.eps
figure(3);
pp = plot(tout,alpha,’-o’);
set(pp,’LineWidth’,1.5,’MarkerSize’,6);
xl = xlabel(’Time’);
yl = ylabel(’Control’);
Copyright © 2006–2012 Anil Vithala Rao Page 160
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
set(xl,’FontSize’,12);
set(yl,’FontSize’,12);
set(gca,’FontName’,’Times’,’FontSize’,12);
grid on;
print -depsc2 orbitRaisingIndirectShootingControl.eps
function C = orbitObjandCons(x);
mu = 1;
r0 = 1;
u0 = 0;
v0 = sqrt(mu/r0);
m0 = 1;
rf = 1.5;
uf = 0;
vf = sqrt(mu/rf);
pfinal = [rf; uf; vf];
pinitial =[r0; u0; v0; m0];
p = [r0; u0; v0; m0; x(1:4)];
target = [pfinal; 0];
tf = x(5);
tspan = linspace(0,tf,100);
opts = odeset(’RelTol’,1e-3);
[tout,pout]=ode15s(@orbitode_bvp,tspan,p,opts);
error = target-[pout(end,1:3) pout(end,8)].’;
C = [0; error];
function pdot = orbitode_bvp(t,p);
mu = 1;
T = 0.1405;
mass_flow_rate = 0.0749;
r = p(1);
u = p(2);
v = p(3);
m = p(4);
lamr = p(5);
lamu = p(6);
Copyright © 2006–2012 Anil Vithala Rao Page 161
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
lamv = p(7);
lamm = p(8);
%sinalpha = lamu./sqrt(lamu.^2+lamv.^2);
%cosalpha = lamv./sqrt(lamu.^2+lamv.^2);
alpha = atan2(lamu,lamv);
sinalpha = sin(alpha);
cosalpha = cos(alpha);
pdot = zeros(8,1);
rdot = u;
udot = v.^2./r-mu./r.^2+(T./m).*sinalpha;
vdot = -(u.*v)./r+(T./m).*cosalpha;
mdot = -mass_flow_rate;
lamrdot = lamu.*(v.^2./r.^2-2*mu./r.^3)-lamv.*(u.*v)./r.^2;
lamudot = -lamr+lamv.*v./r;
lamvdot = -lamu.*(2.*v./r)+lamv.*(u./r);
lammdot = lamu.*(T.*sinalpha./m.^2)+lamv.*(T.*cosalpha./m.^2);
pdot(1:8) = [rdot; udot; vdot; mdot; lamrdot; lamudot; lamvdot; lammdot];
Copyright © 2006–2012 Anil Vithala Rao Page 162
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
State Solution of Orbit Transfer Problem
0 0.5 1 1.5 2 2.5 3 3.50
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
ruvm
Sta
tes
t
Copyright © 2006–2012 Anil Vithala Rao Page 163
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Control Solution of Orbit Transfer Problem
0 0.5 1 1.5 2 2.5 3 3.50
50
100
150
200
250
300
350C
ontr
ol
t
Copyright © 2006–2012 Anil Vithala Rao Page 164
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Costate Solution of Orbit Transfer Problem
0 0.5 1 1.5 2 2.5 3 3.5−20
−15
−10
−5
0
5
10
15
20
25
λrλuλvλm
Cost
ate
s
t
Copyright © 2006–2012 Anil Vithala Rao Page 165
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
The Indirect Multiple-Shooting Method
• Basics of Multiple-Shooting Method
⊲ Break time interval into subintervals
⊲ Shoot over each subinterval
• Implementation of a Multiple-Shooting Method
⊲ Let Is = [ts, ts+1] , (s = 1, . . . , S)
⊲ Ensure following conditions
⋂S
s=1 Is = ∅⋂S
s=1 Is = [t0, tf ]
⊲ Integrate over each subinterval Is
⊲ Need to enforce continuity constraints at interior interfaces
x(t−s ) = x(t+s )
λ(t−s ) = λ(t+s )
Copyright © 2006–2012 Anil Vithala Rao Page 166
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Pros and Cons of Indirect Multiple-Shooting
• Pros
⊲ Integration Performed Over Shorter Time Intervals
⊲ Does Not Allow Hamiltonian Dynamics to “Blow Up”
⊲ Can Overcome Difficulties Associated with Indirect Shooting
⊲ Increases Range of Problems
• Cons
⊲ Need to guess values at beginning of each subinterval
p(t+s ), (s = 2, . . . , S − 1) Become Additional Variables in Problem
Total Set of Variables Increased With Each Added Subinterval
⊲ Many more variables and constraints in problem
⊲ Guessing Values of p(t+s ), (s = 2, . . . , S − 1) Usually Not Intuitive
⊲ Root-Finding Problem of Much Higher-Dimension Than Indirect
Shooting
Copyright © 2006–2012 Anil Vithala Rao Page 167
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Other General Problems with Indirect Methods
• Need to Derive First-Order Optimality Conditions
⊲ Costate Equations
⊲ Transversality Conditions
• Why is Deriving Such Conditions Problematic?
⊲ Functions May Be Very Complex
⊲ Some Functions May Not Even Have Analytic Forms (Table
Interpolation)
⊲ Becomes Very Difficult for Practical Problem
• Multiple-Phase Problems
⊲ Dynamics and Path Constraints May Be Different in Each Phase
⊲ Extremely Cumbersome to Derive Conditions for Complex Problem
• Bottom Line: Don’t Want to Have to Derive Optimality Conditions
Copyright © 2006–2012 Anil Vithala Rao Page 168
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Direct Methods
• Optimality conditions are not derived. Instead
⊲ State and/or Control Parametrized Using Function Approximation
⊲ Cost Approximated Using Quadrature
⊲ Optimal Control Problem Converted to an NLP
⊲ Infinite-dimensional problem ⇐⇒ finite-dimensional problem
• Direct Methods Divided Into Two General Categories
⊲ Control Parametrization: Shooting and Multiple-Shooting
⊲ State & Control Parametrization: Collocation
Copyright © 2006–2012 Anil Vithala Rao Page 169
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Direct Methods
Control Parameterization State & Control Parameterization
Shooting Multiple-Shooting Local Collocation Global Collocation
OrthogonalCollocation
Implicit/ExplicitIntegration
PseudospectralMethods
• Control Parametrization: Simple, But Not As Capable
• State & Control Parametrization: More Capable, But More Involved
• Will Discuss Both Types of Methods
Copyright © 2006–2012 Anil Vithala Rao Page 170
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Direct Shooting Method
• Step 1: Choose a Function Approximation for Control
u(t) =
N∑
k=0
ckψk(t)
• Basis Functions ψ0(t), . . . , ψN (t) Arbitrary
• Usually Choose Polynomials
• Step 2: Guess Unknown Coefficients (c0, . . . , cN )
• Step 3: Integrate Dynamics from t0 to tf
• Iterate on Coefficients Until Cost Minimized and Constraints Satisfied
Copyright © 2006–2012 Anil Vithala Rao Page 171
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example: Linear-Tangent Steering Problem
• Cost Function: J = tf
• Dynamics
x1 = x3
x2 = x4
x3 = a cos(u)
x4 = a sin(u)
• Initial Conditions: (x1(0), x2(0), x3(0), x4(0)) = (0, 0, 0, 0)
• Terminal Conditions: (x2(tf ), x3(tf ), x4(tf )) = (5, 45, 0)
Copyright © 2006–2012 Anil Vithala Rao Page 172
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Code for Linear Tangent Problem
global CONSTANTS
CONSTANTS.polynomialDegree = 3;
CONSTANTS.a = 100;
x10 = 0;
x20 = 0;
x30 = 0;
x40 = 0;
x2f = 5;
x3f = 45;
x4f = 0;
tfMin = 0.01;
tfMax = 100;
CONSTANTS.initialState = [x10; x20; x30; x40];
CONSTANTS.terminalState = [x2f; x3f; x4f];
tfGuess = 10;
z0 = [ones(CONSTANTS.polynomialDegree+1,1); tfGuess];
zlow = [-10*ones(CONSTANTS.polynomialDegree+1,1); tfMin];
zupp = [+10*ones(CONSTANTS.polynomialDegree+1,1); tfMax];
Flow = [tfMin; zeros(size(CONSTANTS.terminalState))];
Fupp = [tfMax; zeros(size(CONSTANTS.terminalState))];
snscreen on
snprint(’snoptmain.out’);
snseti(’Derivative Option’,0);
snseti(’Print Level’,10);
[z,F,inform,zmul,Fmul] = snopt(z0,zlow,zupp,Flow,Fupp,’linearTangentObjandCons’);
p0 = CONSTANTS.initialState;
tspan = [0 z(end)];
opts = odeset(’RelTol’,1e-3);
[tout,pout]=ode45(@linearTangentOde,tspan,p0,opts,z(1:end-1));
uout = polyval(z(1:end-1),tout);
figure(1);
Copyright © 2006–2012 Anil Vithala Rao Page 173
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
pp = plot(tout,pout(:,1),’-o’,tout,pout(:,2),’-d’,tout,pout(:,3),’-s’,tout,pout(:,4),’-v’);
set(pp,’LineWidth’,1.5,’MarkerSize’,8);
xl = xlabel(’Time’);
yl = ylabel(’States’);
ll = legend(’State1(t)’,’State2(t)’,’State3(t)’,’State4(t)’,’Location’,’NorthWest’);
set(xl,’FontSize’,12);
set(yl,’FontSize’,12);
set(ll,’FontSize’,16);
set(gca,’FontName’,’Times’,’FontSize’,12);
grid on;
print -depsc2 linearTangentDirectShootingState.eps
figure(2);
pp = plot(tout,uout,’-o’);
set(pp,’LineWidth’,1.5,’MarkerSize’,8);
xl = xlabel(’Time’);
yl = ylabel(’Control’);
set(xl,’FontSize’,12);
set(yl,’FontSize’,12);
set(gca,’FontName’,’Times’,’FontSize’,12);
grid on;
print -depsc2 linearTangentDirectShootingControl.eps
function C = linearTangentObjandCons(z);
global CONSTANTS
controlCoefs = z(1:end-1);
tf = z(end);
p0 = CONSTANTS.initialState;
tspan = [0 tf];
opts = odeset(’RelTol’,1e-3);
[tout,pout]=ode45(@linearTangentOde,tspan,p0,opts,controlCoefs);
C = [tf; pout(end,2:4).’-CONSTANTS.terminalState];
function pdot=linearTangentOde(t,p,c);
Copyright © 2006–2012 Anil Vithala Rao Page 174
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
global CONSTANTS
u = 0;
u = polyval(c,t);
p1dot = p(3);
p2dot = p(4);
p3dot = CONSTANTS.a*cos(u);
p4dot = CONSTANTS.a*sin(u);
pdot = [p1dot; p2dot; p3dot; p4dot];
Copyright © 2006–2012 Anil Vithala Rao Page 175
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Direct Shooting Solution of Linear-Tangent Problem
0 0.1 0.2 0.3 0.4 0.5 0.6 0.70
5
10
15
20
25
30
35
40
45
50
x1(t)x2(t)x3(t)x4(t)
Sta
tes
t
Copyright © 2006–2012 Anil Vithala Rao Page 176
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7−1.5
−1
−0.5
0
0.5
1
1.5
Contr
ol
t
Copyright © 2006–2012 Anil Vithala Rao Page 177
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example: Direct Shooting Solution of Orbit-Raising Problem
• Will Formulate Problem Using Two Control Parametrization
• Parametrization 1: Global Polynomial
β(t) ≈5∑
k=0
cktk −→ Degree 5 Polynomial
• Parametrization 2: Arc-tangent of Ratio of Two Polynomials
β(t) ≈ tan−1(n(t), d(t))
n(t) =∑5
k=0 nktk
d(t) =∑5
k=0 dktk
−→ Ratio of Two Degree 5 Polynomials
• Objective of Comparison
⊲ Show How Quality of Direct Shooting Solution Depends on Control
Parametrization
⊲ Need Some Insight to Use Appropriate Control Parametrization
Copyright © 2006–2012 Anil Vithala Rao Page 178
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Control Solution to Orbit-Raising Problem Using Polynomial Control
0 0.5 1 1.5 2 2.5 3 3.5−150
−100
−50
0
50
100
150
200
250
300
350
Direct
Indirect
Contr
ol
t
Copyright © 2006–2012 Anil Vithala Rao Page 179
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Control Solution to Orbit-Raising Problem Using tan−1 Control
0 0.5 1 1.5 2 2.5 3 3.50
50
100
150
200
250
300
350
Direct
Indirect
Contr
ol
t
Copyright © 2006–2012 Anil Vithala Rao Page 180
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Code for Direct Shooting Solution of Orbit-Raising Problem
clear snoptcmex
clear all
close all
global CONSTANTS numseg breaks nstates nterminal numcons degree control_type
control_type = ’polynomial’;
% control_type = ’arctangent’;
degree = 5;
numseg = 1;
nstates = 4;
nterminal = 3;
numcons = (numseg-1)*nstates+nterminal;
numvars = (numseg-1)*nstates+numseg*(degree+1)+1;
width = 1/numseg;
breaks = width:width:1;
CONSTANTS.mu = 1;
CONSTANTS.T = 0.1405;
CONSTANTS.Ve = 1.8758;
r0 = 1; rf = 1.5;
u0 = 0; uf = 0;
v0 = 1; vf = sqrt(CONSTANTS.mu/rf);
m0 = 1;
rmin = 1;
rmax = 2;
umin = -0.1;
umax = 0.5;
vmin = 0.5;
vmax = 1.1;
mmax = 1.05;
mmin = 0.5;
stateLow = [rmin; umin; vmin; mmin];
stateUpp = [rmax; umax; vmax; mmax];
tfMin = 2;
Copyright © 2006–2012 Anil Vithala Rao Page 181
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
tfMax = 3.6;
CONSTANTS.initialState = [r0; u0; v0; m0];
CONSTANTS.terminalState = [rf; uf; vf];
tfGuess = 3;
coefsGuess = [zeros(degree,1); 0.2];
z0 = coefsGuess;
coefMax = 3;
zlow = -coefMax*ones(degree+1,1);
zupp = coefMax*ones(degree+1,1);
Flow = [];
Fupp = [];
rr = [0.5; 0.5; 0.5; 0.5];
for i=2:numseg;
z0 = [z0; rr; coefsGuess];
zlow = [zlow; stateLow; -coefMax*ones(degree+1,1)];
zupp = [zupp; stateUpp; coefMax*ones(degree+1,1)];
Flow = [Flow; zeros(nstates,1)];
Fupp = [Fupp; zeros(nstates,1)];
end;
z0 = [z0; tfGuess];
zlow = [zlow; tfMin];
zupp = [zupp; tfMax];
Flow = [Flow; zeros(nterminal,1)];
Fupp = [Fupp; zeros(nterminal,1)];
Flow = [-inf; Flow];
Fupp = [inf; Fupp];
snscreen on
if isequal(control_type,’polynomial’),
snprint(’snoptmainShootingPolynomial.out’);
else
snprint(’snoptmainShootingArctangent.out’);
end;
snsetr(’Optimality Tolerance’,1e-5);
snsetr(’Feasibility Tolerance’,1e-5);
snseti(’Derivative Option’,0);
snseti(’Print Level’,10);
ObjAdd = 0;
Copyright © 2006–2012 Anil Vithala Rao Page 182
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
ObjRow = 1;
A = [];
iAfun = []; jAvar = [];
JJ = ones(numcons+1,numvars);
[iGfun,jGvar]=find(JJ);
userfun = ’orbitRaisingObjandCons’;
[z,F,inform,zmul,Fmul] = snopt(z0,zlow,zupp,Flow,Fupp,userfun,0,1,A,iAfun,jAvar,iGfun,jGvar);
p0 = CONSTANTS.initialState;
tfOptimal = z(end);
tfuse = tfOptimal*breaks;
opts = odeset(’RelTol’,1e-10);
istart = 1;
ifinish = degree+1;
controlCoefs = z(istart:ifinish);
tspan = [0:tfuse(1):tfuse(1)];
[tout,pout]=ode45(@orbitRaisingOde,tspan,p0,opts,controlCoefs);
ttot = tout;
ptot = pout;
if isequal(control_type,’polynomial’),
utot = polyval(controlCoefs,ttot);
else
lc = length(controlCoefs);
numerator = polyval(controlCoefs(1:lc/2),tout);
denominator = polyval(controlCoefs(lc/2+1:end),tout);
utot = atan2(numerator,denominator);
end;
if 0,
for jj=2:numseg;
istart = ifinish+1;
ifinish = istart+nstates+degree;
zuse = z(istart:ifinish);
p0 = zuse(1:nstates);
controlCoefs = zuse(nstates+1:nstates+degree+1);
tspan = [tfuse(jj-1):(tfuse(jj)-tfuse(jj-1))/10:tfuse(jj)];
[tout,pout]=ode45(@orbitRaisingOde,tspan,p0,opts,controlCoefs);
% uout = polyval(controlCoefs,tout);
numerator = polyval(controCoefs(1:lc/2,tout));
Copyright © 2006–2012 Anil Vithala Rao Page 183
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
denominator = polyval(controCoefs(lc/2+1:end,tout));
uout = atan2(numerator,denominator);
% uout = polyval(controlCoefs,tout);
ttot = [ttot; tout];
ptot = [ptot; pout];
utot = [utot; uout];
end;
end;
utot = unwrap(utot)*180/pi;
figure(1);
pp = plot(ttot,ptot(:,1),’-o’,ttot,ptot(:,2),’-d’,ttot,ptot(:,3),’-s’,ttot,ptot(:,4),’-v’);
set(pp,’LineWidth’,1.5,’MarkerSize’,8);
xl = xlabel(’Time’);
yl = ylabel(’States’);
ll = legend(’r’,’u’,’v’,’m’,’Location’,’Northwest’);
set(xl,’FontSize’,12);
set(yl,’FontSize’,12);
set(ll,’FontSize’,16);
set(gca,’FontName’,’Times’,’FontSize’,12);
grid on;
if isequal(control_type,’polynomial’),
print -depsc2 orbitRaisingDirectShootingStatePolynomial.eps
else
print -depsc2 orbitRaisingDirectShootingStateArctangent.eps
end;
currdir = pwd;
cd(’../../../Indirect/orbitRaising’);
load orbitIndirectTraj.mat;
cd(currdir);
tindirect = orbitIndirectTraj(:,1);
uindirect = orbitIndirectTraj(:,end);
figure(2);
pp = plot(ttot,utot,’-o’,tindirect,uindirect,’-d’);
set(pp,’LineWidth’,1.5,’MarkerSize’,8);
xl = xlabel(’Time’);
Copyright © 2006–2012 Anil Vithala Rao Page 184
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
yl = ylabel(’Control’);
ll = legend(’Direct’,’Indirect’,’Location’,’Northwest’);
set(xl,’FontSize’,12);
set(yl,’FontSize’,12);
set(ll,’FontSize’,16);
set(gca,’FontName’,’Times’,’FontSize’,12);
grid on;
if isequal(control_type,’polynomial’),
print -depsc2 orbitRaisingDirectShootingControlPolynomial.eps
else
print -depsc2 orbitRaisingDirectShootingControlArctangent.eps
end;
function C = orbitRaisingObjandCons(z);
global CONSTANTS numseg breaks nstates nterminal numcons degree
tf = z(end);
tfuse = tf*breaks;
C = zeros(numcons+1,1);
opts = odeset(’RelTol’,1e-10);
p0 = CONSTANTS.initialState;
istart = 1;
ifinish = degree+1;
controlCoefs = z(istart:ifinish);
tspan = [0 tfuse(1)];
[tout,pout]=ode45(@orbitRaisingOde,tspan,p0,opts,controlCoefs);
lastStateSave = pout(end,:).’;
lastDerivativeSave = orbitRaisingOde(tspan(end),pout(end,:).’,controlCoefs);
err = [];
derr = [];
for jj=2:numseg;
istart = ifinish+1;
ifinish = istart+nstates+degree;
zuse = z(istart:ifinish);
p0 = zuse(1:nstates);
err(:,jj-1) = p0-lastStateSave;
Copyright © 2006–2012 Anil Vithala Rao Page 185
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
controlCoefs = zuse(nstates+1:nstates+degree+1);
tspan = [tfuse(jj-1) tfuse(jj)];
dp0 = orbitRaisingOde(tspan(1),p0,controlCoefs);
% derr(:,jj-1) = dp0-lastDerivativeSave;
[tout,pout]=ode45(@orbitRaisingOde,tspan,p0,opts,controlCoefs);
lastStateSave = pout(end,:).’;
lastDerivativeSave = orbitRaisingOde(tspan(end),pout(end,:).’,controlCoefs);
end;
errtot = [err; derr];
Errorf = pout(end,1:3).’-CONSTANTS.terminalState;
C = [tf; errtot(:); Errorf];
function pdot=orbitRaisingOde(t,p,c);
global CONSTANTS control_type
r = p(1); u = p(2); v = p(3); m = p(4);
if isequal(control_type,’polynomial’),
alpha = polyval(c,t);
else
lc = length(c);
numerator = polyval(c(1:lc/2),t);
denominator = polyval(c(lc/2+1:end),t);
alpha = atan2(numerator,denominator);
end;
rdot = u;
udot = v^2/r-CONSTANTS.mu/r^2+CONSTANTS.T*sin(alpha)/m;
vdot = -u*v/r +CONSTANTS.T*cos(alpha)/m;
mdot = -CONSTANTS.T/CONSTANTS.Ve;
pdot = [rdot; udot; vdot; mdot];
Copyright © 2006–2012 Anil Vithala Rao Page 186
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Lessons Learned from Direct Shooting
• Pros
⊲ No Need to Derive Complicated Optimality Conditions
⊲ Integrate Only State Dynamics, Not Costate Dynamics
⊲ Coding is Very Straightforward
• Cons
⊲ Solution Very Sensitive to Choice of Control
⊲ Without Insight Into Problem, Low-Quality Solution Produced
• Modification of Direct Shooting: Direct Multiple Shooting
Copyright © 2006–2012 Anil Vithala Rao Page 187
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Direct Multiple-Shooting
• Let t0=Start Time of Phase and tf=Terminus Time of Phase
• Divide Trajectory Into Segments, t0 < t1 < t2 < · · · < tN−1 < tN where
tN = tf
• Points t0, t1, t2, . . . , tN Called Grid Points or Mesh Points
• Parametrize the Control in Each Segment As
u(s)(t) =
N∑
k=0
c(s)k ψ
(s)k (t)
• Connect Segments with Continuity Constraint on State
x(t−s ) = x(t+s ), s = 1, . . . , S − 1
• Total Cost = Sum of Cost in Each Segment
• Enforce Boundary Conditions in First and Last Segment
Copyright © 2006–2012 Anil Vithala Rao Page 188
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Schematic of Multiple-Shooting
Direct Shooting PerformedOn Each Subinterval
State Forced To Be Continuous At Each Segment Interface
Sta
te
Time
Segment Break Times
Copyright © 2006–2012 Anil Vithala Rao Page 189
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example: Orbit-Raising Problem
• Same Problem as Solved Earlier Using Indirect and Direct Shooting
• In This Example We Choose
⊲ Number of Segments = 10
⊲ Degree of Polynomial Approximation in Each Segment = 3
• Problem Divided Into Equal Width Segments
Copyright © 2006–2012 Anil Vithala Rao Page 190
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Direct Multiple-Shooting Solution to Orbit-Raising Problem
0 0.5 1 1.5 2 2.5 3 3.50
50
100
150
200
250
300
350
C
ontr
ol
t
Direct
Indirect
Copyright © 2006–2012 Anil Vithala Rao Page 191
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Comments on Direct Multiple-Shooting
• Improves Solution Over Naively Implemented Direct Shooting
• More Complex Than Direct Shooting
⊲ Need to Add Continuity Constraints on State at Segment Interfaces
⊲ Increases the Size of the Problem (More Variables, More Constraints)
⊲ May Obtain Discontinuous Control (Even if True Optimal Control is
Continuous)
• Benefits of Direct Multiple Shooting
⊲ Improves the Quality of the Control over Direct Shooting
⊲ Allows for Possible Control Discontinuities
Copyright © 2006–2012 Anil Vithala Rao Page 192
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Limitations with Shooting Methods
• Generally Produces Low-Accuracy Solutions
• Computationally Inefficient
• May Not Converge to Solution for Inappropriate Control Approximations
• Serious Limiting Factors: Inclusion of Constraints
• Control Constraints: Difficult to Place Bounds on Control
• State Path Constraints
⊲ Need to Formulate Problem Using Differential-Algebraic Equations
(DAEs)
⊲ DAEs difficult to Solve via Explicit Integration
⊲ Path Constraints Difficult to Include in Problem
Copyright © 2006–2012 Anil Vithala Rao Page 193
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Added Complexity: Phases
• Real Optimal Control Problems are Modeled Using Phases
• What is a Phase? ⇒ a Distinct Chunk of the Trajectory
• How are Phases Implemented?
⊲ Divide Problem into Phases
⊲ Each Phase has
Its Own Dynamics and Path Constraints
Dynamics May Be Different from Phase to Phase
⊲ Phases are Linked Using Continuity Conditions on State
ψmin ≤ ψ(psf , t
sf ,p
s+10 , ts+1
0 ) ≤ ψmax
where s ∈ [1, . . . , S] are the phases in the problem
• Why Phases Are Important
⊲ Consider the Problem of a Two-Stage Launch Vehicle
⊲ Stage 1: First Engine Burns; Dry Mass Jettisoned at End of Stage
⊲ Stage 2: Second Engine Burns; Dry Mass Jettisoned at End of Stage
Copyright © 2006–2012 Anil Vithala Rao Page 194
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Connection Between Stages: Position and Velocity Continuous; Mass
Discontinuous
⊲ Problem Cannot Be Solved Using a Single Phase
• Shooting Methods Become Even More Complex With Multiple-Phase
Problems
• Need Approach That Scales to Complex Problems
• More Sophisticated Approach: Direct Collocation
Copyright © 2006–2012 Anil Vithala Rao Page 195
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Direct Collocation Methods for Optimal Control
• Generally, Structure of Optimal Control Not Known a Priori
• Would Like a Computationally Efficient Method That Is Flexible
⊲ Does Not Require an a Priori Knowledge of Control
⊲ Provides Greater Accuracy As Compared with Shooting
• More Sophisticated Class of Direct Methods: Collocation
⊲ Collocation: State and Control Parameterization Methods
⊲ Allows for Very Flexible Formulation of Optimal Control Problem
Multiple Phases
Path Constraints (Equality and Inequality)
• In Collocation, Dynamics are Not Propagated
⊲ Instead, Differential Equations Converted to Algebraic Constraints
⊲ Leads to NLP: min f(z) subject to g(z) = 0 and hmin ≤ h(z) ≤ hmax
⊲ Entire System Then Optimized Simultaneously for State and Control
Copyright © 2006–2012 Anil Vithala Rao Page 196
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example of Collocation: Euler Forward
• Euler Forward Integration: Given (tk, uk, xk), Determine xk+1
xk+1 = xk + hf(xk, uk, tk)
• However, We Could Write Euler Forward As Follows
x1 = x0 + hf(x0, u0, t0)
x2 = x1 + hf(x1, u1, t1)
...
xM = xM−1 + hf(xM−1, uM−1, tM−1)
• Rearranging, We Have a System of Simultaneous Equations
x1 − x0 − hf(x0, u0, t0) = 0
x2 − x1 − hf(x1, u1, t1) = 0
...
xM − xM−1 − hf(xM−1, uM−1, tM−1) = 0
Copyright © 2006–2012 Anil Vithala Rao Page 197
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Approximation of Cost
J ≈ Φ(x0, t0, xM , tM ) + h
M−1∑
k=1
L[xi, ui, ti]
• Boundary Conditions
φ(x0, t0, xM , tM ) = 0
• Result: We End Up with an NLP of the Form
minimize f(z)
Subject to
g(z) = 0
• NLP Variables: z = (x0, . . . , xM , u0, . . . , uM−1)
• NLP Constraints
⊲ System of Equations Resulting from Euler Discretization
⊲ Boundary Conditions
Copyright © 2006–2012 Anil Vithala Rao Page 198
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Euler Forward Provides Key Features of Collocation
• M=Number of Intervals ≈ Number of Time Steps
• As M Gets Large
⊲ Problem Divided Into Increasingly More Subintervals
⊲ Number of NLP Variables ≈ 2M
• More General Result
⊲ Suppose We Have n States and m Controls
⊲ Number of NLP Variables ≈M(n+m)
⊲ Suppose M = 1000, n = 6, and m = 3 −→M(n+m) ≈ 9000
• Complex Problem
⊲ Thousands to Tens of Thousands Variables and Constraints
⊲ NLP Using Direct Collocation is Huge
• Question: Is Direct Collocation Computationally Viable?
Copyright © 2006–2012 Anil Vithala Rao Page 199
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Why Use a Collocation Method?
• At First Glance
⊲ Seems As If We’ve Made Problem Harder
⊲ Many Equations in Many Unknowns
• As It Happens, Collocation Methods Are
⊲ Computationally Efficient
⊲ Robust to Initial Guesses
• Why?
⊲ Advanced NLP Solvers Designed to Solve Large Sparse Problems
⊲ Using Collocation Leads to a Large Sparse NLP
• What is a Sparse NLP?
⊲ Problem May Have Many Variables and Constraints
⊲ But Derivatives of Functions Have Large Number of Zeros
⊲ Sparsity Makes It Possible to Develop Efficient Iteration Methods
Copyright © 2006–2012 Anil Vithala Rao Page 200
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Collocation Formulation
• Divide Phase into Segments
t0 < t1 < t2 < · · · < tN−1 < tN
where tN = tf
• Points t0, t1, t2, . . . , tN Called Grid Points or Mesh Points
• Now Consider a Segment Is = [tk, tk+1]
⊲ Treat state and control as variables in discretization
⊲ Differential equations replaced by algebraic approximations
⊲ All constraints will have the general form
cL ≤ c(y) ≤ cU
where
c(y) =[
ζ1, ζ2, . . . , ζM−1,φ,g1, . . . ,gM
]
• Note that y = Vector of Variables in Discretized Problem
Copyright © 2006–2012 Anil Vithala Rao Page 201
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Common Discretization Schemes
• Euler Method
⊲ Variables: y = (x1,u1, . . . ,xM ,uM )
⊲ Defects:
ζk = xk+1 − xk − hkfk
• Classical Runge-Kutta Method
⊲ Variables: (x1,u1,u2, . . . , uM ,xM ,uM )
⊲ Defects
ζk = xk+1 − xk − 1
6(k1 + 2k2 + 2k2 + k3)
wherek1 = hkfk
k2 = hkf(
xk + 12k1, uk+1, tk + 1
2hk
)
k3 = hkf(
xk + 12k2, uk+1, tk + 1
2hk
)
k4 = hkf (xk + k3,uk+1, tk + hk)
uk+1 = 12(uk + uk+1)
• Hermite-Simpson Method
Copyright © 2006–2012 Anil Vithala Rao Page 202
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Variables: y = (x1,u1, u2, . . . , uM ,xM ,uM )
⊲ Defects:
ζk = xk+1 − xk − 1
6hk
(
fk + 4fk+1 + fk+1
)
where
xk+1 = 12(xk + xk+1) +
18hk (fk − fk+1)
fk+1 = f(
xk+1, uk+1, tk + 12hk
)
Copyright © 2006–2012 Anil Vithala Rao Page 203
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Variable-Length Phase Durations
• Phase Duration May Be a Variable, i.e., t0 and tf May Not Be Fixed
• For Variable-Length Phase,
⊲ Need to include t0 and/or tf as variables in discretization
⊲ Discretization step must be altered such that
hk = τk (tf − t0) = τk∆t
where ∆t = tf − t0 and 0 < τk < 1
⊲ Essentially, t0 and tf change like an “accordion”
Copyright © 2006–2012 Anil Vithala Rao Page 204
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Highlights of Direct Approach
• Continuous-Time Dynamics =⇒ Algebraic Equations
• Algebraic Equations Have Defects
• Objective: Make Defects Zero
• In Other Words, Defects are Equality Constraints
• Problem is Transcribed to an NLP
• Basic Structure of NLP
⊲ Minimize
f(z)
subject to
g(z) = 0
hmin ≤ h(z) ≤ hmax
=⇒ Contains Disretized Dynamics
Path Constraints, and Boundary Conditions
Copyright © 2006–2012 Anil Vithala Rao Page 205
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Other Positive Attributes of Collocation Methods
• Inclusion of Path Constraints Much Easier
• To See This, Consider Euler Collocation Method Again
• Recall NLP Variables in Euler Scheme: z = (x0, . . . , xM , u0, . . . , uM−1)
• Suppose We Have Path Constraint C(x,u) ≤ 0
• Then We Can Enforce Path Constraint at Collocation Points As
C(xi, ui) ≤ 0, (i = i, . . . ,M − 1)
• Observations
⊲ Path Constraints No More Complicated Than Dynamic Constraints
⊲ Can Be Inequalty or Equality
⊲ Does Not Require Ad Hoc Procedure to Include
Copyright © 2006–2012 Anil Vithala Rao Page 206
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Trade-Off Between Indirect and Direct Methods
• Indirect Methods
⊲ Beautiful mathematics (calculus of variations)
⊲ If solvable, HBVP produces state, control, and costate
⊲ Can gain great insight into structure of solution (e.g., LQR problem)
⊲ We know how close to optimal is the solution (because we have costate)
⊲ However,
Small radii of convergence
Difficult to solve HBVP
Often, cannot even derive optimality conditions
Indirect methods are generally intractable
• Direct Methods
⊲ Mathematics not so elegant (finite-difference methods)
⊲ No costate produced from solution to NLP
⊲ Difficult to gain insight into structure of solution
⊲ Do not know how close we are to “true” optimal solution
Copyright © 2006–2012 Anil Vithala Rao Page 207
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ However,
Large radii of convergence
Much easier to solve NLP than to solve HBVP
No need to derive optimality conditions
Direct methods are highly tractable
• So What is “Ideal”
⊲ Want Direct Method
⊲ Want Direct Method to Have Features of Indirect Method
In particular, want an accurate costate
Then we effectively have same information as an indirect method
⊲ Question: Do Such Methods Exist?
⊲ Answer: Yes
⊲ Question: What Are These Methods?
⊲ Answer: Pseudospectral Methods
Copyright © 2006–2012 Anil Vithala Rao Page 208
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part V: Numerical Integration & Interpolation
Copyright © 2006–2012 Anil Vithala Rao Page 209
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Numerical Quadrature and Interpolation
• Numerical Quadrature
⊲ Need to Accurately Approximate Integral of a Function
⊲ Quadrature Approximations
Sample Function at Points in the Interval (Support Points)
Use a Weighted Sum of These Points (Quadrature Weights)
⊲ Generally Quadrature Approximations Done on τ ∈ [−1,+1].
∫ +1
−1
f(τ)dτ =N∑
i=1
wif(τi)
⊲ Interpolation
Function Values Given at Discrete Points (τ1, . . . , τN )
Need to Construct Continuous Approximation
f(τ) ≈ F (τ) =
N∑
i=1
ciψi(τ)
ψi(τ) are Trial or Basis Functions
Copyright © 2006–2012 Anil Vithala Rao Page 210
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Methods for Numerical Integration
• Shooting and Multiple-Shooting Require Numerical Integration
• Methods for Numerical Integration: Initial-Value Problems
• Consider Integration from ti to ti+1
pi+1 = pi +
∫ ti+1
ti
G(p, t)dt
• Consider a Multi-Stage Method of The Form
τj = ti + hiρj
where 0 ≤ ρ1 ≤ ρ2 ≤ · · · ≤ ρk ≤ 1 and 1 ≤ j ≤ k
• Now Apply a Quadrature Formula of The Form
∫ ti+1
ti
G(p)dt ≈ hi
k∑
j=1
βjGj
where Gj = G(pj , τj)
Copyright © 2006–2012 Anil Vithala Rao Page 211
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Next Apply a Quadrature within a Quadrature of the Form
∫ τj
ti
G(p)dt ≈ hi
k∑
l=1
αjlGl
for 1 ≤ j ≤ k
• Popular Multi-Stage Methods: Runge-Kutta
pi+1 = pi + hi
k∑
j=1
βjGij
where
Gij = G
[(
pi + hi
k∑
l=1
αjlGil
)]
Copyright © 2006–2012 Anil Vithala Rao Page 212
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Runge-Kutta Schemes Visualizable Using a Butcher Array
ρ1 α11 α12 · · · α1k
ρ2 α21 α22 · · · α2k
......
......
...
ρk αk1 αk2 · · · αkk
β1 β2 · · · βk
• Runge-Kutta Schemes Are
⊲ Explicit: αjl = 0 for l ≥ j
⊲ Implicit: otherwise
• Common Runge-Kutta Schemes
⊲ Euler Method (explicit, k = 1)
0 0
1
pi+1 = pi + hiGi
Copyright © 2006–2012 Anil Vithala Rao Page 213
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Trapezoidal Method (implicit, k = 2)
0 0 0
1 1/2 1/2
1/2 1/2
pi+1 = pi +1
2hi (Gi +Gi+1)
⊲ Classical Runge-Kutta Method (explicit, k = 4)
0 0 0 0 0
1/2 1/2 0 0 0
1/2 0 1/2 0 0
1 0 0 1 0
1/6 1/3 1/3 1/6
k1 = hiGi
k2 = hiG(
pi +12k1, ti +
12hi
)
k3 = hiG(
pi +12k2, ti +
12hi
)
k4 = hiG (pi + k3, ti+1)
Copyright © 2006–2012 Anil Vithala Rao Page 214
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
pi+1 = pi +1
6(k1 + 2k2 + 2k3 + k4)
⊲ Hermite-Simpson (implicit, k = 3)
0 0 0 0
1/2 5/24 1/3 −1/24
1 1/6 2/3 1/6
1/6 2/3 1/6
p = 12(pi + pi+1) +
18hi (Gi −Gi+1)
G = G(
p, ti +12hi
)
pi+1 = pi +1
6hi
(
Gi + 4G+Gi+1
)
• Alternate Motivation for Runge-Kutta Methods
⊲ Suppose p(t) is approximated on [ti, ti+1] by the function p(t) as
p(t) ≈ p(t) =
m∑
k=1
ak(t− ti)k
where (a0, . . . , ak) are chosen such that p(ti) = pi
Copyright © 2006–2012 Anil Vithala Rao Page 215
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Choose a finite set of points τ1, . . . , τk, i.e.,
˙p(τj) = G [p(τj)]
⊲ Last equation is called a collocation condition
• Thus, Runge-Kutta Methods are Collocation Methods
• Three Common Forms of Collocation
⊲ Lobatto methods: both endpoints included in collocation
⊲ Radau methods: one endpoint included in collocation
⊲ Gauss methods: neither endpoint included in collocation
• Examples of Different Forms of Collocation
⊲ Midpoint rule ⇐⇒ Gauss method
pi+1 = pi + hiG
[
1
2(pi + pi+1) , ti +
1
2h1
]
⊲ Euler backward rule ⇐⇒ Radau method
pi+1 = pi + hiGi+1
⊲ Hermite-Simpson (see above) ⇐⇒ Lobatto method
Copyright © 2006–2012 Anil Vithala Rao Page 216
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Convergence Rate of Low-Order Integrators
2 2.5 3 4 5
−4
−6
−8
−10
−12
log10Error
log10 Number of Approximating Intervals
3.5 4.5
Figure 2-3. Error vs. Number of Intervals for Trapezoid Rule Approximation of
Copyright © 2006–2012 Anil Vithala Rao Page 217
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Gaussian Quadrature
• Using Evenly Spaced Points =⇒ Poor Convergence of Integral
• Alternative: Choose Points to Minimize Error in Quadrature
Approximation
• Suppose We Want to Approximate
∫ +1
−1
g(τ)dτ Using N Points. Then
∫ +1
−1
p(τ)dτ ≈
N∑
k=1
wkp(τk)
• Error in Quadrature Approximation, E(g)
E(g) =
∫ +1
−1
g(τ)dτ −
N∑
k=1
wkg(τk)
• Gaussian Quadature Points Chosen to Minimize E(p)
• Question: Why Use Gaussian Quadrature Points for Collocation?
• Answer: Exponential Convergence for Smooth Functions
Copyright © 2006–2012 Anil Vithala Rao Page 218
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Family of Gaussian Quadrature Points
• Let PN (τ) Be the N th-Degree Legendre Polynomial
• Gaussian Quadrature Collocation Points
⊲ Legendre-Gauss: Roots of PN (τ)
⊲ Legendre-Gauss-Radau: Roots of PN (τ) + PN−1(τ)
⊲ Legendre-Gauss-Lobatto: Roots of PN−1(τ) Together with τ = −1 and
τ = +1
• Exactness of Gaussian Quadrature Approximations
⊲ Legendre-Gauss (LG): polynomial of degree 2N − 1
⊲ Legendre-Gauss-Radau (LGR): polynomial of degree 2N − 2
⊲ Legendre-Gauss-Lobatto (LGL): polynomial of degree 2N − 3
Copyright © 2006–2012 Anil Vithala Rao Page 219
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Exactness of Gaussian Quadrature
• Let
⊲ (τ1, . . . , τN ) be the roots of PN (τ)
⊲ g(τ) be a polynomial of degree at most 2N − 1
• Will Show That∫ +1
−1
g(τ)dτ
is Exact if g(τ) is a Polynomial of Degree at Most 2N − 1
• First, Because g(τ) is a Polynomial of Degree at Most 2N − 1, g(τ) Can Be
Written as
g(τ) = PN (τ)f(τ) + h(τ)
Where f(τ) and h(τ) are Polynomials of Degree at most N − 1
• Then,∫ +1
−1
g(τ)dτ =
∫ +1
−1
(PN (τ)f(τ) + h(τ)) dτ =
∫ +1
−1
PN (τ)f(τ)dτ+
∫ +1
−1
h(τ)dτ
Copyright © 2006–2012 Anil Vithala Rao Page 220
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Because f(τ) is a Polynomial of Degree at Most N − 1,
f(τ) =
N−1∑
i=0
ciPi(τ), Pi(τ) = ith −Degree Legendre Polynomial
• Next, PN (τ) is Orthogonal to All Other Legendre Polynomials. Therefore
∫ +1
−1
PN (τ)f(τ)dτ =
∫ +1
−1
PN (τ)
N−1∑
i=0
ciPi(τ)dτ
=
N−1∑
i=0
ci
∫ +1
−1
PN (τ)Pi(τ)dτ = 0
• Consequently,∫ +1
−1
g(τ)dτ =
∫ +1
−1
h(τ)dτ
• Recall Now that h(τ) is
⊲ A Polynomial of Degree at Most N − 1
⊲ Thus, h(τ) can be decomposed in term of a basis of N Lagrange
Copyright © 2006–2012 Anil Vithala Rao Page 221
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
polynomials that are each of degree N − 1,
h(τ) =N∑
i=1
h(τi)Li(τ), Li(τ) =N∏
j=1
i 6=j
τ − τjτi − τj
• Therefore,
∫ +1
−1
h(τ)dτ =
N∑
i=1
∫ +1
−1
N∑
i=1
h(τi)Li(τ)dτ =
N∑
i=1
h(τi)
∫ +1
−1
Li(τ)dτ
• Because g(τ) = PN (τ)f(τ) + h(τ)⇒ g(τi) = PN (τi)f(τi) + h(τi)
• But PN (τi)f(τi) = 0 Because (τ1, . . . , τN ) are the Roots of PN (τ)
• Consequently, g(τi) = h(τi)
Copyright © 2006–2012 Anil Vithala Rao Page 222
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• We Then Obtain
∫ +1
−1
g(τ)dτ =
∫ +1
−1
h(τ)dτ =
N∑
i=1
h(τi)
∫ +1
−1
Li(τ)dτ
=
N∑
i=1
g(τi)
∫ +1
−1
Li(τ)dτ
=
N∑
i=1
wig(τi)
Where
wi =
∫ +1
−1
Li(τ)dτ = Quadrature Weight
• Thus,∫ +1
−1
g(τ)dτ =
N∑
i=1
wig(τi)
is Exact if g(τ) is a Polynomial of Degree at Most 2N − 1
Copyright © 2006–2012 Anil Vithala Rao Page 223
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Accuracy of LGR and LGL Quadrature
• Legendre-Gauss-Radau Quadrature
⊲ LGR Points: N − 1 roots on (−1,+1] of PN (τ) + PN−1(τ), Together
with τ1 = −1
⊲ Because τ1 = −1 is specified =⇒ have N − 1 free points
⊲ Can use similar argument for LGR as was used for LG with following
changes
Given N − 1 free points on (−1,+1],
∫ +1
−1
g(τ)dτ =
N∑
i=1
wig(τi), τ1 = −1
is exact if g(τ) is a polynomial of degree at most 2N − 2
• Legendre-Gauss-Lobatto Quadrature
⊲ LGL Points: N − 2 roots on (−1,+1) of PN (τ), Together with τ1 = −1
and τN = +1
⊲ Because τ1 = −1 and τN = +1 are specified =⇒ have N − 2 free points
Copyright © 2006–2012 Anil Vithala Rao Page 224
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Can use similar argument for LGL as was used for LG with following
changes
Given N − 2 free points on (−1,+1),
∫ +1
−1
g(τ)dτ =
N∑
i=1
wig(τi), τ1 = −1
is exact if g(τ) is a polynomial of degree at most 2N − 3
• Key Result: Given Desire to Approximate
∫ +1
−1
g(τ)dτ via N Point
Quadrature,
⊲ LG quadrature is exact if g(τ) is a polynomial of degree 2N − 1 or less
⊲ LGR quadrature is exact if g(τ) is a polynomial of degree 2N − 2 or less
⊲ LGL quadrature is exact if g(τ) is a polynomial of degree 2N − 3 or less
Copyright © 2006–2012 Anil Vithala Rao Page 225
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Formulae for LG, LGR, and LGL Quadrature Weights
• LG
wi =2
1− τ2i P2N (τi)
, i = 1, . . . , N
• LGR
w1 = 2N2
wi = 1(1−τi)P 2
N1(τi)
• LGL
w1 = wN = 2N(N−1)
wi = 2N(N−1)P 2
N1(τi)
Copyright © 2006–2012 Anil Vithala Rao Page 226
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example of Exponential Convergence of Gaussian Quadrature
• Approximate (π/4)∫ +1
−1cos(πτ/2)dτ (Exact Integral =1)
0
N
log10
e
LG
LGR
LGL
2 4 6 8 10 12
-2
-4
-6
-8
-10
-12
-14
-16
• Note: All Gaussian Quadrature Rules Convergence Exponentially
Copyright © 2006–2012 Anil Vithala Rao Page 227
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Function Approximation and Interpolation
• Solving Continuous-Time Problem Numerically
⊲ Continuous problem replaced with discrete approximation
⊲ Discrete approximation require that functions be approximated
⊲ Approximations generally computed using polynomials
• In Principle, Any Polynomial Can Be Used
• Practically, Easier to Utilize Lagrange Interpolating Polynomials
Copyright © 2006–2012 Anil Vithala Rao Page 228
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Lagrange Interpolating Polynomials
• Given Samples of a Function f(t) at Support Points (t1, . . . , tN )
• Using (t1, . . . , tN ), Construct N Lagrange Polynomials of Degree N − 1
Li(t) =
N∏
k=1k 6=i
t− tk
ti − tk, (i = 1, . . . , N),
• Isolation Property of Lagrange Interpolating Polynomials
Li(tk) =
1 , i = k
0 , i 6= k
• Now Construct Interpolant for f(t) Using Li(t), (i = 1, . . . , N)
f(t) ≈ f(t) =
N∑
i=1
ciLi(τ) =⇒ ci = f(ti)
• Consequently, Interpolated Value at Support Points = Function Value!
Copyright © 2006–2012 Anil Vithala Rao Page 229
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Choice of Support Points for Lagrange Polynomials
• Theoretically, Can Choose Any Values for (t1, . . . , tN )
• Reality: Problems Arise When Support Points Not Chosen Well
• Simplest Choice: Uniformly-Spaced Points
• Consider the Following Function
f(τ) =1
1 + 25τ2, τ ∈ [−1,+1]
• Next Four Slides
⊲ 8 and 32 uniformly-spaced points
⊲ 8 and 32 Legendre-Gauss points
⊲ Uniformly-spaced =⇒ Runge phenomenon
⊲ Legendre-Gauss: Runge phenomenon disappears
Copyright © 2006–2012 Anil Vithala Rao Page 230
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Behavior Using 8 Uniformly-Spaced Points
00−1 −0.5 0.5
1
1
0.2
0.4
0.6
0.8
f (τ)
f (τ)
f(τ)
τ
A 8 Uniform Support Points.
Copyright © 2006–2012 Anil Vithala Rao Page 231
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Behavior Using 32 Uniformly-Spaced Points
0
0−1 −0.5 0.5 1
200
400
600
800
−200
f (τ)
f (τ)
f(τ)
τ
B 32 Uniform Support Points.
Copyright © 2006–2012 Anil Vithala Rao Page 232
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Behavior Using 8 LG Points
1.2
−0.2
0
0−1 −0.5 0.5
1
1
0.2
0.4
0.6
0.8
f (τ)f(τ)
τ
f (τ)
A 8 LG Support Points.
Copyright © 2006–2012 Anil Vithala Rao Page 233
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Behavior Using 32 LG Points
00−1 −0.5 0.5
1
1
0.2
0.4
0.6
0.8
f (τ)f(τ)
τ
f (τ)
B 32 LG Support Points.Copyright © 2006–2012 Anil Vithala Rao Page 234
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Motivation for a Pseudospectral Method
• Gaussian Quadrature
⊲ High-accuracy approximation to integral
⊲ Small number of support points needed
• Lagrange Interpolating Polynomials
⊲ Polynomial coefficients = function values at support points
⊲ Poorly chosen support point =⇒ Runge phenomenon
⊲ Choose Gaussian quadrature points =⇒ no Runge phenomenon
• Pseudospectral Method
⊲ Uses Gaussian quadrature points to approximate integral of a function
⊲ Employs Lagrange polynomials for function approximation
• Next Part of Course: Foundations of Pseudospectral Methods
Copyright © 2006–2012 Anil Vithala Rao Page 235
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part VI: Foundations of Pseudospectral Methods
Copyright © 2006–2012 Anil Vithala Rao Page 236
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Pseudospectral Methods
• Collocation Points are Nodes of a Gaussian Quadrature∫ 1
−1f(τ)dτ ≈
N∑
k=1
wif(τi)
• Basis Functions are Lagrange Polynomials
Li(τ) =N∏
j=M
j 6=i
τ − τj
τi − τj, Li(τj) =
0 , i 6= j
1 , i = j=⇒ Isolation Property
• Three Sets of Gaussian Quadrature Collocation Points
⊲ LG: strictly on open interval τ ∈ (−1,+1)
⊲ LGR: half-open interval τ ∈ [−1,+1) or τ ∈ (−1,+1]
⊲ LGL: closed interval τ ∈ [−1,+1]
• High Accuracy+Low-Dimensionality
x(τ) ≈ X(τ) =N∑
k=M
XiLi(τ), (M = 0 or M = 1)
Copyright © 2006–2012 Anil Vithala Rao Page 237
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
X(τi) = Xi =⇒ Due to Isolation Property
• Key Question: What is the Difference Between LG, LGR, and LGL Points?
⊲ First Glance
All three Gaussian quadrature rules extremely accurate
Only differ by one polynomial degree LG → LGR → LGL
• Obtain Exponential Convergence for Integral of a Smooth Function
• Will Begin with Global Pseudospectral Methods
• Simplified Problem Used to Develop Methods
⊲ Minimize the Cost Functional
J = Φ(x(+1))
⊲ Dynamic Constraintsx = f(x,u)
⊲ Initial Conditionx(−1) = x0
Copyright © 2006–2012 Anil Vithala Rao Page 238
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Schematic of Gaussian Quadrature Points
-1 -0.5 0 0.5 1
LG Points Include Neither Endpoint
LGR Points Include Endpoint τ = −1
Flipped LGR Points Include Endpoint τ = +1
LGL Points Include Both Endpoints τ = −1 and τ = +1
τ
LGL
LGR
Flipped LGR
LG
Copyright © 2006–2012 Anil Vithala Rao Page 239
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Notation and Conventions
• This Presentation: Treat State and Control as Row Vectors
⊲ State, x(τ) ∈ Rn −→ x(τ) =
[
x1(τ) · · · xn(τ)]
⊲ Control, u(τ) ∈ Rm −→ u(τ) =
[
u1(τ) · · · um(τ)]
• Vectorization of State and Control Approximations
Xi:j =
Xi
...
Xj
, Ui:j =
Ui
...
Uj
• Differentiation Matrix: Di:j = Columns i Through j of D
Di:j =[
Di · · · Dj
]
Copyright © 2006–2012 Anil Vithala Rao Page 240
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Optimal Control Functions Vectorized in Similar Manner
Fi:j =
f(Xi,Ui)...
f(Xj ,Uj)
• Notation is Similar To That Used for MATLAB Operations
• For Example: DX −→ Matrix-Vector Multiplication
• Collocation Equations Can Then Be Written As
DX = F
Copyright © 2006–2012 Anil Vithala Rao Page 241
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Gauss Pseudospectral Method (GPM)
• State Approximation:
x(τ) ≈ X(τ) =∑N
i=0 XiLi(τ) =⇒ N th-Degree Polynomial
x(τk) ≈ X(τk) =N∑
i=0
XiLi(τk) =N∑
i=0
DkiXi =⇒ (N − 1)th-Degree Polynomial
NLP:
minimize Φ(XN+1)
subject to DX = F(XLG,ULG) , XN+1 = X0 +wTF(XLG,ULG),
X0 = x0
• Can Divide D Into: D = [D0 D1:N ] where D1:N is Square
• Have Proven Following Theorems About D Matrix for GPM
⊲ D1:N is Nonsingular
⊲ D−11:ND0 = −1
⊲ D−11:N = A where A is the Gauss Integration Matrix
Copyright © 2006–2012 Anil Vithala Rao Page 242
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Key Result: GPM is an Implicit Integration Method
XLG = 1X0 +AFLG =⇒ IMPLICIT INTEGRAL FORM OF GPM
• In Other Words
⊲ GPM Can Be Written in Either Differential or Integral Form
⊲ Differential Form ⇐⇒ Integral Form
⊲ GPM Is an Integral Method
Copyright © 2006–2012 Anil Vithala Rao Page 243
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Radau Pseudospectral Method (RPM)
• State Approximation:
x(τ) ≈ X(τ) =∑N+1
i=1 XiLi(τ) =⇒ N th-Degree Polynomial
x(τk) ≈ X(τk) =N∑
i=0
XiLi(τk) =N+1∑
i=1
DkiXi =⇒ (N−1)th-Degree Polynomial
NLP:
minimize Φ(XN+1)
subject to DX = F(XLGR,ULGR)
X1 = x0
• Can Divide D Into: D = [D1 D2:N+1] where D2:N+1 is Square
• Have Proven Following Theorems About D Matrix for RPM
⊲ D2:N+1 is Nonsingular
⊲ D−12:N+1D1 = −1
⊲ D−12:N+1 = A where A is the Radau Integration Matrix
Copyright © 2006–2012 Anil Vithala Rao Page 244
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Key Result: RPM is an Implicit Integration Method
X2:N+1 = 1X0 +AFLGR =⇒ IMPLICIT INTEGRAL FORM OF RPM
• In Other Words
⊲ RPM Can Be Written in Either Differential or Integral Form
⊲ Differential Form ⇐⇒ Integral Form
⊲ RPM Is an Integral Method
Copyright © 2006–2012 Anil Vithala Rao Page 245
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Structure Common to Both GPM and RPM
• Right-Hand and Left-Hand Sides of Collocation Equations
⊲ Right-Hand Side Includes Only Collocation Points
⊲ Left-Hand Side: Includes Collocation Points Plus One Endpoint
⊲ Differentiation Matrices Have One More Column Than Row
(Size= N × (N + 1))
• Key Property of GPM and RPM Differentiation Matrices
⊲ Eliminating First Column, Remaining Square Matrix is Invertible
⊲ Remaining Square Matrix is the Integration Matrix
• GPM/RPM are Integration Methods
Copyright © 2006–2012 Anil Vithala Rao Page 246
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Why Choose Differential Form Over Integral Form?
• GPM/RPM are Integration Methods
⊲ Seems more natural to choose integral form
⊲ Why choose differential form?
• Answer: Differential Form is Much More Sparse
⊲ Collocation Equations in Differential Form
DX = F =⇒ Each Equation Involves Only One Value of Right-Hand Side
⊲ Collocation Equations in Integral Form
X = AF =⇒ Each Equation Involves All Values of Right-Hand Side
• Differential Form Computationally More Efficient
Copyright © 2006–2012 Anil Vithala Rao Page 247
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Lobatto Pseudospectral Method (LPM)
• State Approximation:x(τ) ≈ X(τ) =
∑Ni=1 xiLi(τ) =⇒ (N − 1)th-Degree Polynomial
x(τk) =
N∑
i=1
XiLi(τk) =
N∑
i=1
DkiXi =⇒ (N − 2)th-Degree Polynomial
LPM NLP:
minimize Φ(XN )
subject to DX = F(X,U)
X1 = x0
• LPM Differentiation Matrix is Square and Singular
• Integral and Differential Form of LPM Not Equivalent =⇒ LPM Not an
Integration Method
Copyright © 2006–2012 Anil Vithala Rao Page 248
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Comparison of GPM/RPM/LPM
• Methods Appear Similar, But Appearances are Deceiving
• Gauss and Radau Pseudospectral Methods
⊲ Polynomial Degree of X(τ) = N (=Number of Collocation Points)
⊲ Non-Square and Full Rank Differentiation Matrix
⊲ Differential Form = Implicit Integral Form
• Lobatto Pseudospectral Method (LPM)
⊲ Polynomial Degree of X(τ) = N − 1 (< Number of Collocation Points)
⊲ Square and Singular Differentiation Matrix
⊲ Differential Form 6= Implicit Integral Form
• How Do These Differences Play Out in Practice?
⊲ GPM/RPM State & Control Converge Much Faster Than LPM
⊲ On Even Simple Example, LPM Costate Does Not Converge
• NLPs for GPM/RPM Can Be Smaller Than NLP for LPM
Copyright © 2006–2012 Anil Vithala Rao Page 249
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Costate Estimation Using Pseudospectral Methods
• GPM Costate
λN+1 = ΛN+1 → Costate at τ = +1
λi = Λi/wi +ΛN+1 → Costate at LG Points
λ0 = ΛN+1 −DT0Λ
LG → Costate at τ = −1 Via Integral Approximation of λ
• RPM Costate
λi = Λi/wi → Costate at LGR Points
λN+1 = DT
N+1ΛLGR → Costate at τ = +1 Via Integral Approximation of λ
• LPM Costate
λi = Λi/wi → Costate at LGL Points
• As with State Approximation,
⊲ GPM/RPM Endpoint Costate Obtained via Quadrature
⊲ LPM Costate Not Obtained via Numerical Integration!
Copyright © 2006–2012 Anil Vithala Rao Page 250
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example
Minimize J = −y(2)
Subject To The Dynamic Constraint y = 52 (−y + yu− u2)
Initial Condition: y(0) = −1
Optimal Solution:
y∗(t) = 4/a(t),
u∗(t) = y∗(t)/2,
λ∗y(t) = − exp(2 ln(a(t))− 5t/2)/b,
a(t) = 1 + 3 exp(5t/2),
b = exp(−5) + 6 + 9 exp(5).
Problem Solved Using GPM, RPM, and LPM for N = (5, 10, 15, 20, 25, 30).
Copyright © 2006–2012 Anil Vithala Rao Page 251
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
State Errors Using GPM, RPM, and LPM
-2
-4
-6
-8
-10
-12
-14
-165 10 15 20 25 30
0e y
=log 1
0||y(τ
k)−
y∗(τ
k)|| ∞
N
Gauss
Radau
Lobatto
Copyright © 2006–2012 Anil Vithala Rao Page 252
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Control Errors Using GPM, RPM, and LPM
-2
-4
-6
-8
-10
-12
-14
-165 10 15 20 25 30
0eu=log 1
0||u(τ
k)−
u∗(τ
k)|| ∞
N
Gauss
Radau
Lobatto
Copyright © 2006–2012 Anil Vithala Rao Page 253
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Costate Errors Using GPM, RPM, and LPM
-5
-10
-15
5
5
10
10 15 20 25 30
0
eλy=log 1
0||λy(τ
k)−
λ∗ y(τ
k)|| ∞
N
Gauss
Radau
Lobatto
Copyright © 2006–2012 Anil Vithala Rao Page 254
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example
Minimize J = −r(tf )
Subject To The Dynamic Constraints
r = vr,
θ = vθ/r,
vr = v2θ/r − µ/r2 + a sinβ,
vθ = −vrvθ/r + a cosβ,
Boundary Conditions
(r(0), θ(0), vr(0), vθ(0)) = (1, 0, 0, 1),
(vr(tf ), vθ(tf )) = (0,√
µ/r(tf )),
Where
a ≡ a(t) =T
m0 − |m|t . (1)
Constants: µ = 1, T = 0.1405, m0 = 1, m = 0.0749, and tf = 3.32.
Copyright © 2006–2012 Anil Vithala Rao Page 255
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Costate Solution for N = 64 Using GPM
−2.5
-1.5
-2
1.5
1.5 2 2.5 3 3.5
-1
-0.5
0
0
0.5
0.5
1
1
λ(t)=(λ
r(t),λrv(t),λvr(t),λvθ(t))
[Gauss]
t (TU)
λr(t)λθ(t)λvr(t)λvθ(t)
Copyright © 2006–2012 Anil Vithala Rao Page 256
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Costate Solution for N = 64 Using RPM
−2.5
-1.5
-2
1.5
1.5 2 2.5 3 3.5
-1
-0.5
0
0
0.5
0.5
1
1
λ(t)=(λ
r(t),λrv(t),λvr(t),λvθ(t))
[Radau])
t (TU)
λr(t)λθ(t)λvr(t)λvθ(t)
Copyright © 2006–2012 Anil Vithala Rao Page 257
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Costate Solution for N = 64 Using LPM
-2
-3
-41.5
2
2 2.5
3
3 3.5
4
-1
0
0 0.5
1
1λ(t)=(λ
r(t),λrv(t),λvr(t),λvθ(t))
[Lobatto]
t (TU)
λr(t)λθ(t)λvr(t)λvθ(t)
Copyright © 2006–2012 Anil Vithala Rao Page 258
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Key Features of Solution Errors
• Gauss and Radau State, Control, and Costate Converge Exponentially
• Lobatto Scheme
⊲ State and control errors decrease at much slower rate than either Gauss
or Radau
⊲ Costate does not converge
• Upshot of Examples
⊲ Gauss and Radau
Both are implicit Gaussian quadrature schemes
For a problem with smooth solution, Gauss quadrature converges
exponentially
⊲ Lobatto is not an implicit integration method
⊲ At best, converges much more slowly than either Gauss or Radau
⊲ Costate potentially nonconvergent
Copyright © 2006–2012 Anil Vithala Rao Page 259
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part VII: Pseudospectral Method Implementation
Copyright © 2006–2012 Anil Vithala Rao Page 260
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Use of Multiple-Interval Pseudospectral Methods
• Global Collocation is Adequate for Simple Problems
• Most Problems of Practical Interest
⊲ May Have Rapid Changes in Behavior
⊲ May Have Points of Nonsmoothness
⊲ For Such Problems, Global Collocation is Inadequate
• Multiple-Interval Pseudospectral Methods
⊲ Time Interval t ∈ [t0, tf ] Divided Into Mesh Intervals
⊲ Each Mesh Interval
Different Degree State Approximation
Different Width
• Leads to hp-Adaptive Pseudospectral Methods
Copyright © 2006–2012 Anil Vithala Rao Page 261
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Development of Multiple-Interval Formulation
• Transformation of Time: t = (tf − t0)s/2 + (tf + t0)/2 where s ∈ [−1,+1]
• Minimize the Cost Functional
J = Φ(x(−1), t0,x(+1), tf ) +tf − t0
2
∫ +1
−1
L(x(s),u(s), s; t0, tf ) ds
• Dynamic Constraints
dx
ds=tf − t0
2f(x(s),u(s), s; t0, tf ),
• Path Constraints
cmin ≤ c(x(s),u(s), s; t0, tf ) ≤ cmax,
• Boundary Conditions
bmin ≤ b(x(−1), t0,x(+1), tf ) ≤ bmax.
Copyright © 2006–2012 Anil Vithala Rao Page 262
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Multiple-Interval Bolza Problem
• Divide s ∈ [−1,+1] into K Mesh Intervals [sk−1, sk], k = 1, . . . ,K
• Mesh Points: −1 = s0 < s1 < s2 < · · · < sK = sf = +1.
• Let x(k)(s) and u(k)(s) be the State and Control in mesh interval k.
• Minimize the Cost Functional
J = Φ(x(1)(−1), t0,x(K)(+1), tf ) +
tf − t0
2
K∑
k=1
∫ sk
sk−1
L(x(k)(τ),u(k)(τ), τ ; sk−1, sk) dτ
• Dynamic Constraints in Mesh Interval k = 1, . . . ,K:
dx(k)(s)
ds=
tf − t0
2f(x(k)(s),u(k)(s), s; t0, tf )
• Path Constraints
cmin ≤ c(x(k)(s),u(k)(s), s; t0, tf ) ≤ cmax.
• Boundary Conditions
bmin ≤ b(x(1)(−1), t0,x(K)(+1), tf ) ≤ bmax.
• Continuity in State at Mesh Points: x(s−k ) = x(s+k ), be
Copyright © 2006–2012 Anil Vithala Rao Page 263
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Choice of Multiple-Interval Pseudospectral Method
• Gauss Pseudospectral Method (GPM)
⊲ Additional Quadrature Equation Required to Estimate Final State in a
Mesh Interval
⊲ Control at Mesh Point Not an NLP Variable
• Lobatto Pseudospectral Method (LPM)
⊲ Contains Redundant State and Control at Mesh Point
⊲ Derivative Approximation at Mesh Point Inconsistent Between Adjacent
Mesh Intervals
• Radau Pseudospectral Method (RPM)
⊲ No Redundancy in Control at Mesh Points
⊲ Overlap is Ideal for Multiple-Interval Formulation
⊲ Is Most Natural to Implement
Copyright © 2006–2012 Anil Vithala Rao Page 264
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Schematic of Two-Interval GPM and RPM
GPM
First and LastLG Points
First and LastLG Points
No Value of Controlat Mesh Point
RPM
First and LastLGR Points
First and LastLGR Points
Control atMesh Point
Copyright © 2006–2012 Anil Vithala Rao Page 265
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Observations of Structure GPM and RPM
• GPM
⊲ Has elegant mathematical structure
⊲ Easy to see equivalence between direct and indirect forms
⊲ Unfortunately, difficult to use in multiple -interval form
• RPM
⊲ Not intuitive, but shares same mathematical properties as GPM
⊲ As a result, actually is just as accurate as GPM
⊲ Is much more natural for multiple-interval use
• Conclusion: Radau Method is Best Choice
Copyright © 2006–2012 Anil Vithala Rao Page 266
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Multiple-Interval Radau Pseudospectral Method
• State Approximation in Mesh Interval k ∈ [1, . . . ,K]:
x(k)(s) ≈ X(k)(s) =
Nk+1∑
j=1
X(k)j ℓ
(k)j (s), ℓ
(k)j (s) =
Nk+1∏
l=1l 6=j
s− s(k)l
s(k)j − s
(k)l
,
where (s(k)1 , . . . , s
(k)Nk
) are the LGR Points in Mesh Interval k.
• Derivative Approximation of State
dX(k)(s)
ds=
Nk+1∑
j=1
X(k)j
dℓ(k)j (s)
ds.
• Discrete Approximation of Cost Functional
J ≈ φ(X(1)1 , t0,X
(K)NK+1, tK) +
K∑
k=1
Nk∑
j=1
tf − t0
2w
(k)j g(X
(k)j ,U
(k)j , s
(k)j ; sk−1, sk),
⊲ w(k)j = LGR Quadrature Weights (j = 1, . . . , Nk)
⊲ U(k)j = Control Approximations (j = 1, . . . , Nk)
⊲ X(k)j = State Approximations (j = 1, . . . , Nk + 1)
Copyright © 2006–2012 Anil Vithala Rao Page 267
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Collocation ConditionsNk+1∑
j=1
D(k)ij X
(k)j −
tf − t0
2a(X
(k)i ,U
(k)i , t
(k)i ) = 0, (i = 1, . . . , Nk).
where t = (tf − t0)/2s+ (tf + t0)/2
• Radau Differentiation Matrix (Nk × (Nk + 1))
D(k)ij =
dℓ(k)j (s)
ds
s(k)i
, (i = 1, . . . , Nk, j = 1, . . . , Nk + 1, k = 1, . . . ,K),
• Path Constraints
cmin ≤ c(X(k)i ,U
(k)i , t
(k)i ) ≤ cmax, (i = 1, . . . , Nk).
• Boundary Conditions
bmin ≤ b(X(1)1 , t0,X
(K)NK+1, tf ) ≤ bmax.
• State Continuity at Mesh Points
X(k)Nk+1 = X
(k+1)1 , (k = 1, . . . ,K − 1),
Copyright © 2006–2012 Anil Vithala Rao Page 268
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Vectorized Form of Multiple-Interval Radau Method
• Quantities in Each Mesh Interval
s(k) =[
s(k)i
]1
Nk
, k = 1, . . . , K − 1 , s(K) =[
s(K)i
]1
NK+1,
t(k) =[
t(k)i
]1
Nk
, k = 1, . . . , K − 1 , t(K) =[
t(K)i
]1
NK+1,
X(k) =[
X(k)i
]1
Nk
, k = 1, . . . , K − 1 , X(K) =[
X(K)i
]1
NK+1,
U(k) =
[
U(k)i
]1
Nk
, k = 1, . . . , K , g(k) =[
g(X(k)i
,U(k)i
, t(k)i
)]1
Nk
, k = 1, . . . , K
A(k) =[
a(X(k)i
,U(k)i
, t(k)i
)]1
Nk
, k = 1, . . . , K , C(k) =[
c(X(k)i
,U(k)i
, t(k)i
)]1
Nk
, k = 1, . . . , K
w(k) = [wi]1Nk
, k = 1, . . . , K , N =
K∑
k=1
Nk.
Copyright © 2006–2012 Anil Vithala Rao Page 269
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Cumulative Quantities
s =
s(1)
.
.
.
s(K)
, t =
t(1)
.
.
.
t(K)
,
w =
w(1)
.
.
.
w(K)
, X =
X(1)
.
.
.
X(K)
,
U =
U(1)
.
.
.
U(K)
, g =
g(1)
.
.
.
g(K)
,
A =
A(1)
.
.
.
A(K)
, C =
C(1)
.
.
.
C(K)
.
• Vectorized Version of Cost
J ≈ φ(X1, t0,XN+1, tf ) +tf − t0
2wTg
Copyright © 2006–2012 Anil Vithala Rao Page 270
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Vectorized Version of Dynamics
∆ = DX−tf − t0
2A = 0,
• Vectorized Version of Path Constraints
Cmin ≤ C ≤ Cmax
• Vectorized Version of Boundary Conditions
bmin ≤ b(X1, t0,XN+1, tf ) ≤ bmax.
Copyright © 2006–2012 Anil Vithala Rao Page 271
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Structure of Radau Pseudospectral Differentiation Matrix
Block 1
Block 2
Block 3
Block K
(2) Zeros Except in Blocks
(1) Block k is of Size Nk by Nk+1
(3) Total Size N by N+1
Copyright © 2006–2012 Anil Vithala Rao Page 272
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Key Features of Multiple-Interval Radau Method
• Defect Constraints for Single Interval (Global) Method
⊲ Left-Hand Side: LGR Points + Final Point
⊲ Right-Hand Side: Only LGR Points
⊲ Differentiation Matrix is Size N × (N + 1)
• Defect Constraints for Multiple Interval Method
⊲ Left-Hand Side: LGR Points in Each Mesh Interval + Final Point
⊲ Right-Hand Side: LGR Points in Each Mesh Interval
⊲ Differentiation Matrix is Size N × (N + 1) where
N =
K∑
k=1
Nk
• Equations Have Identical Form for Both Global and Multiple Interval
Method
Copyright © 2006–2012 Anil Vithala Rao Page 273
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Implementation of Radau Pseudospectral Method
• Assume a Method with a Fixed Degree in Each Segment
• Choose
⊲ Number of Segments, S
⊲ Let K = Number LGR Points on Each Segment
⊲ Then N = KS and N + 1 = KS + 1 =Total Number of Points
• State Approximations at N LGR Points Plus Final Point
X =
x1
x2
.
..
xN+1
Copyright © 2006–2012 Anil Vithala Rao Page 274
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Control Approximation Includes Only LGR Points
U =
u1
u2
...
uN
• Treat t0 and tf as Optimization Variables
Copyright © 2006–2012 Anil Vithala Rao Page 275
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
LGR Points, Quadrature Weights, & Differentiation Matrix
• Compute LGR Points and Weights in Each Segment
τ ∈ [−1,+1) = K LGR Points
W ∈ [−1,+1) = K LGR Weights
• Transform the LGR Points on τ ∈ [−1,+1) to [sk−1, sk] via
s =sk − sk−1
2τ +
sk + sk−1
2.
• Transform the LGR Weights on τ ∈ [−1,+1) to [sk−1, sk) via
w =sk − sk−1
2W
• Compute Differentiation Matrix, d, on τ ∈ [−1,+1)
• Transform d from τ ∈ [−1,+1) to D in s ∈ [sk−1, sk) via
D =2
sk − sk−1d
Copyright © 2006–2012 Anil Vithala Rao Page 276
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• s ∈ RN , w ∈ R
N , and D ∈ RN×(N+1) Used to Construct NLP Functions
• Construction of Objective Function
J = Φ(X1, t0,XN+1, tf ) +tf − t0
2
N∑
k=1
wiL(Xk,Uk)
• Construction of N Defect Constraints
∆i =N+1∑
j=1
DijXj −tf − t0
2f(Xi,Ui) = 0, i = 1, . . . , N, k = 1, . . . , n
• Note
⊲ Left-Hand Side of Defects: LGR Points Plus Final Point
N+1∑
j=1
DijXj ⇐= Note That Summation Runs From 1→ N + 1
⊲ Right-Hand Side of Defects: Only LGR Points
tf − t02
f(Xi,Ui)← Note That i ∈ (1, . . . , N)
Copyright © 2006–2012 Anil Vithala Rao Page 277
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Construction of NLP for SNOPT
• SNOPT Needs Following Information
⊲ Bounds on NLP Variables and Constraints
⊲ Objective Function and Constraints File
⊲ Gradient of Objective and Jacobian of Constraints
⊲ Sparsity Pattern of Objective/Constraints
• NLP Functions (Rows)
⊲ First Row for Objective Function Gradient
⊲ Discretized Continuous Functions: Defect and Path Constraints
⊲ Discretized Point Functions: Boundary Conditions
• NLP Variables (Columns)
⊲ Components of State at LGR Points Plus Final Point
⊲ Components of Control at LGR Points Plus Final Point
⊲ Initial and Terminal Values of Time
Copyright © 2006–2012 Anil Vithala Rao Page 278
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
More Details on Sparsity Pattern
• Defect Constraints Consist of Main-Diagonals and Off-Diagonals
• Main Diagonal
⊲ Derivative of ith differential equation with respect to ith state
⊲ Contains differentiation matrix
• Off Diagonal
⊲ Derivative of ith differential equation with respect to jth state (i 6= j)
⊲ Does Not Contain Differentiation Matrix
• Derivatives of Defects with Respect to Controls: Same as Off-Diagonal
Defect
• Derivatives of Path Constraints: Same as Off-Diagonal Defect
• Boundary Conditions: Only Derivatives with Respect to Endpoint Values
of State and Time
Copyright © 2006–2012 Anil Vithala Rao Page 279
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Bounds on NLP Variables
• Bounds on Variables Corresponding to State (LGR Points Plus Final Point)
x0,min ≤ X1 ≤ x0,max ← Initial Condition=First LGR Point
xmin ≤ X2 ≤ xmax ← First Interior Point
......
xmin ≤ XN ≤ xmax ← Last Interior Point=Last LGR Point
xf,min ≤ XN+1 ≤ xf,max ← Terminal Point (Not an LGR Point)
• Bounds on Variables Corresponding to Control
umin ≤ U1 ≤ umax ← First LGR Point
...
umin ≤ UN ≤ umax ← Last LGR Point
• Bounds on Initial and Terminal Time
t0,min ≤ t0 ≤ t0,max ← Bounds on Initial Time
tf,min ≤ tf ≤ tf,max ← Bounds on Terminal Time
Copyright © 2006–2012 Anil Vithala Rao Page 280
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Main Diagonal Block of Defect Constraint
Block 1Nonlinear ElementLinear Element
Block 2
Block 3
Block K
Nonlinear ElementsLie on Diagonal of Differentiation Matrix
First LGR Point No Collocation
Last LGR Point
Copyright © 2006–2012 Anil Vithala Rao Page 281
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Off-Diagonal Block of Defect Constraint
First LGR Point No Collocation
Last LGR Point
Copyright © 2006–2012 Anil Vithala Rao Page 282
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Path Constraints and Boundary Conditions
• Path Constraints
⊲ Included Identically to Off-Diagonals of Defect Constraints
⊲ Result: Path Constraint Sparsity Pattern is Purely Nonlinear
• Boundary Conditions Are Functions of X1, XN+1, t0 and tf
Copyright © 2006–2012 Anil Vithala Rao Page 283
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Path Constraint Sparsity Pattern Block
First LGR Point No Collocation
Last LGR Point
Copyright © 2006–2012 Anil Vithala Rao Page 284
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Quantities on Diagonal Blocks
• State Diagonal Blocks[
D1:N −tf−t0
2diag
(
∂fi∂xi
)
DN+1
]
∈ RN×(N+1)
• State Off-Diagonal Blocks[
−tf−t0
2diag
(
∂fi∂xj
)
0]
∈ RN×(N+1)
• Control Blocks
−tf − t0
2diag
(
∂fi
∂uj
)
∈ RN×N
• State Path Blocks[
diag(
∂ci∂xj
)
0]
∈ RN×(N+1)
• Control Path Blocks
diag
(
∂ci
∂uj
)
∈ RN×N
Copyright © 2006–2012 Anil Vithala Rao Page 285
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Boundary Conditions With Respect to Initial and Terminal State
∂b1∂x1
01×(N−1)∂b1∂x2
01×(N−1)∂b1∂x3
· · · 01×(N−1)∂b1∂xn
∂b2∂x1
01×(N−1)∂b2∂x2
01×(N−1)∂b1∂x3
· · · 01×(N−1)∂b2∂xn
.
.....
.
.....
.
.....
∂bnb∂x1
01×(N−1)∂bnb∂x2
01×(N−1)∂b1∂x3
· · · 01×(N−1)∂bnb∂xn
• Final Two Columns for Derivatives With Respect to t0 and tf⊲ Derivatives with Respect to t0
− 12[f1]
1N +
tf−t02
∂f1∂t0
...
− 12[fnx ]
1N +
tf−t02
∂fn∂t0
∂c1∂t0...
∂cnc∂t0∂b1∂t0...
∂bnb∂t0
Copyright © 2006–2012 Anil Vithala Rao Page 286
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
⊲ Derivatives with Respect to tf
12[f1]
1N +
tf−t02
[
∂f1∂tf
]1
N...
12[fn]
1N +
tf−t02
[
∂fn∂tf
]1
N∂c1∂tf
...∂cnc∂tf∂b1∂tf
.
..∂bnb∂tf
Copyright © 2006–2012 Anil Vithala Rao Page 287
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Complete Sparsity Pattern for RPM Constraint Jacobian
Def
ect
1
State 1 State 2 State ny
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Zeros or Diagonal
Block+D
-Matrix
Zeros or Diagonal
Block+D
-Matrix
D
efec
t 2
Def
ect
ny
Control 1 Control nu
Pat
h 1
Either Zeros or
Diagonal Block
Pat
h n
c
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Bo
un
dar
yC
on
dit
ion
s
Either Zeros orEndpoints only
Either Zeros orEndpoints only All Zeros
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Zeros or Diagonal
Block+D
-Matrix
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
Either Zeros or
Diagonal Block
t0 tf
Copyright © 2006–2012 Anil Vithala Rao Page 288
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example 1
• Minimize the Cost Functional
J =1
2
∫ tf
t0
(x2 + u2)dt
• Dynamic Constraint
x = f(x, u) = −x3 + u
• Boundary Conditions
x(0) = x0(↔ x1 = x0)
x(tf ) = xf (↔ xN+1 = xf )
• This Example: t0 and tf are Fixed
Copyright © 2006–2012 Anil Vithala Rao Page 289
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Bounds on NLP Variables
• Bounds on Variables Corresponding to State (LGR Points Plus Final Point)
x0 ≤ x1 ≤ x0 ← Initial Condition=First LGR Point
xmin ≤ x2 ≤ xmax ← First Interior Point
......
xmin ≤ xN ≤ xmax ← Last Interior Point=Last LGR Point
xf ≤ xN+1 ≤ xf ← Terminal Point (Not an LGR Point)
• Bounds on Variables Corresponding to Control
umin ≤ u1 ≤ umax ← First LGR Point
...
umin ≤ uN ≤ umax ← Last LGR Point
• Bounds on Initial and Terminal Time
t0,min ≤ t0 ≤ t0,max
tf,min ≤ tf ≤ tf,max
Copyright © 2006–2012 Anil Vithala Rao Page 290
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Sparsity Patterns for Hyper-Sensitive Problem
0 20 40 60 80 100 120 140 160 180
0
10
20
30
40
50
60
70
80
90
nz = 573
0 20 40 60 80 100 120 140 160 180
0
10
20
30
40
50
60
70
80
90
nz = 1805
Copyright © 2006–2012 Anil Vithala Rao Page 291
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Example 2
• Minimize the Cost Functional
J = −r(tf )
• Dynamic Constraints (µ = 1)
r = u , u =v2
r−
µ
r2+
T
mw1
v = −uv
r+
T
mw2 , m = −
T
g0Isp
• Boundary Conditions
r(0) = r0 = 1 , u(0) = 0
v(0) =√
µ/r0 = 1 , m(0) = 1
r(tf ) = rf = 1 , u(tf ) = 0
v(tf ) =√
µ/rf , m(tf ) = FREE
Copyright © 2006–2012 Anil Vithala Rao Page 292
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Mesh Refinement
• Usually Desire Solutions of a Specified Accuracy
• Critical to Properly Choose Mesh Points and Number of Collocation Points
• A Priori Knowledge Not Possible in General
• Iterative Method Required to Construct Efficient Mesh
• This Part of Course
⊲ Discussion of Mesh Refinement Algorithm
⊲ Specifically, will Discuss hp–Adaptive Mesh Refinement
Copyright © 2006–2012 Anil Vithala Rao Page 293
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Concept of hp–Adaptive Mesh Refinement
• Limitations with Global Pseudospectral Method
⊲ Accuracy Increased Simply By Increasing Degree of Global
Approximation
Computationally intractable as N increases due to global D Matrix
Error tolerance not achievable using global polynomial approximation
⊲ Need to Divide Time Interval Into Segments
Increases NLP sparsity
Makes high-accuracy solutions possible
• Efficient Implementations of Methods Require Use of
⊲ As few mesh intervals as possible
⊲ The lowest possible polynomial degree approximation in each mesh
interval
Copyright © 2006–2012 Anil Vithala Rao Page 294
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Generation of New Mesh
Mesh Iteration g
Mesh Iteration g+1
Mesh Points Added to Improve Accuracy
Unnecessary Mesh Points Deleted
New Mesh Generated to Improve Solution Accuracy
Degree of Approximating Polynomial in Each Mesh Interval May Change from
Grid g to Grid g+1
Copyright © 2006–2012 Anil Vithala Rao Page 295
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Setting an Initial Mesh
• Solution to General Problem Not Known A Priori
• In Particular, Following are Not Known:
⊲ Regions where solution changes rapidly
⊲ Locations of discontinuities in control or state derivative
• Strategy for Initial Mesh
⊲ Choose Small Number of Evenly Spaced Mesh Intervals
⊲ Use Low-Order Approximation in Each Interval
• Strategy Enables Fast Computation of Solution on First Grid
Copyright © 2006–2012 Anil Vithala Rao Page 296
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
How to Change or Add Mesh Intervals
• Recall Terminology:
⊲ Mesh intervals [sk−1, sk], k = 1, . . . ,K
⊲ Nk =degree of polynomial approximation in mesh interval k
• Mesh Intervals Can Be Changed in One of Two Ways
⊲ Degree of polynomial approximation can be increased
⊲ Location of sk−1 and/or sk can be changed
• Criteria for Changing Polynomial Degree Nk
⊲ Need to assess if solution in mesh interval k is smooth
⊲ If so, can use global polynomial approach and increase Nk
• Criteria for Changing sk−1 and/or sk
⊲ Need to assess if solution is either nonsmooth/rapidly changing
⊲ If so, need to determine locations where nonsmoothness occurs
⊲ Divide mesh interval at locations of predicted rapid changes
Copyright © 2006–2012 Anil Vithala Rao Page 297
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Determining if a Mesh Interval Needs to Be Modified
• Choose an Integer L
• Let(
s(k)1 , ..., s
(k)L
)
Be Arbitrary Points in Mesh Interval k
• Compute a Differentiation Matrix, D, Using(
s(k)1 , ..., s
(k)L
)
• Compute xi(s) at(
s(k)1 , ..., s
(k)L
)
Using D
• Compute Error in Dynamic Constraints at(
s(k)1 , ..., s
(k)L
)
∣
∣
∣x(k)i (t
(k)l )− f
(k)i (X
(k)l ,U
(k)l , t
(k)l )
∣
∣
∣= a
(k)li , i ∈ [1, . . . , L] j ∈ [1, . . . , n].
• Use ali To Determine if Mesh in Interval k Should Be Modified
Copyright © 2006–2012 Anil Vithala Rao Page 298
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Determining if a Mesh Interval Needs to Be Modified (Continued)
• Let x(k)m (s) Be Component of State with Largest Value of a
(k)li
• Let rmax Be a User-Defined Parameter
• Compute Curvature in Mesh Interval k
κ(k)(s) =|x
(k)m (s)|
∣
∣
∣
∣
[
1 + x(k)m (s)2
]3/2∣
∣
∣
∣
.
• Let κ(k)max =Maximum Curvature
• Let κ(k) =Average Curvature
• Let rk = κ(k)max/κ(k)
• If rk < rmax, Then Polynomial Degree Nk Must Be Increased
• If rk > rmax, Then Mesh Interval Must Be Divided
Copyright © 2006–2012 Anil Vithala Rao Page 299
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Increasing Degree of Polynomial or Refining Mesh
• Increasing Polynomial Degree
⊲ New Polynomial Degree, Nk, Computed As
Nk =M + ceil(log10(e(k)max/ǫ)) + 1
⊲ e(k)max =Maximum Value of a
(k)li
• Refining Mesh
⊲ New Number of Mesh Intervals Computed As
nk = ceil(2 log10(e(k)max/ǫ)),
⊲ Locations of New Mesh Points Determined from κ
Let ρ(τ) be given as
ρ(s) = cκ(s)1/3,
where c is a constant chosen so that∫ +1
−1
ρ(ζ)dζ = 1.
Copyright © 2006–2012 Anil Vithala Rao Page 300
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Let F (τ) be given as
F (s) =
∫ s
−1
ρ(ζ)dζ.
New Mesh Points, nk, Chosen So That
F (si) =i− 1
nk, 1 ≤ i ≤ nk + 1.
Copyright © 2006–2012 Anil Vithala Rao Page 301
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
hp–Adaptive Mesh Refinement Algorithm
1. Solve the NLP using the current mesh.
Begin: For k = 1, . . . ,K,
2. If e(k)max ≤ ǫ, then continue (proceed to next k).
3. If either rk ≥ rmax or Nk > M , then refine the kth mesh interval into nk
subintervals, Set the degree of the polynomials on each of the
subintervals to be M and proceed to the next k.
4. Otherwise, set the degree of the polynomials on the kth subinterval to
be Nk.
End: For k = 1, . . . ,K.
5. Return to Step 1 Until Error Tolerance is Met.
Copyright © 2006–2012 Anil Vithala Rao Page 302
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Schematic of Algorithm to Solve Optimal Control Problem
Generate New Mesh
Compute LGR Points, Weights, and Differentiation Matrices
Transcribe and Solve NLP
Untranscribe NLP Solution to Discrete
Optimal Control Problem
SolutionMeets ErrorTolerance?
YESNOQuit
Copyright © 2006–2012 Anil Vithala Rao Page 303
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Generation of Derivatives for NLP Solver
• NLP Solvers Need Derivatives
• Quasi-Newton NLP Solvers (e.g., SNOPT)
⊲ Objective Function Gradient
⊲ Constraint Jacobian
• Typical Methods for Generating NLP Derivatives
⊲ Finite-Differencing
⊲ Analytic Differentiation
⊲ Automatic Differentiation
Copyright © 2006–2012 Anil Vithala Rao Page 304
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Generating NLP Derivatives for a Pseudospectral Method
• Pseudospectral Methods Have Great Deal of Structure
• Key Point: NLP Derivatives Can Be Computed Very Efficiently
• Suppose NLP Given in Following Form
⊲ Minimize f(z)
⊲ Constraints: hmin ≤ h(z) ≤ hmax
• Need to Compute ∇zf and ∇zh
• Using Structure, Only Optimal Control Functions Need to Be Differentiated
• Details Provided in Recent Paper Accepted in Journal of Spacecraft and
Rockets
Copyright © 2006–2012 Anil Vithala Rao Page 305
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Benefit of Computing Derivative of Optimal Control Functions
• Differentiating NLP Functions Inefficient and Memory Intensive
• Optimal Control Functions Much Smaller Than NLP Functions
• Each Collocation Condition: Only Involves One Value of Right-Hand Side
• Optimal Control Functions Only Differentiated at Collocation Points
• Optimal Control Functions Can Be Differentiated Using Any Technique
⊲ Finite-Differencing
⊲ Complex-Step Derivative Approximation
⊲ Analytic Differentiation
⊲ Automatic Differentiation
• Complex-Step is Best Choice
⊲ Produces Extremely Accurate First Derivative
⊲ Perfect for Quasi-Newton NLP Solver
Copyright © 2006–2012 Anil Vithala Rao Page 306
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Complex-Step Derivative Approximation
• Basic Idea: Given a Function of a Complex Variable z = x+ iy
• If f(z) = u(x, y) + iv(x, y) is Analytic, Then f(z) Satisfies
Cauchy-Riemann Equations
∂u
∂x=
∂v
∂y,
∂u
∂y= −∂v
∂x
• We Then Obtain Following Result (Where h is Real):
∂u
∂x= lim
h→0
v(x+ i(y + h))− v(x+ iy)
h
• Setting y = 0, u(x) = f(x), and v(x) = 0, We Obtain
df
dx= lim
h→0
Im[f(x+ ih)]
h
• Thereforedf
dx≈ Im[f(x+ ih)]
h
• Notice That Approximation Has No Subtraction
• Thus, Complex-Step Can Be Extremely Small
Copyright © 2006–2012 Anil Vithala Rao Page 307
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
• Also, Derivative is Accurate to O(h2)
• Easy to Implement on a Computer Using Complex Arithmetic
⊲ All Analytic Functions Defined for Complex Arguments
⊲ Must Avoid Non-Differentiable Functions abs, max, and min
Copyright © 2006–2012 Anil Vithala Rao Page 308
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Using Complex-Step in Pseudospectral Method
• At Endpoints of Interval, Compute Derivative Approximations of
⊲ Mayer Cost
⊲ Boundary Conditions
• At Collocation Points, Compute Derivative Approximations of
⊲ Lagrange cost
⊲ Right-hand side of differential equations
⊲ Path constraints
• Insert Optimal Control Derivatives Into NLP Derivative Functions
Copyright © 2006–2012 Anil Vithala Rao Page 309
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
General Approach to Software Design
• Suppose We Are Given Mesh Points (s1, . . . , sK) on s ∈ [−1,+1]
• Develop Following Subroutines First Given (s1, . . . , sK)
⊲ LGR Points, Weights, and Differentiation Matrix
⊲ NLP Sparsity Pattern Generator
⊲ NLP Functions
Objective Function: Discrete Version of Bolza Cost
Constraint Function: Defects, Paths, Boundary Conditions
⊲ SNOPT Wrapper, Where SNOPT Calls Objective and Constaints
• Repeat Above Steps
⊲ Using (s1, . . . , sK) computed from mesh refinement
⊲ Until user-specified accuracy tolerance on mesh is met
Copyright © 2006–2012 Anil Vithala Rao Page 310
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Part VIII: Wrap-Up
Copyright © 2006–2012 Anil Vithala Rao Page 311
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Review of Topics Covered
• Calculus of Variations
• Optimal Control Theory
• Nonlinear Optimization
• Numerical Integration and Interpolation
• Foundations of Pseudospectral Methods
• Practical Implementation Using Radau Pseudospectral Method
Copyright © 2006–2012 Anil Vithala Rao Page 312
May Be Redistributed Provided Copies Are Not Made for Profit or Direct Commercial Advantage
Summary
• Numerical Methods for Differential Equations
⊲ Single-Step Methods
⊲ Multiple-Step Methods
⊲ Multiple-Stage Methods
• Explicit and Implicit Schemes
⊲ Explicit
Easier to Implement
Has Poorer Stability Properties
⊲ Implicit
More Work to Implement
Much Better Stability and Accuracy
Copyright © 2006–2012 Anil Vithala Rao Page 313