heinz sch˜attler urszula ledzewicz - siueuledzew/papers/avner.pdf · 2012. 9. 25. · this paper...

Manuscript submitted to Website: http://AIMsciences.orgAIMS’ JournalsVolume X, Number 0X, XX 200X pp. X–XX

LYAPUNOV–SCHMIDT REDUCTIONFOR OPTIMAL CONTROL PROBLEMS

Heinz SchättlerDept. of Electrical and Systems Engineering

Washington UniversitySt. Louis, Mo 63130, USA

Urszula LedzewiczDept. of Mathematics and Statistics

Southern Illinois University at EdwardsvilleEdwardsville, Il 62026, USA

This paper is dedicated to Avner Friedman on the occasion of his 80th birthday.

(Communicated by Aim Sciences)

Abstract. In this paper, we use the method of characteristics to study sin-gularities in the flow of a parameterized family of extremals for an optimalcontrol problem. By means of the Lyapunov–Schmidt reduction a characteri-zation of fold and cusp points is given. Examples illustrate the local behaviorsof the flow near these singular points. Singularities of fold type correspond tothe typical conjugate points as they arise for the classical problem of minimumsurfaces of revolution in the calculus of variations and local optimality of tra-jectories ceases at fold points. Simple cusp points, on the other hand, generatea cut-locus that limits the optimality of close-by trajectories globally to timesprior to the conjugate points.

1. Introduction. In optimization problems on function spaces, there still existsignificant gaps between the theories of necessary and sufficient conditions for op-timality. For a finite-dimensional deterministic optimal control problem, the mainnecessary conditions for optimality are given by the Pontryagin maximum prin-ciple (for example, see [12, 3, 15]), a Lagrange multiplier type result. Sufficientconditions, on the other hand, center around the value function and are based onthe dynamic programming principle and solutions to the Hamilton–Jacobi–Bellman(HJB) equation, the combination of a first-order PDE with a finite-dimensional min-imization problem. The connection between these two vastly different approachesis made through the method of characteristics: the necessary conditions for op-timality given in the Pontryagin maximum principle also define the characteristicequations for the Hamilton–Jacobi–Bellman equation (for instance, see [1]). Thisallows to construct smooth solutions to the Hamilton–Jacobi–Bellman equation inregions in the state space that are covered diffeomorphically by a field of extremals—controlled trajectories that satisfy the necessary conditions for optimality and aregenerated through an analysis of the conditions of the maximum principle. In gen-eral, however, except for few exceptional cases like the linear–quadratic regulator

2000 Mathematics Subject Classification. Primary: 49J15; Secondary: 93C50.Key words and phrases. optimal control, maximum principle, Lyapunov–Schmidt reduction,

fold and simple cusp singularities.

1

2 HEINZ SCHÄTTLER AND URSZULA LEDZEWICZ

problem, such a covering will not exist globally since the value function for anoptimal control problem typically exhibits singularities. The difficulties in findingsolutions to optimal control problems, or, equivalently, in finding solutions to theHamilton–Jacobi–Bellman equation, precisely lie in analyzing these sets where thevalue function loses differentiability or is not even continuous. The latter is relatedto questions about local controllability of the system, the former to singularities inthe flow of extremals.

In this paper, we outline how the method of characteristics can be used to studysingularities in the flow of a parameterized family of extremals. The Lyapunov–Schmidt reduction procedure is systematically employed to give easily verifiable,explicit characterizations for the two most common (least degenerate) type of thesesingularities, fold and simple cusp points. These two cases are commonly encoun-tered in low-dimensional optimal control problems (related to the fact that theseare the only generic singularities in dimension 2 [16]) and often form determiningstructures in the optimal solutions. Singularities that are of fold type correspond tothe typical picture of conjugate points as arises in the classical problem of determin-ing the minimum surfaces of revolution in the calculus of variations and, as in thatexample, local optimality of trajectories ceases at fold points. However, a differentgeometric picture emerges near a simple cusp point. In this case, the local flow ofextremals generates a cut-locus between the trajectories which creates a shock inthe solution of the Hamilton–Jacobi–Bellman equation. This has global implicationsand trajectories lose optimality already at times prior to the conjugate point. Thesegeometric properties have already been the topic of the paper [9] where the localstructure of the value function near these singularities was analyzed for systemsin normal form. Here we complement those technical constructions with simpleexamples that illustrate the local geometric properties of the parameterized flow ofextremals and the resulting implications on the optimality of controlled trajecto-ries will be discussed. For these examples, all required computations can readilybe done explicitly. The use of the Lyapunov–Schmidt reduction for the optimalcontrol problem introduced in this paper provides some of the fundamental under-lying steps missing in the earlier constructions in [9]. We especially highlight theconnections between the conditions of the maximum principle and the eigenvectorsfor eigenvalue 0 that arise in the Lyapunov–Schmidt characterizations of corank 1singular points.

2. Parameterized Families of Extremals. We briefly review the method ofcharacteristics in optimal control. In order to simplify the presentation, and sinceit brings out the main features more clearly without unnecessary technical burden,we consider a fixed horizon optimal control problem without terminal constraints.The reader is referred to our text [15] for the general formulation.

[OC]: For a fixed terminal time T and given initial conditions (t0, x0) ∈ R× Rn,minimize the functional

J (u) =∫ T

t0

L(s, x(s), u(s))ds + ϕ(x(T )) (1)

over all locally bounded, Lebesgue measurable functions u that take valuesin a prescribed control set U , u : [t0, T ] → U ⊂ Rm, t 7→ u(t), for which thesolution x of the initial value problem

ẋ(t) = f(t, x(t), u(t)), x(t0) = x0, (2)

LYAPUNOV–SCHMIDT REDUCTION FOR OPTIMAL CONTROL PROBLEMS 3

exists over the full interval [t0, T ].

We assume that all the functions defining the data of the problem formulationare continuous and r-times continuously differentiable in x and u with r ≥ 1. Inprinciple, the control set U can be an arbitrary subset of Rm, but the situationswe study here typically arise for controls that take values in the interior of thecontrol set and in all our examples U = Rm. We call a locally bounded, Lebesguemeasurable function u with values in U (a.e.) an admissible control and the solutionx of the differential equation (2) for u is the corresponding trajectory; the pair (x, u)is called a controlled trajectory. Also, H denotes the control Hamiltonian definedby,

H = H(t.λ0, λ, x, u) = λ0L(t, x, u) + λf(t, x, u). (3)

with λ0 ∈ R and λ an n-dimensional row vector, λ ∈ (Rn)∗.Necessary conditions for optimality are given by the Pontryagin maximum princi-

ple [12, 3, 15]. If (x∗, u∗) is an optimal controlled trajectory defined over the interval[t0, T ], then there exist a constant λ0 ≥ 0 and a co-vector λ : [t0, T ] → (Rn)∗, theso-called adjoint variable, such that the following conditions are satisfied:

1. Nontriviality of the multipliers: (λ0, λ(t)) 6= 0 for all t ∈ [t0, T ];2. Adjoint equation: the adjoint variable λ is a solution to the time-varying linear

differential equation

λ̇(t) = −∂H∂x

(t, λ0, λ(t), x∗(t), u∗(t)) (4)

= −λ0Lx(t, x∗(t), u∗(t))− λ(t)fx(t, x∗(t), u∗(t))3. Minimum condition: everywhere in [t0, T ] we have that

H(t, λ0, λ(t), x∗(t), u∗(t)) = minv∈U

H(t, λ0, λ(t), x∗(t), v). (5)

4. Transversality condition:

λ(T ) = λ0∇ϕ(x∗(T )). (6)A parameterized family of extremals is a collection of controlled trajectories

and multipliers that satisfy these conditions. More specifically, we want that theparameterizations are smooth and we assume that with p a parameter, the controlsu = u(t, p) are continuous and for t fixed are r-times continuously differentiable inthe parameter p with the partial derivatives continuous in (t, p). We write u ∈ C0,rfor this class of functions. In principle, the parameter p could be anything, butfor problem [OC] there is a canonical choice taking the value of the trajectory atthe endpoint, p = x(T ), and then integrating the dynamics and adjoint equationbackward from the terminal T while maintaining the minimum condition. Thisleads to the following formal definition:

Definition 2.1. (Cr-parameterized family of extremals for [OC]) Given anopen subset P of Rn and an r-times continuously differentiable function t− : P →(−∞, T ), t 7→ t−(p), let D = {(t, p) : p ∈ P, t−(p) ≤ t ≤ T}. A Cr-parameterizedfamily E of extremals (or extremal lifts) with domain D consists of

1. a family of controlled trajectories (x, u) : D → Rn×U , (t, p) 7→ (x(t, p), u(t, p)),such that u ∈ C0,r(D) and

ẋ(t, p) = f(t, x(t, p), u(t, p)), x(T, p) = p. (7)


2. a non-negative multiplier λ0 ∈ Cr−1(P ) and co-state λ : D → (Rn)∗, λ =λ(t, p), so that (λ0(p), λ(t, p)) 6= (0, 0) for all (t, p) ∈ D and the adjoint equa-tion,

λ̇(t, p) = −λ0(p)Lx(t, x(t, p), u(t, p))− λ(t, p)fx(t, x(t, p), u(t, p)), (8)is satisfied on the interval [t−(p), T ] with terminal condition

λ(T, p) = λ0(p)∂ϕ

∂x(p) (9)

3. the controls u = u(t, p) solve the minimization problem

H(t, λ0(p), λ(t, p), x(t, p), u(t, p)) = minv∈U

H(t, λ0(p), λ(t, p), x(t, p), v); (10)

This definition provides the framework for our constructions. It merely formalizesthat all controlled trajectories in the family E satisfy the conditions of the maximumprinciple while some smoothness properties are satisfied by the parametrization.Analogous definitions can be given for general formulations of the optimal controlproblem (see, for instance, [13, 15]), but will not be needed for the purpose ofthis paper. A similar approach was pursued by L.C. Young under the notion of“lines of flights” and “concourse of flight” [17], but our formulations avoid theimprecise notion of a “descriptive” mapping. Note that it has not been assumedthat the parametrization E of extremals covers the state-space injectively and ourobjective precisely is to use this framework to analyze the geometry of the flow ofthe associated controlled trajectories as injectivity becomes lost (e.g., at conjugatepoints).

The degree r in the definition denotes the smoothness of the parametrizationof the controls in the parameter p, u ∈ C0,r. The control u extends as a C0,r-function onto an open neighborhood of D and it thus follows from classical resultsabout solutions to ODEs that the trajectories x(t, p) and their time-derivativesẋ(t, p) are r-times continuously differentiable in p and that these derivatives arecontinuous jointly in (t, p) in an open neighborhood of D, i.e., x ∈ C1,r(D). Forthe adjoint variable there is a loss of differentiability since the Lagrangian L andthe dynamics f are differentiated in x. The condition λ0 ∈ Cr−1(P ) implies thatthe boundary values λ(T, p) also lie in Cr−1(P ) and thus the multipliers lie in λ ∈C1,r−1. In particular, for a C1-parameterized family of extremals only continuity inp is required. If the data defining the problem [OC] possess an additional degree ofdifferentiability in x and if the the multiplier λ0 is r-times continuously differentiablewith respect to p, then it follows that λ ∈ C1,r as well. In such a case, we call E anicely Cr-parameterized family of extremals. Also, if λ0(p) > 0 for all p ∈ P , thenall extremals are normal and by diving by λ0(p) we may assume that λ0(p) ≡ 1 andwe call such a family normal.

Definition 2.2. (flow of controlled trajectories) Let E be a Cr-parameterizedfamily of extremals. The flow associated with the controlled trajectories (x, u) is themapping

z : D → R× Rn, (t, p) 7−→ z(t, p) =(

tx(t, p)

),

i.e., is defined in terms of the graphs of the corresponding trajectories. We say theflow z is a C1,r-mapping on an open set Q ⊂ D if the restriction of z to Q is con-tinuously differentiable in (t, p) and r times differentiable in p with derivatives thatare jointly continuous in (t, p). If z ∈ C1,r(Q) is injective and the Jacobian matrix


Dz(t, p) is nonsingular everywhere on Q, then we say z is a C1,r-diffeomorphismonto its image z(Q).

Definition 2.3. (parameterized cost or cost-to-go function) Given a Cr-parameterized family E of extremals with domain D, the corresponding parameter-ized cost or cost-to-go function is defined as

C : D → R, (t, p) 7→ C(t, p) =∫ T

t

L(s, x(s, p), u(s, p)) ds + ϕ(p).

It represents the value of the objective J(u) for the control u = u(·, p) if the initialcondition at time t is given by x(t, p).

The results given below are all classical. For proofs that employ the frameworkpresented here we refer the interested reader to our text [15] or the paper [10].Essential in the constructions is the following relation between the multiplier λ andthe cost C.

Lemma 2.1. (shadow price lemma) [10, 15] Let E be a C1-parameterized familyof extremal lifts with domain D. Then for all (t, p) ∈ D

λ0(p)∂C

∂p(t, p) = λ(t, p)

∂x

∂p(t, p) (11)

For normal extremals, the shadow price lemma implies that the cost-to-go func-tion is a classical solution to the Hamilton–Jacobi–Bellman equation on a regionG of the state space that is covered injectively by the corresponding flow of tra-jectories. If z is an injective C1,r-map from some open set Q ⊂ D onto a regionG ⊂ R× Rn, then z is a C1,r-diffeomorphism if and only if the Jacobian matrix∂x∂p is non-singular on D. In this case, for t fixed, the map z(t, ·) : p 7→ x(t, p)is a Cr-diffeomorphism and its inverse z−1(t, ·) : x 7→ z−1(t, x) also is r-timescontinuously differentiable in x. Then the value V E = V corresponding to theparameterized family can be defined in the state space as

V E = V : G → R, V = C ◦z−1,and is continuously differentiable. The shadow price lemma allows to identify themultiplier λ(t, p) with the gradient ∂V

E∂x (t, x(t, p)) and the minimum condition (10)

then becomes the Hamilton–Jacobi–Bellman equation.

Theorem 2.1. [10] Let E be a Cr-parameterized family of normal extremals forproblem [OC] and suppose the restriction of the flow z to some open set Q ⊂ D isa C1,r-diffeomorphism onto an open subset G ⊂ R× Rn of the (t, x)-space. Thenthe function

V E = V : G → R, V = C ◦z−1,is continuously differentiable in (t, x) and r-times continuously differentiable in xfor fixed t. The function

u∗ : G → R, u∗ = u ◦z−1,is an admissible feedback control that is continuous and r-times continuously dif-ferentiable in x for fixed t. Together, the pair (V, u∗) is a classical solution of theHamilton–Jacobi–Bellman equation,

∂V

∂t(t, x) + min

u∈U

{∂V

∂x(t, x)f(t, x, u) + L(t, x, u)

}≡ 0, (12)


on G. Furthermore, the following identities hold in the parameter space on Q:∂V

∂t(t, x(t, p)) = −H(t, λ(t, p), x(t, p), u(t, p)) (13)

∂V

∂x(t, x(t, p)) = λ(t, p). (14)

Definition 2.4. (local field of extremals) A Cr-parameterized local field of ex-tremals F for problem [OC] is a Cr-parameterized family of normal extremals suchthat the associated flow z : D → R× Rn, (t, p) 7→ z(t, p), is a C1,r-diffeomorphism.

Since we do not impose terminal constraints at the final time, the flow z extendsas a C1,r-diffeomorphism onto an open neighborhood of the domain D. We justremark that this definition and the entire construction can easily be modified toallow for terminal constraints given by a smooth embedded submanifold, even ifthe terminal time T is defined implicitly as considered in [4].

Corollary 2.1. [10] Let F be a Cr-parameterized local field of extremals for problem[OC] and suppose the associated flow z covers a domain G. Then, given any initialcondition (t0, x0) ∈ G, x0 = x(t0, p0), the open-loop control ū(t) = u(t, p0), t0 ≤ t ≤T , is optimal when compared with any other admissible controlled trajectory (x, u)with the same initial condition for which the graph of x lies in G.

Hence the local optimality of a controlled trajectory (x(·, p0), u(·, p0)) is con-nected with the local injectivity of the flow z along this trajectory and thus closelyrelated to the regularity of the differential of the flow, Dz(·, p0). The followingobservation follows from a standard compactness argument.

Lemma 2.2. If Dz(t, p0), or, equivalently, ∂x∂p (t, p0), is nonsingular over an in-terval [τ, T ], then there exists a neighborhood P of p0 so that the restriction of theflow z to [τ, T ]× P is a C1,r-diffeomorphism.Proof. It follows from the inverse function theorem that for every time s ∈ [τ, T ]there exists a neighborhood Ds of (s, p0) such that the restriction of z to Ds is aC1,r-diffeomorphism. Without loss of generality we may take Ds in the form Ds =Is × Ps where Is is an open interval and Ps an open neighborhood of p0. The sets{Ds : s ∈ [τ, T ]} form an open cover of the compact line segment [τ, T ]× {p0} andthus there exists a finite subcover {Dsi : si ∈ [τ, T ], i = 1, . . . , r}. Let P =

⋂ri=1 Psi .

Then the map z is a C1,r-diffeomorphism on D = {(t, p) : p ∈ P, τ ≤ t ≤ T}.For, if z(s1, p1) = z(s2, p2), then, since the flow map z is defined in terms of thegraphs of the trajectories, we have s1 = s2 and this time lies in one of the intervalIsi . But z ¹ Isi ×P is a C1,r-diffeomorphism and since both p1 and p2 lie in P , wehave p1 = p2 as well. Thus z is injective on D. Furthermore, z has a differentiableinverse by the inverse function theorem. ¤

In this case, the controlled reference trajectory (x̄, ū) = (x(·, p0), u(·, p0)) thusgives a strong local minimum for the problem [OC] in the following classical form:there exists an ² > 0 such that for any other admissible controlled trajectory (x, u)with the same initial condition, x(τ) = x̄(τ), that satisfies ‖x(t)− x̄(t)‖ < ² forall t ∈ [τ, T ], we have that J (ū) ≤ J (u). The regularity of the flow z alongthe reference controlled trajectory (x̄, ū) thus is the optimal control version of thestrengthened Jacobi condition and, as in the calculus of variations, it can equiv-alently be formulated in terms of the Jacobi equation or solutions to equivalentRiccati differential equations (see, for example, [4, Chapter 6] or [15]).


3. Singular Points and the Lyapunov–Schmidt Reduction. Loss of localoptimality often is in some way, directly or indirectly, related to points where thedifferential Dz(t, p0) becomes singular.

Definition 3.1. (singular points) Let E be a C1-parameterized family of normalextremals. A point (t0, p0) ∈ D, t0 < T , is a corank ` > 0 singular point if thematrix

Dz(t0, p0) =(

1 0∂x∂t (t0, p0)

∂x∂p (t0, p0)

),

or equivalently, ∂x∂p (t0, p0), has rank n− `. We call a singular point (t0, p0) t-regularif for ∆(t, p) = det (Dz(t, p)) we have that ∂∆∂t (t0, p0) 6= 0.

For a t-regular singular point, by the implicit function theorem, there exists anopen neighborhood W = (t0 − ε, t0 + ε)×Bδ(p0) of (t0, p0), with the property thatthe equation ∆(t, p) = 0 has a unique solution on W given in the form t = σ(p)with a continuously differentiable function σ : Bδ(p0) → (t0 − ε, t0 + ε), p 7→ σ(p).This is the situation we are interested in for an optimal control problem. Since therank of a matrix is a lower semicontinuous function, if (t0, p0) is a t-regular corank1 singular point, then for δ sufficiently small all points in the singular set will be ofcorank 1. He henceforth assume that this is the case, i.e.,

(A): The singular set S is a codimension 1 embedded submanifold of D thatentirely consists of corank 1 singular points and can be described as the graphof a continuously differentiable function σ : P → D, p 7→ σ(p),

S = {(t, p) ∈ int(D) : t = σ(p)} = gr(σ).Lemma 3.1. Under assumption (A), there exist a nonzero C1 right-eigenvectorfield v : P → Rn, v = v(p), and a nonzero C1 left-eigenvector field w : P → (Rn)∗,w = w(p), for the eigenvalue 0:

(∂x

∂p(σ(p), p)

)v(p) = 0 and w(p)

(∂x

∂p(σ(p), p)

)= 0.

This immediately follows from the implicit function theorem and v and w aresimply smooth selections of left- and right-eigenvectors for the eigenvalue 0. Forthe optimal control problem there exists a remarkable relation between these left-and right-eigenvectors that follows from the useful fact below, itself a corollary ofthe shadow price lemma, Lemma 2.1.

Proposition 3.1. For any C2-parameterized family of normal extremals, the matrix

Ξ(t, p) =∂λ

∂p(t, p)

∂x

∂p(t, p) (15)

is symmetric.

Proof. By the shadow price lemma, the partial derivative ∂C∂pj (t, p) of the parame-terized cost with respect to the parameter pj is given by

∂C

∂pj(t, p) =

n∑

k=1

λk(t, p)∂xk∂pj

(t, p) = λ(t, p)∂x

∂pj(t, p), j = 1, . . . , n,


where ∂x∂pj (t, p) is the column vector of the partial derivatives of the components ofx with respect to pj . Differentiating this equation with respect to pi gives

∂2C

∂pi∂pj(t, p) =

n∑

k=1

(∂λk∂pi

(t, p)∂xk∂pj

(t, p) + λk(t, p)∂2xk

∂pi∂pj(t, p)

).

For a C2-parameterized family of extremals, the mixed second partial derivativesof the functions C and xk are equal and thus we have that

n∑

k=1

(∂λk∂pi

(t, p)∂xk∂pj

(t, p))

=n∑

k=1

(∂λk∂pj

(t, p)∂xk∂pi

(t, p))

.

But these terms, respectively, are the (i, j) and (j, i) entries of the matrix Ξ. ¤

Corollary 3.1. Let

w(p) = vT (p)∂λ

∂p(σ(p), p). (16)

If nonzero, then w is a left-eigenvector field for the eigenvalue 0 of ∂x∂p (σ(p), p) on P .

Proof. We have that

w(p)∂x

∂p(σ(p), p) = vT (p)

∂λ

∂p(σ(p), p)

∂x

∂p(σ(p), p) = vT (p)Ξ(t, p)

= (Ξ(t, p)v(p))T =(

∂λ

∂p(σ(p), p)

∂x

∂p(σ(p), p)v(p)

)T= 0.

¤For the optimal control problem [OC] it can be shown that w is always nonzero

for a C1-parameterized family of extremals if the strengthened Legendre conditionis satisfied (i.e., if ∂H∂u ≡ 0 and ∂

2H∂u2 is positive definite along the reference controlled

extremal) [14, 15].In terms of the full differential of the flow map z, Dz(t, p), the right-eigenvector

vector field is given by

V : P → Rn+1, p 7→ V (p) =(

0v(p)

)

and corank 1 singularities are broadly classified as fold or cusp points dependingon whether this vector field V is transversal to the tangent space T(t0,p0)S of thesingular set S at (t0, p0) or not [7].

Definition 3.2. (fold and cusp points) A corank 1 singular point is called a foldpoint if

T(t0,p0)S ⊕ lin span {V (p0)} = Rn+1;it is called a cusp point if

V (p0) ∈ T(t0,p0)S.In our setup, we have the following simple criterion:

Lemma 3.2. The point (t0, p0), t0 = σ(p0), is a fold point if and only if the Liederivative of σ along the vector field v does not vanish at p0, i.e.,

Lvσ(p0) = ∇σ(p0)v(p0) 6= 0.


Proof. Since S is the graph of the function σ, S = {(t, p) : t−σ(p) = 0}, the tangentspace to S at (t0, p0) consists of all vectors that are orthogonal to (1,−∇σ(p0)). ¤

Singular points can be characterized in terms of the left- and right-eigenvectors vand w of the matrix ∂x∂p (t, p). This gives rise to a more convenient description of thesingular set S as the zero set of a scalar function known as the Lyapunov–Schmidtreduction [8]. Let

ζ : D → R, (t, p) 7→ ζ(t, p) = w(p)∂x∂p

(t, p)v(p). (17)

Note that with

w(p) = v(t)T∂λ

∂p(t, p),

this function is of the form

ζ(t, p) = w(p)∂x

∂p(t, p)v(p) = v(p)T Ξ(t, p)v(p) = 〈v(p), Ξ(t, p)v(p)〉

so that ζ is a symmetric quadratic form. Denote the zero set of ζ in D by Z. Clearly,Z contains the singular set S. If the gradient of ζ, ∇ζ, does not vanish at a singularpoint (t0, p0), t0 = σ(p0), then Z is an embedded n-dimensional manifold near(t0, p0) that contains S, under our assumptions itself an embedded n-dimensionalmanifold. Hence in a sufficiently small neighborhood, S and Z are equal. Thegradient of ζ at a point (σ(p), p) ∈ S is easily calculated: dropping the arguments,we have that

ζ = w∂x

∂pv =

n∑

i=1

n∑

j=1

wi∂xi∂pj

vj

and thus the partial derivative with respect to t is simply given by the quadraticform

∂ζ

∂t=

n∑

i=1

n∑

j=1

wi∂2xi∂t∂pj

vj = w∂2x

∂t∂pv.

The partial derivatives with respect to p simplify at a singular point since w and vare the left- and right-eigenvectors of ∂x∂p (σ(p), p). Generally,

∂

∂pk

(w

∂x

∂pv

)=

n∑

i=1

n∑

j=1

(∂wi∂pk

∂xi∂pj

vj + wi∂2xi

∂pk∂pjvj + wi

∂xi∂pj

∂vj∂pk

)

=n∑

i=1

∂wi∂pk

n∑

j=1

∂xi∂pj

vj

+

n∑

i=1

n∑

j=1

wi∂2xi

∂pk∂pjvj

+n∑

j=1

(n∑

i=1

wi∂xi∂pj

)∂vj∂pk

and, upon evaluating at a singular point, we obtain that

∂

∂pk

(w

∂x

∂pv

)=

n∑

i=1

n∑

j=1

wi∂2xi

∂pk∂pjvj

since both ∂x∂pv and w∂x∂p vanish. These partial derivatives can be expressed in a

more convenient and compact form if we consider the directional derivative in the


direction of a tangent vector z in p-space. For then we have that

〈∇pζ, z〉 =n∑

k=1

∂

∂pk

(w

∂x

∂pv

)zk =

n∑

k=1

n∑

i=1

wi

n∑

j=1

∂2xi∂pk∂pj

vj

zk

=n∑

i=1

wi

n∑

j=1

n∑

k=1

∂2xi∂pk∂pj

vjzk

=

〈w,

∂2x

∂p2(v, z)

〉= w

∂2x

∂p2(v, z).

Thus, this directional derivative is the inner product of the left eigenvector w withthe column vector ∂

2x∂p2 (v, z) whose entries are the quadratic forms of the second

derivatives of the components xi of the state with respect to p acting on the vectorsv and z. Summarizing this calculation, at a singular point (σ(p), p), p ∈ P , thedirectional derivative in direction of the vector z = (τ, z) can be expressed as

〈∇ζ(σ(p), p), z〉 =〈

w(p),∂2x

∂t∂p(σ(p), p)v(p)τ +

∂2x

∂p2(σ(p), p)

(v(p), z

)〉. (18)

We thus have the following statement:

Theorem 3.1. Suppose (t0, p0) is a t-regular corank 1 singular point for whichthe gradient ∇ζ(σ(p0), p0) does not vanish. Then there exist an open neighborhoodW = (t0 − ε, t0 + ε) × P of (t0, p0) and a C1-function σ defined on P , σ : P →(t0 − ε, t0 + ε) so that the singular set S is given by S = (t, p) ∈ W : t = σ(p)}.Given a C1 right-eigenvector field v : P → Rn, ∂x∂p (σ(p), p)v(p) ≡ 0, and a C1 left-eigenvector field w : P → (Rn)∗, w(p)∂x∂p (σ(p), p) ≡ 0, the set S can be describedas

S = {(t, p) ∈ W : ζ(t, p) = w(p)∂x∂p

(t, p)v(p) = 0} (19)and the tangent space to S at (σ(p), p) is given by

T(σ(p),p)S ={

z =(

τz

)∈ Rn+1 : (20)

w(p)(

∂2x

∂t∂p(σ(p), p)v(p) · τ + ∂

2x

∂p2(σ(p), p) (v(p), z)

)= 0.

}

Corollary 3.2. Under the same assumptions, (t0, p0) is a fold singularity if andonly if with v0 = v(p0) and w0 = w(p0) we have that

w0∂2x

∂p2(t0, p0) (v0, v0) 6= 0. (21)

Proof. The point (t0, p0) is a fold if and only if the vector V0 = (0, v0)T is transver-sal to S at (t0, p0). Assuming that the gradient ∇ζ(t0, p0) does not vanish, this isequivalent to w0 ∂

2x∂p2 (t0, p0) (v0, v0) 6= 0. Note that this condition by itself guarantees

that ∇ζ(t0, p0) does not vanish. ¤A singular point (t0, p0) of a C2-parameterized family E of normal extremals thus

is a cusp point if

w0∂2x

∂p2(t0, p0) (v0, v0) = 0. (22)

This extra equality constraint typically restricts the set of all cusp points tobe a lower-dimensional subset of S. In the least degenerate case, i.e., when no


additional equality relations are satisfied, the set C of cusp points is a codimension1 submanifold of S, hence a codimension 2 submanifold of D.

Definition 3.3. (simple cusp points) A cusp point (t0, p0), t0 = σ(p0), is calledsimple if the vector V0 = (0, v0)

T is transversal to the submanifold C of cusp pointsin T(t0,p0)S, i.e.,

T(t0,p0)C ⊕ lin span {V (p0)} = T(t0,p0)S.By Lemma 3.2, the set of cusp points is also given by

C = {(t, p) ∈ D : t = σ(p), Lvσ(p) = ∇σ(p)v(p) = 0} ,i.e., it is the set of points where the Lie derivative of the function σ in direction ofthe zero eigenvector v vanishes. If the gradient of this function in p does not vanishat p0, then C is a codimension 1 embedded submanifold of S and (t0, p0) ∈ C,t0 = σ(p0), is simple if and only if the second Lie derivative L2vσ of σ in directionof v does not vanish at p0. (In this case V (p0) is not tangent to the submanifolddefined by the equation Lvσ(p) = 0, but it is a tangent vector to S at (t0, p0).)Note that

L2vσ(p0) =

(∂

∂p |p0∇σ(p)v(p)

)· v0 = ∂

2σ

∂p2(p0)(v0, v0) +∇σ(p0)∂v

∂p(p0)v0

with ∂2σ

∂p2 (p0)(v0, v0) denoting the quadratic form defined by the Hessian matrix ofσ acting on the eigenvector v0 and ∂v∂p (p) the Jacobian matrix of the vector fieldv. In terms of the Lyapunov–Schmidt reduction, ζ(t, p) = w(p)∂x∂p (t, p)v(p), if thegradient ∇ζ(σ(p0), p0) does not vanish, then the set C of cusp points is given by

C ={

(t, p) ∈ W : w(p)∂x∂p

(t, p)v(p) = 0, w(p)∂2x

∂p2(t, p) (v(p), v(p)) = 0.

}

For example, the condition that ∇ζ(σ(p0), p0) 6= 0 can simply be guaranteed by∂ζ

∂t(t0, p0) = w0

∂2x

∂t∂p(t0, p0)v0 6= 0.

Making this assumption, simple cusp points can be described as follows:

Proposition 3.2. Suppose (t0, p0) is a t-regular corank 1 singular point for which∂ζ∂t (t0, p0) = w0

∂2x∂t∂p (t0, p0)v0 6= 0. Then (t0, p0) is a simple cusp if and only if

w0∂2x

∂p2(t0, p0) (v0, v0) = 0

and

w0

(∂3x

∂p3(t0, p0) (v0, v0, v0) + 3

∂2x

∂p2(t0, p0)

(∂v

∂p(p0)v0, v0

))6= 0 (23)

Proof. For all p ∈ P we have that∂x

∂p(σ(p), p)v(p) = 0.

Differentiating, and letting the derivative act upon v(p) gives

∂2x

∂t∂p(σ(p), p)v(p) · Lvσ(p) + ∂

2x

∂p2(σ(p), p) (v(p), v(p)) +

∂x

∂p(σ(p), p)

∂v

∂p(p)v(p) = 0.


Multiplying on the left with the left-eigenvector w(p) annihilates the last term.Furthermore, for a cusp point Lvσ(p) = 0, and thus w0 ∂

2x∂p2 (t0, p0) (v0, v0) = 0

follows. Now differentiate this relation once more, evaluate the result at the cusppoint, and multiply with v(p) on the right and with w(p) on the left. When doing so,all terms that include Lvσ(p0), w0 ∂x∂p (t0, p0) or w0

∂2x∂p2 (t0, p0) (v0, v0) vanish. This

leaves us with(w0

∂2x

∂t∂p(t0, p0)v0

)L2vσ(p0)+

w0∂3x

∂p3(t0, p0) (v0, v0, v0) + 3w0

∂2x

∂p2(t0, p0)

(∂v

∂p(p0)v0, v0

)= 0.

By assumption, w0 ∂2x

∂t∂p (t0, p0)v0 is nonzero and thus L2vσ(p0) 6= 0 if and only if (23)

holds. ¤

Corollary 3.3. The tangent space to the submanifold C of cusp points at (σ(p), p)is given by

T(σ(p),p)C ={

z =(

τz

)∈ Rn+1 : w(p)

(∂2x

∂t∂p(σ(p), p)v(p) · τ (24)

+∂3x

∂p3(σ(p), p) (v(p), v(p), z) + 3

∂2x

∂p2(σ(p), p)

(∂v

∂p(p)v(p), z

))= 0

}.

4. Fold Singularities and Conjugate Points. The name for the fold singularityhas its origin in the geometric properties of the mapping at such a point thatresemble those of a quadratic function. These mapping properties are exactly thesame as in the classical problem for the minimum surfaces of revolution in thecalculus of variations if only smooth extremals are considered. In the language ofoptimal control, this problem can be formulated as to minimize an objective of theform

J(u) =∫ T

0

x√

1 + u2dt (25)

over all controls u : [0, T ] → R subject to the trivial dynamics ẋ = u and fixedboundary conditions. If we fix the initial point to be x(0) = 1, then it is well-knownthat the extremals are catenaries [2] and we can parameterize the full family in theform

x(t, p) =cosh (p + t cosh p)

cosh p, p ∈ R, t > 0. (26)

Figure 1 on the left depicts this family and on the right the corresponding pa-rameterized value V E is shown as the graph of a multivalued function defined over(t, x)-space clearly showing the fold property along the envelope of the family.

We give another simple one-dimensional example which, like the catenaries, en-compasses all the features of fold singularities. It fits the model discussed here andhas the advantage that in contrast to the minimum surface problem all compu-tations are readily done explicitly. The normal form of the fold is built into thepenalty term ϕ in the objective.

[Fold]: For a fixed terminal time T , minimize the objective

J(u) =12

∫ Tt0

u2dt +13x(T )3


0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Figure 1. The family of catenaries (left) for the problem of findingminimum surfaces of revolution in the calculus of variations and thecorresponding multivalued function V E = C ◦z−1 (right).

over all piecewise continuous functions u : [t0, T ] → R subject to the dynamicsẋ = u.

It is straightforward to define a real-analytic parameterized family E of extremalsfor this problem: The Hamiltonian is given by H = 12λ0u

2 + λu and the adjointequation is λ̇ ≡ 0 with terminal condition λ(T ) = λ0x(T )2. The nontrivialitycondition on the multipliers implies that extremals are normal and we set λ0 ≡ 1.Furthermore, multipliers and controls are constant. We choose as domain D for theparameterization D = {(t, p) : t ≤ T, p ∈ R} with p denoting the terminal point,p = x(T ). All extremals can then be described in the form

x(t, p) = p + (T − t)p2, u(t, p) = −p2, λ(t, p) = p2,and the parameterized cost is given by

C(t, p) =12(T − t)p4 + 1

3p3.

This defines a real-analytic parameterized family of extremals.For this 1-dimensional problem the singular set is given by the solutions to

∂x∂p (t, p) = 0 and formally left- and right-eigenvector fields are given by the constantsv(t, p) ≡ 1 and w(t, p) ≡ 1, but we also could take w(t, p) = v(t, p)T ∂λ∂p (t, p) = 2p 6=0. Thus

S = {(t, p) ∈ D : t = σ(p) = T + 12p

, p < 0}

and since ∂2x

∂p2 (t, p) = 2(T − t) > 0 all singular points are fold points. Figure 2 showsthe flow of extremals for p < 0.

Figure 3 shows four slices of the value function in the state space for t = 0.6,t = 0.8, t = 1, and t = 1.6. For t = 1.6 the fold point does not lie in the rangeshown and thus the corresponding value function V E = C ◦ z−1 is single-valued.For the other three time slices, the value function is multivalued and shows thecharacteristic behavior near a fold singularity.

These mapping properties can be established in general near a fold point. LetM = z(S) be the image of the singular manifold S under the flow z. It fol-lows from the chain rule that the tangent space to M at the point z(σ(p), p) =


0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

time t

sta

te x

x(t,p)

Figure 2. The flow of the parameterized family of extremals Eover the interval [0, T ] for T = 2 and p < 0.

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

V = V(0.6,⋅)

state x

va

lue

V

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

V = V(0.8,⋅)

state x

va

lue

V

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

V = V(1,⋅)

state x

va

lue

V

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

state x

va

lue

V

V = V(1.6,⋅)

Figure 3. Slices of the value corresponding to the parameterizedfamily E of extremals for t = const for the problem [Fold].

(σ(p), x(σ(p), p)) is given by the image of the tangent space to S, T(σ(p),p)S, un-der the differential Dz. At a corank 1 singular point the differential Dz(t, p) hasrank n with null-space spanned by V (p) = (0, v(p))T . For a fold point this vec-tor does not lie in the tangent space to S at (σ(p), p) and therefore the imageTz(σ(p),p)M = Dz

(T(σ(p),p)S

)is n-dimensional. In other words, the fold condi-

tion guarantees that the restriction of the mapping z to its singular set S is a


diffeomorphism and thus M also is an n-dimensional manifold. Furthermore,(−w(p)∂x

∂t(σ(p), p), w(p)

)Dz(σ(p), p) = (0, 0)

and thus n =(−w(p)∂x∂t (σ(p), p), w(p)

)is a normal vector to Tz(σ(p),p)M .

Let γ± : [0, ε] → D, s 7→ (σ(p), p ±√

sv(p), be a small line-segment in theparameter space starting at the point (σ(p), p) ∈ S. Since (0, v(p))T is transversalto S, by making the neighborhood P of p0 smaller, if necessary, we can assume thatγ±(s) /∈ S for all 0 < s ≤ ε and p ∈ P . Let φ± = z◦γ± be the images of the curvesγ± under the flow z. Since ∂x∂p (σ(p), p)v(p) = 0, by Taylor’s theorem we have that

x(σ(p), p±√sv(p)) = x(σ(p), p) + 12s∂2x

∂p2(σ(p), p) (v(p), v(p)) + o(s)

and thus

φ±(s) = z (γ±(s)) =(

σ(p)x(σ(p), p)

)+

12s

(0

∂2x∂p2 (σ(p), p) (v(p), v(p))

)+ o(s).

Taking the inner product of the tangent vector

φ̇± = φ̇±(0) =12

(0

∂2x∂p2 (σ(p), p) (v(p), v(p))

)

with the normal vector n to Tz(σ(p),p)M , we get that〈n,φ̇±

〉=

12w(p)

∂2x

∂p2(σ(p), p) (v(p), v(p))

which has constant nonzero sign on P . It thus follows that both curves φ+ and φ−point to the same side of the tangent space to M at z(σ(p), p). Thus, if we definethe curves γ : (−ε, ε) → D, s 7→ (σ(p), p + sv(p), and φ = z ◦ γ, then φ has order 1contact with M at φ(0), i.e., φ is tangent to M at φ(0), but lies to one side of M .In other words, the image touches M in the point φ(0) ∈ M , but then “folds” back.Since this holds for all controlled trajectories in a neighborhood of p0, it follows thatthe flow z is 2 : 1 near the singular set S. While the restriction z ¹ S is 1 : 1 andmaps S diffeomorphically onto M , away from M the mapping is 2 : 1. There existopen neighborhoods V of (t0, p0) and G of (t0, x0) = (t0, x(t0, p0)) such that S splitsV into two connected components V+ and V−, V = V− ∪S ∪ V+, with the propertythat the flow z restricted to V+ or V− is a C1,2 diffeomorphism and the restrictionsmap V+, respectively V−, onto a region G+ ⊂ G, z(V+) = G+ = z(V−). Thesegeometric properties imply that fold points of the flow z are conjugate points in thesense of the calculus of variations and local optimality will be lost at the fold point.This can be shown in great generality, for example, by using a geometric argumentbased on envelopes that mimics the classical reasoning for the family of catenaries[14, 15].

5. Simple Cusp Singularities and Cut-Loci. The geometry of the flow nearcusp singularities is more intricate and so are its implications on local optimality ofthe controlled trajectories. We again illustrate these features for a one-dimensionalexample that incorporates the normal form of a simple cusp into the penalty termϕ in the objective.


[Simple Cusp]: [11] For a fixed terminal time T , minimize the objective

J(u) =12

∫ Tt0

u2dt +12

(x(T )4 − x(T )2) (27)

over all piecewise continuous functions u : [t0, T ] → R subject to the dynamicsẋ = u.

This simple regulator problem over a finite interval is related to Burgers’s equa-tion, a fundamental partial differential equation in fluid mechanics that is a pro-totype for equations that develop shock waves (e.g., see [5, 6]). The penalty termis a multi-modal function with a local maximum at x = 0 and global minima atx = ± 12

√2. As we shall see, this has significant implications.

Again it is straightforward to construct a real-analytic parameterized family Eof extremals. As in the example above, extremals are normal and the multipliersand controls are constant. A simple application of the conditions of the maximumprinciple shows that all extremals can be parameterized over D = {(t, p) : t ≤T, p ∈ R} with p denoting the terminal point, p = x(T ), in the form

x(t, p) = p + (t− T )(p− 2p3), u(t, p) = p− 2p3, λ(t, p) = 2p3 − p,and the parameterized cost becomes

C(t, p) =12(T − t)(p− 2p3)2 + 1

2(p4 − p2). (28)

All functions are polynomial and this allows explicit computations in the analysisof the flow z(t, p) = (t, x(t, p)). Since the parameter set is one-dimensional, thesingular set S is given by the solutions to the equation ∂x∂p (t, p) = 0 and solving fort gives

t = σ(p) = T − 11− 6p2 (29)

which is finite and less than T for |p| < 1√6. Since

∂2x

∂p2(t, p) = −12p(t− T ).

the map has fold singularities at (σ(p), p) for p ∈ (− 1√6, 1√

6), p 6= 0 and (t, p) =

(T −1, 0) is a simple cusp point. For, ∂2x∂t∂p (t, p) = 1−6p2 is positive on the singularset and ∂

3x∂p3 (T − 1, 0) = 12. Since ∂v∂p (p) ≡ 0, this is equivalent to the transversality

condition (23).A detailed analysis of the mapping properties of the flow z for this example has

been carried out in [11] and we briefly recall the important features. For α ≥ 0,define curves

Φα :(− 1√

α,

1√α

)→ R2, p 7→ (tα(p), p), tα(p) = T − 11− αp2 ,

and denote the half-curves for positive and negative values of p by Φ+α and Φ−α ,

respectively. Note that σ = t6 defines the singular set. Except for the trivialintersection in the point (T − 1, 0) which all curves have in common, the curves Φαdefine a foliation of the subset D̃ = {(t, p) ∈ D : t ≤ T − 1} and the image of D̃


under z is the subset G̃ = (−∞, T −1]×R of G. But there exist nontrivial overlapsin the map. An explicit calculation verifies that

x(tα(p), p) =(2− α)p31− αp2 (30)

and using p2 = 1α(1− 1T−tα

), the image of the curve Φα is given by

x = ±2− αα

√(T − t− 1)3

α (T − t) , t ≤ T − 1.

In particular, the image of the singular set S under the flow z is given by the curve

Gc = z(S) =

(t,±

√227

(T − t− 1)3(T − t) : t ≤ T − 1

,

a cusp, which gives the singularity its name. It follows from (30) that

z(t6(p), p) = z(t1.5(−2p),−2p), |p| < 1√6, (31)

and thus the curve Φ1.5 is also mapped onto GC . But note that the curve

G+ =

(t,

√227

(T − t− 1)3(T − t) : t < T − 1

= z

(Φ+6

)= z

(Φ−1.5

)

is the image of the semi-curves Φ+6 and Φ−1.5 while

G− =

(t,−

√227

(T − t− 1)3(T − t) : t < T − 1

= z

(Φ−6

)= z

(Φ+1.5

)

is the image of Φ−6 and Φ+1.5.

The regions and submanifolds defined below form a partition (in fact, a stratifi-cation [11]) of the domain of the parameterized family of extremals into embeddedanalytic submanifolds,

D = {DT , D0, D−1 , D01, D+1 , Dcp− , Dcp+ , D−, D+, Dc}

(see Figure 4), where DT = {T} × R, Dc = {(T − 1, 0)},

D0 = {(t, p) ∈ D : t1.5(p) < t < T},D+1 = {(t, p) ∈ D : t6(p) < t < t1.5(p), p > 0} ,D−1 = {(t, p) ∈ D : t6(p) < t < t1.5(p), p < 0} ,D01 = {(t, p) ∈ D : t < t6(p), p ∈ R},


and

D+ =

{(t, p) ∈ D : t = t1.5(p), 0 < p <

√23

},

D− =

{(t, p) ∈ D : t = t1.5(p), −

√23

< p < 0

},

Dcp− ={

(t, p) ∈ D : t = t6(p), 0 < p < 1√6

},

Dcp+ ={

(t, p) ∈ D : t = t6(p), − 1√6

< p < 0}

.

Accordingly, partition the range of z, G = z(D), into the open sets

G0 =

(t, x) ∈ G : |x| >

√227

(T − t− 1)3(T − t) , t ≤ T − 1

∪ (T − 1, T )× R,

G1 =

(t, x) ∈ G : |x| <

√227

(T − t− 1)3(T − t) , t < T − 1

,

the curves G− and G+ defined above, the cusp point Gc = {(T − 1, 0)}, and theterminal manifold GT = {T} × R, (see Figure 4),

G = {GT , G0, G−, G+, Gc, G1}.

T t

p

Φ1.5

Φ6

D10

D1+

D1−

p=0

D0 DT

Dc

D+

D−

D+cp

D−cp

Φ6

Φ1.5

x

t

G0=F(D

0)

G1=F(D

1)

G−=F(Φ

6+)=F(Φ

1.5− )

Gc=(T−1,0)

GT

x=0

G+=F(Φ

6−)=F(Φ

1.5+ )

T

Figure 4. Stratifications of the domain D (left) and the range G(right) that are compatible with the map z.

The embedded submanifolds in the collections D and G give a natural decompo-sition of the domain and the range of the flow of extremals into embedded subman-ifolds and z maps each submanifold in D diffeomorphically onto exactly one of thesubmanifolds in G. We say that these decompositions are compatible with the flow


map z. Specifically,GT = z(DT ), G0 = z(D0), Gc = z(Dc),G− = z(D−) = z(Dcp− ), G+ = z(D+) = z(D

cp+ ),

G1 = z(D−1 ) = z(D01) = z(D

+1 ).

Thus the flow z is 3 : 1 onto G1, 2 : 1 onto the branches G− and G+ of fold points,and 1 : 1 otherwise. In particular, for each initial point in G0 there only exists oneextremal and this one is optimal. But we need to see which of the three values isoptimal over G1. The parameterized cost C(t, p) for the family is easily evaluated.The value C along the curves Φα is given by

C(tα(p), p) =(4− α)p2 + (α− 3)

1− αp2 p4.

For α = 2 it follows that C(t2(p), p) = −p4 and thus the values for the two trajec-tories corresponding to ±p are equal. Furthermore, by (30) the curves

Dcl− ={

(t, p) ∈ D : t = t2(p),− 1√2

< p < 0}

and

Dcl+ ={

(t, p) ∈ D : t = t2(p), 0 < p < 1√2

}

both are mapped diffeomorphically onto the half-line

Ĝ = {(t, 0) ∈ G1 : t < T − 1}.Thus, given an initial condition (t, 0) ∈ Ĝ, the two trajectories which are determinedby the pre-images (t,±p) in Dcl− and Dcl+ have the same value for the objective.Define the value V E = V of the parameterized family E of extremals as V E = C◦z−,where z− now denotes the multivalued map which assigns to a point (t, x) allpossible parameter values (t, p) that satisfy z(t, p) = (t, x). Thus the map z− issingle-valued on GT ∪G0 ∪Gc, has two values on G− and G+ and three values onG1. If we define πi, i = −, 0, +, as the inverse parameter maps for the restrictionsof z to Di1, then the associated value V E has three sections over the set G1 whichwe denote by

Vi : G1 → R, Vi(t, x) = C(t, πi(t, x)).We have shown that

V+(t, 0) ≡ V−(t, 0) for (t, 0) ∈ Ĝ (32)and V0 will never be minimal. In fact,

V0(t, x) > V±(t, x) for all (t, x) ∈ G1.Analyzing the map z further, it can be seen that z maps each of the regions

D+opt ={(t, p) ∈ D+1 : t2(p) < t < t1.5(p)

}

andD−not = {(t, p) ∈ D−2 : t6(p) < t < t2(p)}

diffeomorphically onto Ĝ+ = G1 ∩ {(t, x) ∈ G : x > 0}. Analogously, the regionsD−opt =

{(t, p) ∈ D−1 : t2(p) < t < t1.5(p)

}

andD+not = {(t, p) ∈ D+1 : t6(p) < t < t2(p)}


are mapped diffeomorphically onto Ĝ− = G1 ∩{(t, x) ∈ G : x < 0}. Computing thevalues one obtains that

V+(t, x) < V−(t, x) for all (t, x) ∈ Ĝ+and

V+(t, x) > V−(t, x) for all (t, x) ∈ Ĝ−This gives the following result:

Theorem 5.1. A parametrization of the globally optimal trajectories for the optimalcontrol problem [Simple Cusp] is obtained if the domain is restricted to

Dopt = {(t, p) ∈ D : t2(p) ≤ t ≤ T}.The curve Φ2 gives a parametrization of the cut-locus Γ of the two branches

corresponding to the parameterizations over D+1 and D−1 , respectively, and for initial

points on Γ there exist two optimal trajectories in the family. In the interior of Doptthe parametrization is an analytic diffeomorphism.

0.6 0.8 1 1.2 1.4 1.6 1.8 2−0.05

0

0.05

0.1

0.15

0.2

time t

sta

te x

x(t,p), p>0

GC

=(T−1,0) 0.6 0.8 1 1.2 1.4 1.6 1.8 2

−0.2

−0.15

−0.1

−0.05

0

0.05

time t

sta

te x

x(t,p), p 0 (left) and p < 0 (right); the bottomrow gives a combination of these two flows (left) and the resultingoptimal synthesis of controlled trajectories (right).

Figure 5 illustrates the flow of the parameterized family of extremals: in the toprow it shows the two fields of locally optimal trajectories defined for p > 0 and


p < 0. While the controlled trajectories (x(·, p), u(·, p)) for p 6= 0 are strong localminima over the interval [t0, T ] if t0 > σ(p) = t6(p), they are globally only optimalif t0 ≥ t2(p) > t6(p). The simple cusp point at (T − 1, 0) generates a cut-locus(intersection) between the two branches of the parameterized family for p > 0 andp < 0 that limits the global optimality of trajectories at time t2(p) and thus thecontrolled trajectories for p 6= 0 already lose global optimality prior to the conjugatepoint. The optimal synthesis is shown in the bottom right of Figure 5.

Figure 6 shows various time-slices of the associated multivalued cost functionV E = C ◦z−1 in the (t, x)-space that illustrate the changes in the cost as the simplecusp point is passed. Interestingly, taken together, the graph of this multivaluedfunction is qualitatively identical with the singular set of the swallow tail singularity,the next singularity in the order of cusps. The same observation holds for the foldsingularities in which case the corresponding multi-valued value took the form ofthe singular set of a simple cusp. These relations also are a consequence of thefundamental relation for optimality expressed in the shadow price lemma.

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

V = V(0.4,⋅)

state x

va

lue

V

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

V = V(0.6,⋅)

state x

va

lue

V

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

state x

va

lue

V

V = V(0.8,⋅)

−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

V = V(1,⋅) va

lue

V

state x

Figure 6. Slices of the multivalued function V E correspondingto the parameterized family E of extremals for t = const for theproblem [Simple Cusp].

Note that we have σ(0) = t2(0) = T − 1 for the extremal (x(·, 0), u(·, 0)) thatpasses through the simple cusp point and that this controlled trajectory remainsglobally optimal over the compact interval [T − 1, T ]. Thus this is an examplewhere the trajectory only loses optimality after the conjugate point has been crossed


(we are thinking of integrating the field backwards). On the contrary, for foldsingularities, trajectories lose optimality at the conjugate point, i.e., are no longeroptimal on the intervals [tcp(p), T ] if tcp(p) denotes the corresponding conjugatetime.

We close this section with the remark that these geometric properties are valid ingeneral near simple cusp points. This can be shown by transforming the system intonormal form and then analyzing the resulting system. The computations, however,are quite technical and we refer the reader to [9].

6. Conclusion. We have illustrated the geometric structure of the flow z of aparameterized family of extremals near fold and simple cusp singularities and itsimplications on the optimality of the trajectories in the family. Fold points generatethe typical behavior of trajectories that are strong local minima and lose optimalityat the fold point. However, the corresponding behavior of optimal trajectories israrely, if ever seen in a regular synthesis of optimal trajectories. This is explainedby the behavior of extremals near a simple cusp point. The simple cusp point gen-erates a region in the state space which is covered 3 : 1. In this region there existtwo locally minimizing and one locally maximizing branch. The changes from thelocally minimizing to the maximizing branch occur at the fold-loci and there trajec-tories lose strong local optimality. However, the two minimizing branches intersectand generate a cut-locus that limits the optimality of the close-by trajectories andindeed eliminates these trajectories from optimality near the cusp point prior to theconjugate point. In the language of PDE, the simple cusp point generates a shockin the solution to the Hamilton–Jacobi–Bellman equation and thus has significantimplications on the structure of globally optimal controlled trajectories.

Acknowledgements. This material is based upon research supported by the Na-tional Science Foundation (NSF) under collaborative research grants DMS 1008209/1008221. U. Ledzewicz’s research also was partially supported by an SIUE “STEP”grant in Summer 2011. Material from this paper will also be used in the forthcomingtext “Geometric Optimal Control—Theory, Methods and Examples” by the authorsto be published by Springer-Verlag.

REFERENCES

[1] L. Berkovitz, “Optimal Control Theory”, Springer-Verlag, New York, 1974.[2] G. Bliss, “Calculus of Variations”, The Mathematical Association of America, 1925.[3] A. Bressan and B. Piccoli, “Introduction to the Mathematical Theory of Control”, American

Institute of Mathematical Sciences (AIMS), 2007.[4] A.E. Bryson, Jr. and Y.C. Ho, “Applied Optimal Control”, Revised Printing, Hemisphere

Publishing Company, New York, 1975.[5] C.I. Byrnes and H. Frankowska, Unicité des solutions optimales et absence de chocs pour

les équations d’Hamilton–Jacobi–Bellman et de Riccati, C.R. Acad. Sci. Paris, Série I, 315,(1992), 427–431.

[6] C.I. Byrnes and A. Jhemi, Shock waves for Riccati partial differential equations arising innonlinear optimal control, in “Systems, Models and Feedback: Theory and Applications”,(eds. A. Isidori and T.J. Tarn), Birkhäuser, (1992), 211–225.

[7] M. Golubitsky and V. Guillemin, “Stable Mappings and their Singularities”, Springer-Verlag,New York, 1973.

[8] M. Golubitsky and D.G. Schaefer, “Singularities and Groups in Bifurcation Theory, Vol. I”,Applied Mathematical Sciences, vol. 51, Springer-Verlag, New York, 1985.


[9] M. Kiefer and H. Schättler, Parametrized families of extremals and singularities in solutionsto the Hamilton–Jacobi–Bellman equation, SIAM J. on Control and Optimization, 37, (1999),1346–1371.

[10] J. Noble and H. Schättler, Sufficient conditions for relative minima of broken extremals, J.of Mathematical Analysis and Applications, 269, (2002), 98–128.

[11] U. Ledzewicz, A. Nowakowski and H. Schättler, Stratifiable families of extremals and suffi-cient conditions for optimality in optimal control problems, J. of Optimization Theory andApplications (JOTA), 122, (2004), 105–130.

[12] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F. Mishchenko, “The Mathemat-ical Theory of Optimal Processes”, MacMillan, New York, 1964.

[13] H. Schättler and U. Ledzewicz, Synthesis of optimal controlled trajectories with chatteringarcs, Dynamics of Continuous, Discrete and Impulsive Systems Series B: Applications &Algorithms, 19, (2012) 161–186.

[14] H. Schättler and U. Ledzewicz, Perturbation Feedback Control—A Geometric Interpretation,Numerical Algebra, Control and Applications, (2012), to appear.

[15] H. Schättler and U. Ledzewicz, “Geometric Optimal Control—Theory, Methods and Exam-ples”, Springer-Verlag, New York, 2012.

[16] H. Whitney, Elementary structure of real algebraic varieties, Ann. Math., 66, (1957), 545-556.[17] L.C. Young, “Lectures on the Calculus of Variations and Optimal Control Theory”, W.B.

Saunders, Philadelphia, 1969.

E-mail address: [email protected]; [email protected]

heinz sch˜attler urszula ledzewicz - siueuledzew/papers/avner.pdf · 2012. 9. 25. · this paper...

Documents