1488 AIAA JOURNAL VOL. 6, NO. 8

Optimal Programming Problems with a Bounded State SpaceJASON L. SPEYER* AND ARTHUR E. BRYSON jR.f

Harvard University, Cambridge, Mass.

A new set of necessary conditions are presented for solutions to optimal programming prob-lems with a state variable inequality constraint (SVIC). On a constrained arc, the dimensionof the state space is reduced, and the influence functions associated with this reduced statespace are shown to be unique and continuous across junctures with unconstrained arcs. It isalso shown that unconstrained arcs must satisfy certain "tangency" constraints at both endsof a constrained arc and that explicit use of this fact must be made if the SVIC is adjoineddirectly to the performance index.

1. Introduction

^EVERAL investigators have discussed necessary condi-^ tions for solutions to optimal programming problems witha state variable inequality constraint.1"5 Most of these in-vestigators recognized that, at a point where the systementers onto a constrained arc (an "entry point"), all feasibleunconstrained arcs passing through this point must satisfycertain tangency constraints, namely, these arcs must havezero values of the state variable constraint function and all ofits time derivatives that do not involve the control variable(say p-1 of them). Since control of the constraint functionis realized only by changing its pth time derivative, no finitecontrol will keep the system on the constraint boundary if thepath reaching the constraint boundary does not meet thesetangency constraints.

In this paper, we point out that the tangency constraintsalso apply to the unconstrained arc at a point where the sys-tem leaves a constrained arc (an "exit point"). These exit-point tangency constraints are satisfied automatically if thenecessary conditions of Ref. 3 are used. However, if oneuses the "direct adjoining" approach of Ref. 4, we shall showthat explicit use must be made of the tangency constraints atboth ends.

2. Reduced Dimension of the State Spaceon a Constrained Arc

Dreyfus7 made use of the fact that the dimension of thestate space is reduced on a constrained arc. If the SVIC is

s(x,t) < 0 s, a scalar function (1)

and the system equations are

x = f(x,u,t)

x} an n-vector; (2)

f, an ^-vector function; u, a scalar

then, as in Ref. 3, we shall call (1) a pth-order SVIC if p timesderivatives of s(x,t)} substituting f(x,u,t) for x from (2), arerequired before u appears explicitly in the result.

Let y(x,t) be defined as the p vector

where sw = drsdir (3)

Presented at the AIAA 1968 Joint Automatic Control Con-ference, Ann Arbor, Mich., June 26-28, 1968 (no paper number;published in bound volume of conference papers); submittedNovember 28, 1967; revision received April 3, 1968.

* Graduate Student, sponsored by the Raytheon FellowshipProgram, Division of Engineering and Applied Physics.

t Professor of Mechanical Engineering, Division of Engineeringand Applied Physics. Associate Fellow AIAA.

and on a constrained arc let c(x,u}t) be defined as the scalar

c(x,u,t) = 8&(x,u,t) = 0 (4)

Now y must be identically zero on a constrained arc, so thedimension of the state space is reduced to n-p. Let us choosen-p functions

~zi(x,t)

(5)

so that the p-vector y and the n-p vector z together span then-dimensional x-space. In particular, the n by n Jacobianmatrix

must be nonsingular in the bounded state space.The implicit relation (4) defines u as a function of x and t.

On a constrained arc (y = 0), it defines u as a function of zand t. Thus on a constrained arc it is possible to express thesystem equations in the form

z = g(z,t) (6)

3. Problem Statement

Find u(t) mt0 < t < tf to minimize

J = 4>[x(tf),tf} + f" L[x(t),u(t),t]dt (7)J to

subject to the constraints

x = f[x(t),u(t\t] x(k) and fo specified (8)

*[x(tf),tf] = 0 (9)

8[x(t),t] < 0 (10)

where <£ and s are known scalar functions, x is an n vector and\fr" is a q vector (q < n + 1).

4. Necessary Conditions on Unconstrained Arcs

Assume a constrained arc for t\ < t < tz and unconstrainedarcs for fo < t < t\ and ti<t<tf. Then on the unconstrainedarcs, solutions must satisfy Eqs. (8) and

0 =where H = \Tf + L

(H)(12)

The influence function or Lagrange multipler X is an n vectorand H is the variational Hamiltonian. At the terminal point,

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.4

793

AUGUST 1968 PROGRAMMING PROBLEMS WITH A BOUNDED STATE SPACE 1489

the solution must satisfy (9) and

H(tf) = -

where <£ =

(13)

(14)where the Lagrange multiglier v is a q vector of constants.At the initial point, the solution must satisfy the boundaryconditions of (8).

5. Necessary Conditions on a Constrained Arc

On a constrained arc,y(x,t) = 0 (15)

where the p-vector y(x,t) was defined in (3), z — g(z,t)f andwhere the n-p vector z was defined in Eqs. (4-6). In addi-tion, a reduced set of influence functions /z, of dimension n-p;is defined by

where G = ̂ g(z^ + £(^) (16)Oz

where L(z,t) = L(x)u)t)1 u is eliminated through (4), and x isexpressed in terms of z with y = 0 through (3) and (5).

6. Necessary Conditions at a Juncture betweena Constrained and an Unconstrained Arc

At a juncture between a constrained and an unconstrainedarc, it is necessary that the unconstrained arc satisfy thetangency constraints (see Sec. 1):

•[x(td,ti] = 0 i = 1,2 (17)The quantities /j(£») may be expressed in terms of X(£») as fol-lows. We may write the augmented performance index as

(H - \Tx)dt +

P (G - ^z)dt + ftf (H - \T±)dt (18)•7 £i + J h^

where the values of the variables just before and just after thejuncture are indicated by ti~ and ti+, respectively. From(18), it follows thatdJ = -\T 5x\t^- -VT 54+fa" + (H - \TX)\tl- dk ~

(G - v?z)\h*Mi +However

(19)

dz

f y* ~ ~ yt ~]= ... dx + ... \ d t

L z* _ _ ' z* Jor

wheredx = M(dy - ytdt) + N(dz - ztdt)

[M\N] =

and where ( )x and ( )z means b( )/d# and d(the juncture t = fr, we have

dyfa) = 0

) + £(*i~)<ftiSubstituting (22-26) into (20), we have

dJ = [p? - \TN]t=tidz(ti) + [^T(Myt + Nzt) +H - G]i=hdtl+ ..

(20)

(21)

(22)

Now, at

(23)(24)(25)

(26)

For a stationary solution, the coefficient of dz(ti) and dtmust vanish,

^T = \TN at t = t (27)0 = H + \T(Myt + Nzt) aU = *i (28)

For the unconstrained arc in to < t < ti, the n specifiedinital state variables of (8), the p conditions (17), the n — pconditions (27), and the single condition (28) provide 2n + 1conditions for the 2nth-order differential equation system ofEqs. (8) and (11) with the unspecified time t\.

Similarly, for the unconstrained arc in Z2 < t < £/, the pconditions (17) and the n — p + 1 conditions (27) and (28)applied at t = t% and the q + n + 1 conditions (9, 13, and 14)provide 2n + 2 + q conditions for the 2nth-order system (8)and (11) with the unspecified times t% and tf and the q con-stants v.

If y and z were used as the state variables on the uncon-strained arcs as well as on the constrained arc, the multipliersassociated with z are seen from (28) to be continuous acrossthe juncture points. The example to follow illustrates thispoint.

In general, these two-point boundary-value problems forthe two unconstrained arcs cannot be solved separately sincethey are coupled through z(t) and id(t). However, there arecases in which they can be solved separately (see Ref. 8).

7. Comparison with Previous Work

In Refs. 3 and 7, the SVIC was regarded as an entry-pointtangency constraint [Eq. (18) with t = ti]y followed by a con-trol-and-state variable constraint [Eq. (4)]. Any solutionsatisfying these two constraints automatically satisfies theexit-point tangency constraint. The transversality condi-tion (28) at the exit point was stated in Refs. 3 and 6 in a waysuch that continuity of u across the exit point is assumed.Although this will often be the case, it is not necessary. Usewas not made of the reduced state space on constrained arcsin Refs. 3 and 6; as a result, the influence functions X»(0,i = 1, . . . n, are not uniquely determined on the constrainedarc. However, it is clear from the present work that n-plinear combinations of the X/s are uniquely determined [seeEq. (27)] by the conditions stated in Ref. 3 once a suitablechoice of n-p state variables is made as in (5).

In Ref. 4, the necessity of tangency constraints on the un-constrained arcs was not recognized. Hence, there were toofew necessary conditions stated to determine an extremalsolution satisfying the constraints. It is shown in the Appen-dix that explicit use of the tangency constraints is required atboth the entry and the exit points when the inequality con-straint (1) is adjoined directly to the performance index as wasthe case in Ref. 4. When this is done, the influence functions\i(t), i = 1, .. ., n are uniquely determined on a constrainedarc and are either continuous or have unique discontinuitiesat the entry and exit points. In the Appendix, the necessaryconditions at the entry and exit points are similar, just asthey are in the previous section [see Eqs. (28) and (29)].

8. Analytic Example

The preceding discussion is illustrated here by a problemwhose three-dimensional state space is bounded by a second-order SVIC. The optimal unconstrained arcs for this prob-lem cannot be calculated separately and then joined togetheras in Ref. 8. The problem is to find the control program u,over the interval te [0, 3 ] which minimizes

1 ?dt

subject to the constraintsh(G) = 1.6, h(S) = 0v(0) = -1,0(3) = -1

(29)

(30)(31)

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.4

793

1490 J. L. SPEYER AND A. E. BRYSON JR. AIAA JOURNAL

Fig. 1 Time history of the state variable.

a(0) = 0, a(3) = 0 (32)

a = u . (33)with a SVIC given as

s = v - vc(£) > 0 (34)where vc(t) is given as

vc = -W + I2t - 8 (35)The variables h, v, a, and t can be thought of as position,velocity, acceleration, and time, respectively. These statevariables on the constrained arc are subscripted by c.

Consider the new state space as denned in Sec. 2,

y\v - vc(tr\_a — oc(0_

(36)

2 = h (37)c = u + 8 (38)

On the constraint boundary, the system equation is onlyh = ve(t) (39)

The variational Hamiltonian is defined on the unconstrainedarcs as

H = \hv + \va + \au + u2/2 (40)The corresponding Euler-Langrange necessary conditions are

\H = 0 (41)

X. - -X» (42)

Xa = -X, (43)

On the constrained arc, the Hamiltonian is

G = fja)e(t) + 32 (44)

with the associated multiplier equation

A = 0 (45)

At the junctures to the constrained arc, Eqs. (28) and (29)must be satisfied where

M =

then

o on rn r-«cn1 0 N = 0 zt = 0,yt = (46)

Lo ij LoJ L-W.J

H KHJNOARY r ———

1 21 I

3

Xv(t)

HCONSTRAINT I ____BOUNDARY P

(47)

Fig. 2 Time history of the influence functions and controlvariable.

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.4

793

AUGUST 1968 PROGRAMMING PROBLEMS WITH A BOUNDED STATE SPACE 1491

\hVe(t) = \hV + \v(a - ae) + Xa(^ - uc) (48)

At the junctures, tangency requires v = vc and a = ac, soEq. (48) implies that at the juncture

(49)

In general, it can be shown that if the Hamiltonian is regularthe control will be continuous across a juncture point.9

The time histories of the state variables and influence func-tions which satisfy the previous necessary conditions aregiven in Figs. 1 and 2, respectively. The dashed paths areoptimal solutions without the inclusion of the SVIC. Thesolid paths are optimal solutions that include the SVIC of(34).

The constrained arc lies between the juncture times 1 and2. The reduced state space is represented only by A, and thecorresponding multiplier is X&. The values of \v and Xa onthe constraint boundary are of no consequence. The opti-mal unconstrained arcs cannot be calculated separately be-cause of the need to satisfy the initial and terminal positionconstraints.

Appendix: Handling State Variable InequalityConstraints by Direct Adjoining

In this scheme the SVIC (10) is directly adjoined to theperformance index (7) by a Lag-range multiplier 77 in themanner of Chang.4 However, it is also necessary to adjointo J the tangency conditions at the juncture points to theconstraint boundary. The variational Hamiltonian is nowdefined as

H = L •+ \Tf +(17 = 0, s < 0

( n y£ o, s - oThe Euler-Lagrange equations are then

dw-

dw dw

(Al)

(A2)

(A3)

On constrained arcs, we may reduce the dimension of thestate vector to n — p by using the p equalities

s(x,t) = ,t) = ... = s("-»(x,t) = 0 (A4)

In a similar manner, we may reduce the dimension of the Xvector to n — p by using the p equalities

Hu(x,\,t) = Hu^(x,\t) = ... = Hj*-»(x,\t) = 0 (A5)

In (A 5) , u was expressed in terms of x and t by

s<V(x,u,t) = 0 (A6)

and 77 is determined in terms of x, X, and t by

HvW(x,\,t,ri) = 0 (A7)

The X vector is, in general, discontinuous at both the entryand exit points of the constraint boundary, but these dis-continuities are unique. These discontinuities can be ex-pressed in terms of x and X on the unconstrained side of thejuncture to the constraint boundary. Let t* be the time of ajuncture between constrained and unconstrained arcs (eitheran entry or an exit point). Then by adjoining the tangencyconditions at the juncture points, the necessary conditionscorresponding to (28) and (29) are

u (A8)

(A9)

where ( )c denotes ( ) on the constrained side of the juncture,and no subscript denotes ( ) on the unconstrained side of thejuncture, vi is a p vector of multipliers associated with thetangency conditions (18). The p-equations (A5), the (n + 1)equations (A8), and (A9) determine the p-quantities vf,the n-quantities Xc(^-), and the time ti in terms of X(£») andthe state (since tyfox is a function of the state) on the un-constrained side of the juncture. It is then not surprisingthat if (A8) is substituted into (28), JJL is found as a linearcombination of Xc independent of the ^/s. Thus in thelower-dimensional state space, the necessary conditions ob-tained here reduce to those of the main text.

References1 Gamkrelidze, R. V., "Optimal Processes with Restricted

Phase Coordinates," Izvestia Akademiya Nauk SSSR, Ser. Mat.24, 1960, pp. 315-356; also Pontryagin, L. S., Boltyanskii, V. G.,Gamkrelidze, R. V., and Mischenko, E. F., The MathematicalTheory of Optimal Processes, Interscience Publishers, New York,1962, English transl., Chap. VI.

2 Berkovitz, L. D., "On Control Problems with Bounded StateVariables," Journal of Mathematics and Analytical Applications,Vol. 5, 1962, pp. 488-501.

3 Bryson, A. E., Jr., Denham, W. F., and Dreyfus, S. E., "Opti-mal Programming Problems with Inequality Constraints I:Necessary Conditions for Extremal Solutions," AIAA Journal,Vol. 1, No. 11, Nov. 1963, pp. 2544-2551.

4 Chang, S. S. L., "Optimal Control in Bounded Phase Space,"Automatica, Vol. I, 1963, pp. 55-67; also TR 400-37, Aug. 1961,Dept. of Electrical Engineering, New York Univ.

5 Dreyfus, S. E., Dynamic Programming and the Calculus ofVariations, Academic, New York and London, 1965.

6 Denham, W. F. and Bryson, A. E., Jr., "Optimal Program-ming Problems with Inequality Constraints II: Solution bySteepest-Ascent," AIAA Journal, Vol. 2, No. 1, Jan. 1964, pp.25-34.

7 Dreyfus, S. E., "Variational Problems with State VariableInequality Constraints," Ph.D. dissertation, 1962, HarvardUniv.; also Paper p-2605, July 1962, Rand Corp.

8 Speyer, J. L., Mehra, R. K., and Bryson, A. E., Jr., "TheSeparate Computation of Arcs for Optimal Flight Paths withState Variable Inequality Constraints," Proceedings of the Col-loquium on Advanced Problems and Methods for Space FlightOptimization, Liege, Belgium (held June 19-23, 1967), PergamonPress, New York.

9 Paiewon&ky, B. et al., "On Optimal Control with BoundedState Variables" Rept. 60, July 1964, Aeronautical ResearchAssociates of Princeton.

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.4

793