introduction to control theory including optimal control

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.3 _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

___________________________________________________________________________________________________________

Chapter 1 System Dynamics and Differential Equation

1

1. System Dynamics and Differential Equations 1.1 Introduction - Investigation of dynamics behavior of idealized components, and the collection of interacting components or systems are equivalent to studying differential equations. - Dynamics behavior of systems ⇔ differential equations. 1.2 Some System Equations Inductance circuit: In electromagnetic theory it is known that, for a coil, in Fig.1.1, having an inductance L , the electromotive force (e.m.f.) E is proportional to the rate of change of the current I , at the instant considered, that is,

dtdILE = or ∫= dtE

LI 1 (1.1)

L

E Fig. 1.1

Capacitance circuit: Similarly, from electrostatics theory we know, in Fig. 1.2, that the voltageV and current I through a capacitor of capacitance C are related at the instant t by

∫= dtIC

E 1 or dtdECI = (1.2)

C

E Fig. 1.2

Dashpot device: Fig. 1.3 illustrates a dashpot device which consists of a piston sliding in an oil filled cylinder. The motion of the piston relate to the cylinder is resisted by the oil, and this viscous drag can be assumed to be proportional to the velocity of the piston. If the applied force is )(tf and the corresponding displacement is )(ty , then the Newton’s Law of motion is

dtdyf µ= (1.3)

where the mass of the piston is considered negligible, and µ is the viscous damping coefficient.

)(tf

)(ty

Fig.1.3

Liquid level: To analyze the control of the level of a liquid in a tank we must consider the input and output regulations. Fig.

1.4 shows a tank with: inflow rate iq , outflow rate oq , head level h , and the cross-section are of the tank is A .

inflow

h (head) discharge q0

valve Fig. 1.4

If io qqq −= is the net rate of inflow into a tank, over a period tδ , and hδ is the corresponding change in the head level, then hAtq δδ = and in the limit as 0→tδ

dtdhAq = or ∫= dtq

Ah 1 (1.4)

All these equations (1.1)-(1.4) have something in common: they can all be written in the form

xdtdya = (1.5)

Equation (1.5) is interesting because:

- its solution is also the solution to any one of the systems considered. - it shows the direct analogies which can be formulate between quite different types of components and systems. - it has very important implications in mathematical modeling because solving a differential equation leads to the solution of a vast number of problems in different disciplines, all of which are modeled by the same equation.

Over small ranges of the variables involved the loss of accuracy may be very small and the simplification of calculations may be great. It is known that the flow rate through a restriction such as a discharge valve is of the form

PVq = where P is the pressure across the valve andV is coefficient dependent on the properties of the liquid and the geometry of the valve. Fig.1.5 shows the relation between the pressure and the flow rate.

flow rate

assumedlinear

q

1q

1PP (pressure)

Fig. 1.5


___________________________________________________________________________________________________________


2

Assume that in the neighborhood of the pressure 1PP =

'Rrateflowinchange

pressureinchange≈

where 'R is a constant called the resistance of the valve at the point considered. This type of assumption, which assumes that an inherently nonlinear situation can be approximated by a linear one albeit in a restricted range of the variable, is fundamental to many applications of control theory. At a given point, the pressure ghP ρ= , define )/(' gRR ρ= , we can rewrite the above relation as

oqhR /= (1.6) A two tank systems with one inflow )(q and two discharge valves is shown in Fig. 1.6.

inflow

1h2h1R 2R

q

1A 2A

1 2

Fig. 1.6

For tank 1

)(121

1

11 hh

Rq

dtdh

A −−=

⇒ 211

1111

11

111 hRA

hRA

qAdt

dhh +−==& (1.7)

For tank 2

22

211

22

1)(1 hR

hhRdt

dhA −−=

⇒ 2212

112

22

1111 hRRA

hRAdt

dhh ⎟⎟

⎠

⎞⎜⎜⎝

⎛+−==& (1.8)

We can write (1.7) and (1.8) in matrix form as follows

qAhh

RRARA

RARAhh

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡+⎥⎦

⎤⎢⎣⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎟⎟⎠

⎞⎜⎜⎝

⎛+−

−

=⎥⎦

⎤⎢⎣

⎡

0

1

1111

11

121

21212

1111

2

1&

&

that is, in the form

qBhAh +=& (1.9) where

⎥⎦

⎤⎢⎣

⎡=

2

1hhh &

&,

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎟⎟⎠

⎞⎜⎜⎝

⎛+−

−

=

21212

11111111

11

RRARA

RARAA ,

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡=

0

1

1AB

We can also write the set of simultaneous first order equation (1.7) and (1.8) as one differential equation of the second order. On differentiating (1.7), we obtain

2212

112

21111 h

RRAh

RAh &&&&

⎟⎟⎠

⎞⎜⎜⎝

⎛+−= (1.10)

From (1.7) and (1.8),

221

21

2

1

211

2211

21

2

1

211

1111

1

11

11111

111

hRA

hAA

qA

hRA

hRRA

hAA

qA

hRA

hRA

qA

h

−−=

+⎟⎟⎠

⎞⎜⎜⎝

⎛+−−=

+−=

&

&

&

Then

22121

211212121

2212

22121

211121

2

111111

111111

hRRAA

hRARRA

qRAA

hRRA

hRRAA

hRA

qRAA

h

−⎥⎥⎦

⎤

⎢⎢⎣

⎡+⎟⎟

⎠

⎞⎜⎜⎝

⎛+−=

⎟⎟⎠

⎞⎜⎜⎝

⎛+−−−=

&

&&&&

or

qRAA

hRRAA

hRARRA

h121

22121

211212

2111111

=+⎥⎥⎦

⎤

⎢⎢⎣

⎡+⎟⎟

⎠

⎞⎜⎜⎝

⎛++ &&&

(1.11) Equation (1.11) is a differential equation of order 2 of the form xbyayay 021 =++ &&& . 1.3 System Control Consider a control system: a tank with an inflow iq and a discharge oq in Fig. 1.7

R

float

inflow outflow

lever

regulator valve Fig. 1.7

The purpose of controller is to maintain the desired level h . The control system works very simply. Suppose for some reason, the level of the liquid in the tank rises above h . The float senses this rise, and communicates it via the lever to the regulating valve which reduces or stops the inflow rate. This situation will continue until the level of the liquid again stabilizes at its steady state h . The above illustration is a simple example of a feedback control system. We can illustrate the situation schematically by use of a block diagram as in Fig. 1.8.

plantregulator1h ε x 2h

2h

Fig. 1.8


___________________________________________________________________________________________________________


3

with

1h : desired value of the head

2h : actual value of head

21 hh −=ε : error The regulator receives the signal ε which transforms into a movement x of the lever which in turn influences the position of the regulator valve. To obtain the dynamics characteristics of this system we consider a deviation ε from the desired level 1h over a short time interval dt . If iq and oq are the change in the inflow and outflow, from (1.4),

oi qqdtdA −=ε

where (1.6), Rqo /ε= . So that the equation can be written as

iqRdtdRA =+εε

the change in the inflow, depends on the characteristics of the regulator valve and may be of the form

εKqi = , K is a constant Another type of control system is known as an open loop control system, in which the output of the system is not involved in its control. Basically the control on the plant is exercised by a controller as shown in Fig. 1.9.

plantcontrolleroutputinput

Fig. 1.9

1.4 Mathematical Models and Differential Equations Many dynamic systems are characterized by differential equations. The process involved, that is, the use of physical laws together with various assumptions of linearity, etc., is known as mathematical modeling. Linear differential equation

uyyy =−+ 32 &&& → time-invariant (autonomous) system uyyty =+− &&& 2 → time-varying (non-autonomous) system

Nonlinear differential equation

uyyy =−+ 22 2&&&

uyyyy =+− &&& 2 1.5 The Classical and Modern Control Theory Classical control theory is based on Laplace transforms and applies to linear autonomous systems with SISO. A function called transfer function relating the input-output relationship of the system is defined. One of the objectives of control

theory is to design – in terms of the transfer function – a system which satisfies certain assigned specifications. This objective is primarily achieved by a trial and error approach. Modern control theory is not only applicable to linear autonomous systems but also to time-varying systems and it is useful when dealing with nonlinear systems. In particularly it is applicable to MIMO systems – in contrast to classical control theory. The approach is based on the concept of state. It is the consequence of an important characteristic of a dynamical system, namely that its instantaneous behavior is dependent on its past history, so that the behavior of the system at time 0tt > can be determined given

(1) the forcing function (that is, the input), and (2) the state of the system at 0tt =

In contrast to the trial and error approach of classical control, it is often possible in modern control theory to make use of optimal methods.


___________________________________________________________________________________________________________

Chapter 2 Transfer Functions and Block Diagrams

4

2. Transfer Functions and Block Diagrams 2.1 Introduction - Review of Laplace transform - Using Laplace transform to solve a differential equation 2.2 Review of Laplace Transforms Definition: The Laplace transform of )(tf , a sectionally continuous function of time, denoted by )]([ tfL , is defined as

)()()]([0

sFdtetftf st == ∫∞

−L (2.1)

where ωσ is += , σ and ω are real variables. )(tf is called

inverse transform of )(sF , such as )()]([1 tfsF =−L .

Example 2.1 _______________________________________

Consider the step functions⎩⎨⎧

≥<

=000

)(tforctfor

tf , c is a

constant as shown in Fig. 2.1. Find the Laplace transform

c)(tf

t0 Fig. 2.1

sce

sce

scdtecsF st

tstst +−=⎥⎦⎤

⎢⎣⎡−== −

→∞

∞−

∞−∫ lim1)(

00

So long as σ (=real part of s ) is positive, 0lim =−

→∞

stt

e , hence

scsF =)( (so long as 0>σ )

__________________________________________________________________________________________ Some properties of Laplace transforms A is a constant and )()]([ sFtf =L

(1) )()]([)]([ sFAtfAtfA == LL (2) )()()]([)]([)]()([ 212121 sFsFsfsftftf +=+=+ LLL (3) Differential

)0()()( fssFtfdtd

−=⎥⎦⎤

⎢⎣⎡L (2.2)

Proof

vduudvuvd +=)(

⇒ ∫∫∫∞∞∞

+=000

)( vduudvuvd

⇒ ∫∫∞

∞∞

−=0

00

][ vduuvudv

Consider sts

st evtfddu

dtedvtfu

−− −==

⇒==

1)]([)(

then

⎥⎦⎤

⎢⎣⎡+=

+⎥⎦⎤

⎢⎣⎡−=

⎟⎠⎞

⎜⎝⎛−−⎥

⎦

⎤⎢⎣

⎡⎟⎠⎞

⎜⎝⎛−=

=

∫

∫

∫

∞−

∞−

∞−

∞−

∞−

)(1)0(

)]([11)(

)]([11)(

)()(

00

00

0

tfdtd

ssf

dtetfdtd

se

stf

tfdes

es

tf

dtetfsF

stst

stst

st

L

Hence )0()()( fssFtfdtd

−=⎥⎦⎤

⎢⎣⎡L

(4) Integration

ssFdxxf

t )()(0

=⎥⎦

⎤⎢⎣

⎡∫L (2.5)

Proof

ssF

dtetfs

dtdxxfdtde

s

dxxfdes

es

dxxf

dtedxxfdxxf

st

tst

tst

stt

sttt

)(

)(1

)(1

)(1

1)(

)()(

0

0 0

0 0

00

0 00

=

=

⎥⎦

⎤⎢⎣

⎡⎟⎠⎞

⎜⎝⎛−−=

⎥⎦

⎤⎢⎣

⎡⎟⎠⎞

⎜⎝⎛−−

⎥⎥⎦

⎤

⎢⎢⎣

⎡⎟⎠⎞

⎜⎝⎛−⎟⎟⎠

⎞⎜⎜⎝

⎛=

⎟⎟⎠

⎞⎜⎜⎝

⎛=⎥

⎦

⎤⎢⎣

⎡

∫

∫ ∫

∫ ∫

∫

∫ ∫∫

∞−

∞−

∞−

∞−

∞−L

(5) The delay function

)()]([ sFetf sαα −=−L (2.6) Proof Given the Laplace transform )(sF of a function )(tf such that

0)( =tf for 0<t . We wish to find the Laplace transform of )( α−tf where 0)( =−αtf for α<t .(Fig. 2.2)

)(tf

t0

)( α−tf

t0 α Fig. 2.2


___________________________________________________________________________________________________________


5

Consider the function

∫∞

−=0

)()( dxexfsF sx .

Let α−= tx , then

)]([

)(

)()(

0

0

)(

α

α

α

α

α

α

−=

−=

−=

∫∫

∞−

∞−−

tfe

dtetfe

dtetfsF

s

sts

ts

L

hence )()]([ sFetf sαα −=−L (6) The unit impulse function Consider the function shown in Fig. 2.3

⎪⎩

⎪⎨⎧

>

≤≤=

ct

ctctf0

01)(

)(tf

t0

c1

c Fig. 2.3

We can express )(tf in terms of step functions as

)(1)(1)( ctuc

tuc

tf −−=

that is, a step function beginning at 0=t minus a step function beginning at ct = . This leads to the definition of the unit impulse function or the Dirac delta function denoted by )(tδ defined by

cctutut

c

)()(lim)(0

−−=

→δ

Using the results of Example 2.1 and (2.6)

)(

)1(lim

)1(1lim)]([

0

0

scdcd

edcd

esc

t

sc

c

scc

−

→

−

→

−=

−=δL

1lim0

==−

→ ses cs

c (2.7)

(7) Time scale change Suppose that, [ ] )()( sFtf =L , find [ ])/( αtfL , 0>α

αα

αα

α txdxexf

dtetftf

xs

ts

==

⎟⎠⎞

⎜⎝⎛=⎥

⎦

⎤⎢⎣

⎡⎟⎠⎞

⎜⎝⎛

∫

∫∞

−

∞−

,)(0

0L

)( sF αα= (2.8)

(8) Initial value theorem This theorem makes it possible to find )0(f directly from

)(sF , without actually evaluating the inverse Laplace transform. The theorem states that provided

)(lim ssFs→∞

exists, then

)(lim)0( ssFfs→∞

= (2.9)

Since )0()()]('[ fxsFtf −=L and 0)('lim0

=∫∞

−

∞→dtetf st

s,

hence )0()(lim0 fssFt

−=→∞

, and (2.9) follows.

(9) Final value theorem This theorem makes it possible to obtain )(∞f directly from

)(sF . Provided the indicated limits exist

)(lim)()(lim0

ssFftfss →∞→

=∞= (2.10)

Indeed, as above

)0()()(')]('[0

fssFdtetftf st −== ∫∞

−L

Now

)0()()()(')('lim 0000fftfdttfdtetf st

s−∞=== ∞∞∞

−

→ ∫∫

Hence

)0()(lim)0()(0

fssFffs

−=−∞→

, and (2.10) follows.

2.3 Applications to differential equations

By taking the Laplace transform of a differential equation it is transformed into an algebraic one in the variable s . This equation is rearranged so that all the terms involving the dependent variable are on one side and the solution is obtained by taking the inverse transform.

Let denote dtdyDy = , 2

22

dtydyD = , …

Example 2.6 _______________________________________

Solve the equation teyDyyD 32 2 =+− subject to the initial conditions 1)0( =y and 2)0(' =y . Taking Laplace transform

][][]2[][ 32 teyDyyD LLLL =+−

⇒ 3

1)()]0()([2)]0(')0()([ 2−

=+−−−−s

sYyssYysysYs

⇒ 3

1)()12( 2−

=−+−s

ssYss

⇒ 2

2

)1)(3(13)(

−−

+−=

sssssY

⇒ 34/1

)1(2/1

14/3)( 2 −

+−

+−

=sss

sY

Taking inverse transform


___________________________________________________________________________________________________________


6

⎥⎦⎤

⎢⎣⎡

−+

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−+⎥⎦⎤

⎢⎣⎡

−= −−−−

31

41

)1(1

21

11

43)]([ 1

2111

ssssY LLLL

⇒ ttt eteety 341

21

43)( ++=

__________________________________________________________________________________________ 2.4 Transfer functions

system)(tu )(ty

Fig. 2.6

In general form the thn order SISO system (Fig. 2.6) may have an associated differential equation

ubuDbuDbyayDayDa mmm

nnn +++=+++ −− KK 1

101

10 (2.11)

where )(tu : input )(ty : output. mn ≥ : proper system mn < : improper system

If all initial conditions are zero, taking the Laplace transform of (2.11) yields

)()()()( 110

110 sUbsbsbsYasasa m

mmn

nn +++=+++ −− KK

Definition The transfer function )(sG of a linear autonomous system is the ratio of the Laplace transform of the output to the Laplace transform of the input, with the restriction that all initial conditions are zero. From the above equation, it follows that

nnn

mmm

asasabsbsb

sUsYsG

+++

+++==

−

−

K

K1

10

110

)()()( (2.12)

The transfer function )(sG depends only on the system parameters and is independent of the input. (2.12) is also written as

)()()( sUsGsY = (2.13) Example 2.7 _______________________________________

Obtain the transfer function for a fluid tank having a cross-sectional area A with one inflow )(tq , a head )(th and with an outflow valve offering a resistance R .

Using equation (1.7): 211

1111

11

111 hRA

hRA

qAdt

dhh +−==& ,

the system can be described as follows

hR

qdtdhA 1

−= or RqhdtdhRA =+

On taking the Laplace transform with zero initial conditions

RsQsHsHsRA )()()( =+

Hence

sRAR

sQsHsG

+==

1)()()(

Note that )(sG is dependent only on the system characteristics such as A and R . __________________________________________________________________________________________

Example 2.8 _______________________________________

Find the transfer function of the system described by the equation uyDyyD =++ 232 where )(tu is the input and

)(ty is the output. Taking Laplace transform both sides with zero-initial conditions, we obtain

)()()23( 2 sUsYss =++

Hence)2)(1(

1)()(

)(++

==sssQ

sHsG

__________________________________________________________________________________________ 2.5 Block Diagrams

Block diagrams The equation )()()( sUsGsY = is represented by the Fig.2.7

G(s))(sU )(sY

Fig. 2.7

Sensing diagram Using for representation of addition and/or subtraction of signals.

)(sR

)(sC

)()()( sCsRsE −=

Fig. 2.8

Identity diagram To show the relation )()()( 21 sYsYsY ==

)(sY

)()(2 sYsY =

)()(1 sYsY =

Fig. 2.9

Reduction of block diagram

)()()()()( sAsBsG

sUsY

== (2.15)

A(s))(sU )(sX

B(s))(sY

B(s)A(s))(sU )(sY

Fig. 2.10


___________________________________________________________________________________________________________


7

Block in parallel )()()()()()( sCsBsAsRsAsC −=

⇒)()(1

)()()()(

sBsAsA

sRsCsG

+== (2.16)

)(sR )(sC

A(s)

B(s)

)(sE

)(sR )(sC

)()(1)()(

sBsAsAsG

+=

Fig. 2.11

Example 2.9 _______________________________________

Obtain the system transfer function for the positive feedback system shown in Fig. 2.12

)(sR )(sCA(s)

B(s)

)(sE

Fig. 2.12

)()()()( sCsBsRsE +=

)()()( sEsAsC = ⇒ )()()]()(1[)( sRsAsBsAsC =−

and )()(1

)()()()(

sBsAsA

sRsCsG

−==

A special case and an important case of a feedback system is the unity feedback system shown in Fig. 2.1.3

)(sR )(sCA(s)

)(sE

Fig. 2.13

The unity feedback system transfer function is

)(1)()(sA

sAsG+

= (2.17)

__________________________________________________________________________________________ Example 2.10_______________________________________

Consider a system having the transfer function2

)(−

=s

KsA .

Time response of this system to a unit impulse 1)(),()()( == sUsUsAsY

⇒ teKty 2)( = For a unity feedback closed loop system

)2()(1

)()(−+

=+

=KsK

sAsAsG

⇒ tKeKty )2()( −−= __________________________________________________________________________________________


___________________________________________________________________________________________________________

Chapter 3 State Space Formulation 8

3. State Space Formulation 3.1 Introduction (Review) 3.1.1 Eigenvalues and eigenvectors Consider a matrix A of order nn× . If there exists a vector

0x ≠ and a scalar λ such that

xx λ=A then x is called an eigenvector of A . λ is called an eigenvalue of A . The above equation can be written in the form

0x =− ][ AIλ where I is the unit matrix (of order nn× ). It is known that the above homogeneous equation has non-trivial (that is, non-zero) solution only if the matrix ][ AI −λ is singular, that is if

0)det( =− AIλ This is an equation in λ of great importance. We denoted it by )(λc , so that

0)det()( =−= AIc λλ It is called the characteristic equation of A . Written out in full, this equation has the form

0=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

−−

=

nnnn

nn

aaa

aaaaaa

c

λ

λλ

λ

LMOMM

LL

21

2222111211

)(

On expansion of determinant, )(λc is seen to be a polynomial of degree n in λ , having the form

0)())((

)(

21

11

1

=−−−=++++= −

−

n

nnnn bbbc

λλλλλλλλλλ

L

K

nλλλ ,,, 21 L , the roots of 0)( =λc , are the eigenvalues of A .

Assuming that A has distinct eigenvalue nλλλ ,,, 21 L , the corresponding eigenvectors nxxx ,,, 21 L are linearly indepen-dent. The (partitioned) matrix

[ ]nX xxx L21= is called the modal matrix of A. Since

iiA xx λ= ),,2,1( ni L= it follows that

XXA Λ=

where

},,,{

00

0000

2121

n

n

diag λλλ

λ

λλ

L

LMOMM

LL

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=Λ

is called eigenvalue matrix of A . Hence

XAX 1−=Λ 3.1.2 The Cayley-Hamilton theorem Given a square matrix A (of order nn× ) and integers r and s , then

rsrssrsr AAAAAA === ++ This property leads to the following definition. Definition Corresponding to a polynomial in a scalar variable λ

kkkk bbbf ++++= −− λλλλ 11

1)( K define the (square) matrix )(Af , called a matrix polynomial, by

IbAbAbAAf kkkk ++++= −−

11

1)( K where I is the unit matrix of order nn× . For example, corresponding to 32)( 2 +−= λλλf and

⎥⎦⎤

⎢⎣⎡−= 31

11A , we have

⎥⎦⎤

⎢⎣⎡−=⎥⎦

⎤⎢⎣⎡+⎥⎦

⎤⎢⎣⎡−−⎥⎦

⎤⎢⎣⎡−⎥⎦

⎤⎢⎣⎡−= 52

211001331

1123111

3111)(Af

Of particularly interest to us are polynomials f having the property that 0)( =Af . For every matrix A one such polynomial can be found by the Cayley-Hamilton theorem which states that: Every square matrix A satisfies its own characteristic equations. For the above example, the characteristic equation of A is

44))(()( 221 +−=−−= λλλλλλλc

where 21,λλ are eigenvalues of A . So that

⎥⎦⎤

⎢⎣⎡=

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡−−⎥⎦

⎤⎢⎣⎡−⎥⎦

⎤⎢⎣⎡−=

+−=

0000

1001431

1143111

3111

44)( 2 IAAAc

In fact the Cayley-Hamilton theorem guarantees the existence of a polynomial )(λc of degree n such that 0)( =Ac .


___________________________________________________________________________________________________________


3.2 State Space Forms Consider the system equation in the form

uyayayay nnnn =++++ −− &K 1

)1(1

)( (3.1) It is assumed that )0(,),0(),0( )1( −nyyy L& are known. If we

define )1(21 ,,, −=== n

n yxyxyx L& , (3.1) is written as

uxaxaxaxxx

xxxx

nnnn

nn

+−−−−−==

==

−

−

1211

1

32

21

K&

&

M

&

&

which can be written as a vector-matrix differential equation

u

xx

xx

aaaaxx

xx

nn

nnnn ⎥

⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

=

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

−−

−10

00

1000

00000010

1

21

1211

21

MM

LL

MMOMMLL

&&M&&

(3.2)

that is, as uBA += xx& , where x , A and B are defined in equation (3.2). The output of the system

[ ]⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=

nx

xx

yM

L 21

001 (3.3)

that is, as, where [ ]001 L=C . The combination of equations (3.2) and (3.3) in the form

⎩⎨⎧

=+=

xxx

CyuBA&

(3.4)

are known as the state equations of the system considered. The matrix A in (3.2) is said to be in companion form. The components of x are called the state variables nxxx ,,, 21 L . The corresponding n-dimensional space called the state space. Any state of the system is represented by a point in the state space. Example 3.1 _______________________________________

Obtain two forms of state equations of the system defined by

uyyyy =−+− 22 &&&&&& where matrix A corresponding to one of the forms should be diagonal. (a) A in companion form Let the state variable as yxyxyx &&& === 321 ,, . Then

uxxx

xxx

⎥⎥⎦

⎤

⎢⎢⎣

⎡+

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

100

212100010

321

321

&&&

(3.5)

and

[ ]⎥⎥

⎦

⎤

⎢⎢

⎣

⎡=

321

001xxx

y

(2) Diagonal form Let

yiyiyix

yiyiyix

yyx

&&&

&&&

&&

)21()2(

)21()2(

101

21

51

3

101

21

51

2

51

51

1

+−+−=

+−+−+=

+=

(3.6)

where 12 −=i . Then

uii

xxx

ii

xxx

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−+−+

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

2121

2

101

0000002

321

321

&&&

(3.7)

and

[ ]⎥⎥

⎦

⎤

⎢⎢

⎣

⎡=

321

111xxx

y

__________________________________________________________________________________________ In general form, a MIMO system has state equations of the form

⎩⎨⎧

+=+=

uxuxx

DCyBA&

(3.14)

and a SISO system has state equations of the form as (3.4). 3.3 Using the transfer function to define state variables It is sometimes possible to define suitable state variables by considering the partial fraction expansion of the transfer function. For example, given the system differential equation

uuyyy 323 +=++ &&&& The corresponding transfer function is

21

12

)2)(1(3

)()()(

+−

+=

+++

==ssss

ssUsYsG

Hence

2)()(

1)(2)(

)()()(

2

1

21

+−=

+=

+=

ssUsX

ssUsX

sXsXsY

On taking the inverse Laplace transforms, we obtain

uxxuxx

−−=+−=

22

112

2&

&

or

uxx

xx

⎥⎦⎤

⎢⎣⎡−+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦

⎤⎢⎣⎡

12

2001

21

21&&

[ ] ⎥⎦⎤

⎢⎣⎡=

2111 x

xy


___________________________________________________________________________________________________________


We can of course make a different choice of state variables, for example

2)()(

1)()(

)()(2)(

2

1

21

+=

+=

−=

ssUsX

ssUsX

sXsXsY

then

uxxuxx+−=

+−=

22

112&

&

Now the state equations are

uxx

xx

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦

⎤⎢⎣⎡

11

2001

21

21&&

[ ] ⎥⎦⎤

⎢⎣⎡−=

2112 x

xy

3.4 Direct solution of the state equation By a solution to the state equation

uBA += xx& (3.15) we mean finding x at any time t given )(tu and the value of x at initial time 00 =t , that is, given 00 )( xx =t . It is instructive to consider first the solution of the corresponding scalar differential equation

ubxax +=& (3.16) given 0xx = at 0=t . The solution of (3.15) is found by an analogous method. Multiplying the equation by the integrating factor ate− , we obtain

ubeaxxe atat −− =− )( &

or ubexedtd atat −− =][

Integrating the result between 0 and t gives

∫ −− =−t

aat dubexxe0

0 )( τττ

that is

444 3444 21321

ergralintparticular

t )t(a

functionarycomplement

at d)(ubexex ∫ −−+=0

0 τττ (3.17)

Example 3.3 _______________________________________

Solve the equation

txx 43 =+& given that 2=x when 0=t . Since 3−=a , 4=b and tu = , on substituting into equation (3.17) we obtain

( )94

343

922

913

913

31330

333

42

42

−+=

+−+=

+=

−

−−

−− ∫

te

eetee

deeex

t

tttt

ttt τττ

__________________________________________________________________________________________ To use this method to solve the vector matrix equation (3.15), we must first define a matrix function which is the analogue of the exponential, and we must also define the derivative and the integral of a matrix.

Definition Since ∑∞

=0

1 nz z!n

e (all z ), we define

KK +++=+++==∑∞

220

021

211 A

!AIA

!AAA

!ne nA

(where IA ≡0 ) for every square matrix A . Example 3.4 _______________________________________

Evaluate Ate given ⎥⎥⎦

⎤

⎢⎢⎣

⎡−=

111001001

A

We must calculate 2A , 3A ,…

K===⎥⎥⎦

⎤

⎢⎢⎣

⎡−= 32

111001001

AAA

It follows that

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−+−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−+−−

+⎥⎥⎦

⎤

⎢⎢⎣

⎡=

−+=

⎟⎠⎞

⎜⎝⎛ ++++=

++++=

ttttt

tttt

t

t

At

eeeee

eeee

e

eAI

tttAI

tAtAtAIe

1101100

111001001

100010001

)1(

31

!21

31

!21

32

3322

K

K

__________________________________________________________________________________________ In fact, the evaluation of the matrix exponential is not quite as difficult as it may appear at first sight. We can make use of the Cayley-Hamilton theorem which assures us that every square matrix satisfies its own characteristic equation. One direct and useful method is the following. We know that if A has distinct eigenvalues, say nλλλ ,,, 21 L , there exists a non-singular matrix P such that

Λ==⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=− },,,{

00

0000

2121

1n

n

diagPAP λλλ

λ

λλ

L

LMOMM

LL

We then have 1−Λ= PPA , so that 1211112 )())(( −−−−− Λ=ΛΛ=ΛΛ= PPPPPPPPPPA


___________________________________________________________________________________________________________


and in general

1−Λ= PPA rr ),2,1( L=r

If we consider a matrix polynomial, say IAAAf +−= 2)( 2 , we can write it as

121

1

12

1112

)}(,),(),({

)(

)2(

2)(

−

−

−

−−−

=

Λ=

+Λ−Λ=

+Λ−Λ=

PfffdiagP

PfP

PIP

PIPPPPPAf

nλλλ L

Since (for the above example)

)}(,),(,)({

1200

0120002

100

010001

00

0000

2

00

0000

2)(

21

2

222

21

21

2

22

21

2

n

nn

nn

fffdiag

If

λλλ

λλ

λλλ

λ

λλ

λ

λλ

λλλ

L

LMOMM

L

L

LMOMM

LL

LMOMM

LL

LMOMM

L

L

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+−

+−−

=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡+

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

+−=

The results holds for the general case when )(xf is a polynomial of degree n . In generalizing the above, taking

AteAf =)( , we obtain 1},,,{ 21 −= PeeediagPe tttAt nλλλ L (3.18)

Example 3.5 _______________________________________

Find Ate given ⎥⎦⎤

⎢⎣⎡

−−= 3120A

The eigenvalues of A are 11 −=λ and 22 −=λ . It can be

verified that ⎥⎦⎤

⎢⎣⎡

−−= 1112P and that ⎥⎦

⎤⎢⎣⎡

−−=−21

111P . ( P can

be found from 1−Λ= PPA ). Using (3.18), we obtain

⎥⎦

⎤⎢⎣

⎡+−+−−−=

⎥⎦⎤

⎢⎣⎡

−−⎥⎦

⎤⎢⎣

⎡⎥⎦⎤

⎢⎣⎡

−−=

−−−−

−−−−

−

−

tttttttt

ttAt

eeeeeeee

eee

2222

2

2222

2111

00

1112

From the above discussion it is clear that we can also write

Ate in its spectral or modal form as

tn

ttAt neAeAeAe λλλ +++= K2121 (3.19)

where nλλλ ,,, 21 L are the eigenvalues of A and ,,, 21 LAA

nA are matrices of the same order as the matrix A . In the above example, we can write

ttAt eee 21121

1122 −−

⎥⎦⎤

⎢⎣⎡ −−+⎥⎦

⎤⎢⎣⎡

−−=

In the solution to the unforced system equation 0xx Ate= , the eigenvalues nλλλ ,,, 21 L are called the poles and

ttt neee λλλ ,,, 21 L are called the modes of the system. __________________________________________________________________________________________ We now define the derivative and the integral of a matrix )(tA whose elements are functions of t .

Definition Let )],([)( tatA ij= then

(1) )],([)()( ijadtdtAtA

dtd

== & and

(2) ],)([)( ∫∫ = dttadttA ij

that is, each element of the matrix is differentiated (or integrated).

For example, if ⎥⎦⎤

⎢⎣⎡

+= 32

2sin62t

ttA , then

⎥⎦⎤

⎢⎣⎡= 02

2cos26t

tA& , and

Cttt

ttAdt +

⎥⎥⎦

⎤

⎢⎢⎣

⎡

+

−=∫ 32

2cos33

31

212

( C is a constant matrix)

From the definition a number of rules follows: ifα and β are constants, and A and B are matrices,

(i) BABAdtd && βαβα +=+ )(

(ii) ∫ ∫∫ +=+b

a

b

a

b

aBdtAdtdtBA βαβα )(

(iii) BABABAdtd && +=)(

(iv) If A is a constant matrix, then AtAt eAedtd

=)(

⊗ Note that although the above rules are analogous o the rules for scalar functions, we must not be dulled into accepting for matrix functions all the rules valid for scalar functions. For example, although xxnx nn

dtd &1)( −= in general,

it is not true that AAnA nndtd &1)( −= .For example,

when ⎥⎦⎤

⎢⎣⎡=

3022 ttA ,

then ⎥⎦⎤

⎢⎣⎡ += 00

664)(232 ttA

dtd

and ⎥⎦⎤

⎢⎣⎡= 00

44223 ttAA &

so that AAAdtd &22 ≠ .

We now return to our original problem, to solve equation (3.15): uBA += xx& . Rewrite the equation in the form

uBA =− xx&

⇒ )()( uBeAe AtAt −− =− xx&


___________________________________________________________________________________________________________


or uBeedtd AtAt −− =)( x

On integration, this becomes

τττ duBetet AAt ∫ −− =−0

)()0()( xx

so that

τττ duBeett tAAt ∫ −==0

)( )()0()( xx (3.20)

Example 3.6 _______________________________________

A system is characterized by the state equation

uxx

xx

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦⎤

⎢⎣⎡

10

3120

21

21&&

If the forcing function 1)( =tu for 0≥t and [ ]T11)0( −=x . Find the state x of the system at time t . We have already evaluated Ate in Example 3.5. It follows that

⎥⎦⎤

⎢⎣⎡−=

⎥⎦⎤

⎢⎣⎡−⎥

⎦

⎤⎢⎣

⎡+−+−−−=

−

−−−−

−−−−

11

11

2222)0(

2

2222

t

ttttttttAt

e

eeeeeeeee x

⎥⎦

⎤⎢⎣

⎡−+−=

⎥⎦

⎤⎢⎣

⎡−

−=

⎥⎦⎤

⎢⎣⎡=

−−

−−

−−−−

−−−−

−−

∫∫∫

tttt

t

tttt

t tAt tA

eeee

deeee

deduBe

22

0)()(2)(2)(

0

)(

0

)(

21

222

10)(

τ

τττ

ττττ

ττ

Hence the state of the system at time t is

⎥⎦

⎤⎢⎣

⎡−+−=⎥

⎦

⎤⎢⎣

⎡−+−+⎥

⎦

⎤⎢⎣

⎡−

= −−

−−

−−

−−

−

−

tttt

tttt

tt

eeee

eeee

eet 2

22

22

2

222121)(x

__________________________________________________________________________________________ The matrix Ate in the solution equation (3.20) is of special interest to control engineers; they call it the state-transition matrix and denote it by )(tΦ , that is,

Atet =)(Φ (3.21) For the unforced system (such as 0)( =tu ) the solution (Eq. (3.20)) becomes

)0()()( xΦx tt = so that )(tΦ transforms the system from its state )0(x at some initial time 0=t to the state )(tx at some subsequent time t - hence the name given to the matrix.

Since Iee AtAt =− . It follows that AtAt ee −− =1][ .

Hence )()(1 tet At −== −− ΦΦ

Also )()()( )( ττ ττ −===− −− teeet tAAAt ΦΦΦ .

With this notation equation (3.20) becomes

τττ duBtttt

∫ −==0

)()()0()()( ΦxΦx (3.22)

3.5 Solution of the state equation by Laplace transforms Since the state equation is in a vector form we must first define the Laplace transform of a vector. Definition

Let ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=

nx

xx

tM21

)(x

We define )(

)(

)()(

)]([

)]([)]([

)]([ 21

21

s

sX

sXsX

tx

txtx

t

nn

Xx =⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=MM

L

LL

L

From this definition, we can find all the results we need. For example,

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

−−

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=

)0()(

)0()()0()(

)]([

)]([)]([

)]([ 2211

21

nnn xssX

xssXxssX

tx

txtx

tM

&M&&

&

L

LL

L x

Now we can solve the state equation (3.15). Taking the transform of uBA += xx& , we obtain

)()()0()( sUBsAss +=− XxX where )]([)( tusU L= or )()0()()( sUBsAsI +=− xX Unless s happens to be an eigenvalue of A , the matrix

)( AsI − is non-singular, so that the above equation can be solved giving

)()()0()()( 11 sUBAsIAsIs −− −+−= xX (3.23) and the solution )(tx is found by taking the inverse Laplace transform of equation (3.23). Definition

1)( −− AsI is called the resolvent matrix of the system. On comparing equation (3.22) and (3.23) we find that

}){()( 11 −− −= AsIt LΦ Not only the use of transforms a relatively simple method for evaluating the transition matrix, but indeed it allows us to calculate the state )(tx without having to evaluate integrals. Example 3.7 _______________________________________

Use Laplace transform to evaluate the state )(tx of the system describe in Example 3.6.

uxx

xx

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦⎤

⎢⎣⎡

10

3120

21

21&&


___________________________________________________________________________________________________________


For this system

⎥⎦⎤

⎢⎣⎡

+−=− 31

2)( ssAsI , so that

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

++

+−

++

+−

+−

++−

+=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

++++−

+++++

=

⎥⎦⎤

⎢⎣⎡−+

++=− −

22

11

21

11

22

12

21

12

)2)(1()2)(1(1

)2)(1(2

)2)(1(3

123

2)3(1)( 1

ssss

ssss

sss

ss

sssss

ss

ssAsI

So that

)(2222}){( 22

2211 teeeeeeeeASI tttt

ttttΦ=⎥

⎦

⎤⎢⎣

⎡+−+−−−=− −−−−

−−−−−−L

Hence the complementary function is as in Example 3.6. For the particular integral, we note that since

stu 1)}({ =L , then

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+−

+

++

+−

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

++

++=

⎥⎦⎤

⎢⎣⎡⎥⎦⎤

⎢⎣⎡−+

++=− −

21

11

21

121

)2)(1(1

)2)(1(2

10

1231

)2)(1(1)()( 1

ss

sss

ss

sss

ss

ssssBUAsI

On taking the inverse Laplace transform, we obtain

⎥⎦

⎤⎢⎣

⎡−+−=− −−

−−−−tt

tt

eeeesBUAsI 2

211 21)}(){(L

which is the particular integral part of the solution obtained in Example 3.6. __________________________________________________________________________________________ 3.6 The transformation from the companion to the diagonal state form Note that the choice of the state vector is not unique. We now assume that with one choice of the state vector the state equations are

xuxx

CyBA

=+=&

(3.24)

where A is the matrix of order nn× and B and C are matrices of appropriate order. Consider any non-singular matrix T of order nn× . Let

zx T= (3.25) then z is also a state vector and equation (3.24) can be written as

zyuzz

TCBTAT

=+=&

or as

zyuzz

1

11C

BA=

+=& (3.26)

where ,1

1 TATA −= ,11 BTB −= TCC =1 . The transformation

(3.25) is called state-transformation, and the matrices A and 1A are similar. We are particular interested in the

transformation when 1A is diagonal (usually denoted by Λ ) and A is in the companion form, such as (3.2)

u

xx

xx

aaaaxx

xx

nn

nnnn ⎥

⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

=

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

−−

−10

00

1000

00000010

1

21

1211

21

MM

LL

MMOMMLL

&&M&&

It is assumed that the matrix A has distinct eigenvalues

nλλλ ,,, 21 L . Corresponding to the eigenvalue iλ there is the eigenvector ix such that

iii xA λ=x ),,2,1( ni L= (3.27) We define the matrixV whose columns are the eigenvectors

[ ]nV xxx L21= V is called the modal matrix, it is non-singular and can be used as the transformation matrix T above. We can write the n equations defined by equation (3.27) as

VVA Λ= (3.28)

where

},,,{

00

0000

2121

n

n

diag λλλ

λ

λλ

L

LMOMM

LL

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=Λ

From equation (3.28) we obtain

VAV 1−=Λ (3.29) The matrix A has the companion form

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

=

−12101000

01000010

naaaa

A

LL

MOMMMLL

so that its characteristic equation is

0)det( 011

1 =++++=− −− aaaAsI n

nn λλλ K ]

By solving this equation we obtain the eigenvalues

nλλλ ,,, 21 L . The corresponding eigenvectors have an interesting form. Consider one of the eigenvalues λ and the


___________________________________________________________________________________________________________


corresponding eigenvector [ ]Tnααα L21=x . Then the equation xx λ=A , corresponds to the system of equations

⎪⎪⎩

⎪⎪⎨

⎧

=

==

−1

23

12

nn αλα

αλααλα

M

Setting 11 =α , we obtain [ ]Tn 121 −= λλλ Lx . Hence the modal matrix in this case takes the form

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=−−− 1

11

11

1

111111

nnnV

λλλ

λλλ

LMOMM

LL

(3.30)

In this form V is called Vandermonde matrix; it is non-singular since it has benn assumed that nλλλ ,,, 21 L are distinct. We now consider the problem of obtaining the transformation from the companion form to diagonal form. Example 3.8 _______________________________________

Having chosen the state vector so that

uyyyy =−− 2_2 &&&&&& is written as the equation (in companion form)

[ ]⎥⎥

⎦

⎤

⎢⎢

⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡+

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

321

321

321

001

100

212100010

xxx

y

uxxx

xxx

&&&

Find the transformation which will transform this into a state equation with A in diagonal form. The characteristic equation of A is

0)1)(2(22)det( 223 =+−=−+−=− λλλλλλ AI that is ,21 =λ i=2λ and i−=3λ .

From (3.30) the modal matrix is

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−−=

1142

111iiV

and its inverse can be shown to be

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−−+−−+=−

iiiiiiV

421048421048

404

2011

The transformation is defined by equation (3.25), that is

zx V= or xz 1−=V . The original choice for x was

[ ]Tyyyx &&&=

so the transformation x1−V gives the new choice of the state vector

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−−+−−+=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

yyy

iiiiii

zzz

&&&

421048421048

404

201

321

The state equations are now in the form of equation (3.26), that is

zzz

1

11Cy

uBA=

+=&

where },,2{1

1 iidiagVAVA −== −

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−+−== −

iiBVB

2121

2

1011

1

[ ]1111 == VCC __________________________________________________________________________________________ 3.7 The transfer function from the state equation Consider the system in state space form of Eq.(3.24)

xuxx

CyBA

=+=&

(3.24)

Taking the Laplace transforms we obtain

)()()()()(

sXCsYsUBsXAsXs

=+=

⇒ )()()( 1 sUBAsIsX −−=

and BAsICsUsYsG 1)()()()( −−== (3.31)

Example 3.9 _______________________________________

Calculate the transfer function from the system whose state equations are

[ ]xxx

2121

4321

−=

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

−−=

y

u&

[ ]

[ ]

)2)(1(23

21

)2)(1(2

)2)(1(3

)2)(1(2

)2)(1(4

21

21

432121)(

1

1

+++

−=

⎥⎦⎤

⎢⎣⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

++−

++

++−

+++

−=

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

+−−−=

−

−

sss

sss

ss

sssss

sssG

__________________________________________________________________________________________ Example 3.10_______________________________________

Given that the system

xuxx

CyBA

=+=&


___________________________________________________________________________________________________________


has the transfer function )(sG , find the transfer function of the system

zzz

TCyuBTATT

=+= −− 11&

If the transfer function is )(1 sG

)()(

})({

)()(

1

111

1111

sGBAsIC

BTTAsITTC

BTATTsITCsG

=−=

−=

−=

−

−−−

−−−

so that )()(1 sGsG = . __________________________________________________________________________________________


___________________________________________________________________________________________________________

Chapter 4 Transient and Steady State Response Analysis

16

C.4 Transient and Steady State Response Analysis 4.1 Introduction Many applications of control theory are to servomechanisms which are systems using the feedback principle designed so that the output will follow the input. Hence there is a need for studying the time response of the system. The time response of a system may be considered in two parts:

• Transient response: this part reduces to zero as ∞→t • Steady-state response: response of the system as ∞→t

4.2 Response of the first order systems Consider the output of a linear system in the form

)()()( sUsGsY = (4.1) where

)(sY : Laplace transform of the output )(sG : transfer function of the system )(sU : Laplace transform of the input

Consider the first order system of the form uyya =+& , its transfer function is

)(1

1)( sUas

sY+

=

For a transient response analysis it is customary to use a reference unit step function )(tu for which

ssU 1)( =

It then follows that

assssasY

/111

)1(1)(

+−=

+= (4.2)

On taking the inverse Laplace of equation (4.2), we obtain

{ 321parttransient

at

partstatesteady

ety /1)( −

−

−= )0( ≥t (4.3)

Both of the input and the output to the system are shown in Fig. 4.1. The response has an exponential form. The constant a is called the time constant of the system.

63.0

00.1

a t

)(tu

Fig. 4.1

Notice that when at = , then 63.01)()( 1 =−== −eayty . The

response is in two-parts, the transient part ate /− , which approaches zero as ∞→t and the steady-state part 1, which is the output when ∞→t . If the derivative of the input are involved in the differential equation of the system, that is, if uubyya +=+ && , then its transfer function is

)()(11)( sU

pszsKsU

asbssY

++

=++

= (4.4)

where abK /=

bz /1= : the zero of the system ap /1= : the pole of the system

When ssU /1)( = , Eq. (4.4) can be written as

psK

sK

sY+

−= 21)( , where pzKK =1 and

ppzKK −

=2

Hence,

{ 43421parttransient

pt

partstatesteady

eKKty −

−

−= 21)( (4.5)

With the assumption that 0>> pz , this response is shown in Fig. 4.2.

2K

1K

t

21 KK −

pteKKy −−= 21

pteK −2

Fig. 4.2

We note that the responses to the systems (Fig. 4.1 and Fig. 4.2) have the same form, except for the constant terms 1K and

2K . It appears that the role of the numerator of the transfer function is to determine these constants, that is, the size of

)(ty , but its form is determined by the denominator. 4.3 Response of second order systems An example of a second order system is a spring-dashpot arrangement shown in Fig. 4.3. Applying Newton’s law, we find

)(tuykyyM +−−= &&& µ where k is spring constant, µ is damping coefficient, y is the distance of the system from its position of equilibrium point, and it is assumed that 0)0()0( == yy & .


___________________________________________________________________________________________________________


17

Mk

)(tu

y

µ

Fig. 4.3

Hence ykyyMtu ++= &&& µ)( On taking Laplace transforms, we obtain

)()(1)(21

22sU

asasKsU

kssMsY

++=

++=

µ

where MK /1= , Ma /1 µ= , Mka /2 = . Applying a unit step input, we obtain

))(()(

21 pspssKsY

++= (4.6)

where2

4 2211

2,1aaa

p−±

= , 1p and 2p are the poles of the

transfer function21

2)(

asasKsG

++= , that is, the zeros of the

denominator of G(s). There are there cases to be considered: Case 1: 2

21 4aa > → over-damped system

In this case 1p and 2p are both real and unequal. Eq. (4.6) can be written as

2

3

1

21)(ps

Kps

Ks

KsY

++

++= (4.7)

where

2211 a

Kpp

KK == ,)( 211

2 pppKK−

= ,)( 122

3 pppKK−

=

(notice that 0321 =++ KKK ). On taking Laplace transform of Eq.(4.7), we obtain

tptp eKeKKty 21321)( −− ++= (4.8)

The transient part of the solution is seen to be

tptp eKeK 2132

−− + .

Case 2: 221 4aa = → critically damped system

In this case, the poles are equal: papp === 2/121 , and

2321

2 )()()(

ps

Kps

Ks

K

pssKsY

++

++=

+= (4.9)

Hence ptpt etKeKKty −− ++= 321)( , where 21 / pKK = ,

22 / pKK −= and pKK /3 −= so that

)1()(2

ptpt etpepKty −− −−= (4.10)

Case 3: 221 4aa < → under-damped system

In this case, the poles 1p and 2p are complex conjugate having

the form βα ip ±=2,1 where 2/1a=α and 2122

1 4 aa −=β .

Hence

2

3

1

21)(ps

Kps

Ks

KsY

++

++= ,

where

,221

βα +=

KK ,)(2

)(222

βαβαβ

+

−−=

iKK)(2

)(223

βαβαβ

+

+−=

iKK

(Notice that 2K and 3K are complex conjugates) It follows that

]sin)(cos)[(

)(

23321

)(3

)(21

tiKKtKKeK

eKeKKtyt

titi

ββα

βαβα

−+++=

++=−

−−+−

(using the relation tite ti βββ sincos += )

tteKK t ββαβ

βαβαα sincos(

2222−−

++

+= − (4.11)

)sin(12

2

2222εβ

βα

βαβαα ++

+−

+= − teKK t (4.12)

where αβε /tan = Notice that when 0=t , 0)( =ty . The there cases discussed above are plotted in Fig. 4.4.

t

)(tu)(ty

1

under damped system

critically damped systemover damped system

Fig. 4.4

From Fig. 4.4, we see that the importance of damping (note that µµ ,/1 Ma = being the damping factor). We would expect that when the damping is 0 (that is, 01 =a ) the system should oscillate indefinitely. Indeed when 01 =a , then

0=α , and 2a=β and since 1sin =ε and 0cos =ε , then 2/πε = , Eq. (4.12) becomes

[ ]taaKta

aKty 2

22

2cos1

2sin1)( −=⎥

⎦

⎤⎢⎣

⎡⎟⎠⎞

⎜⎝⎛ +−=

π

This response of the undamped system is shown in Fig.4.5.


___________________________________________________________________________________________________________


18

t

)(ty

0 2

4aπ

2

2aπ

Fig. 4.5

There are two important constants associated with each second order system.

• The undamped natural frequency nω of the system is the

frequency of the response shown in Fig. 4.5: 2an =ω • The damping ratio ξ of the system is the ratio of the

actual damping )( 1Ma=µ to the value of the damping

cµ , which results in the system being critically damped

(that is, when 21 2 aa = ). Hence 2

1

2 aa

c==

µµξ .

We can write equation (4.12) in terms of these constants. We note that ξωna 21 = and 2

2 na ω= . Hence

2212

2212

2122

1

1

4

2

41/1

ξβα

−=

−=

−+=+

aa

a

aaa

Eq. (4.12) becomes

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛+

−−= − )sin(

1

11)(22 εω

ξωξω teKty t

n

n (4.13)

where

21 ξωω −= n and ξξ

ε21

tan−

= .

It is conventional to choose 1/ 2 =aK and then plot graphs of the ‘normalised’ response )(ty against tω for various values of the damping ratioξ . There typical graphs are shown in Fig. 4.6. Some definitions

t

)(ty

0.19.0

5.0

1.0

dt

rise time

maximum overshoot steady-stateerror ess

Fig. 4.7

(1) Overshoot defined as

%100 valuedesired final

overshoot maximum×

(2) Time delay dt , the time required for a system response

to reach 50% of its final value. (3) Rise time, the time required for the system response to

rise from 10% to 90% of its final value. (4) Settling time, the time required for the eventual settling

down of the system response to be within (normally) 5% of its final value.

(5) Steady-state error sse , the difference between the steady

state response and the input. In fact, one can often improve one of the parameters but at the expense of the other. For example, the overshoot can be decreased at the expense of the time delay. In general, the quality of a system may be judged by various standards. Since the purpose of a servomechanism is to make the output follow the input signal, we may define expressions which will describe dynamics accuracy in the transient state. Such expression are based on the system error, which in its simplest form is the difference between the input and the output and is denoted by )(te , that is, )()()( tutyte −= , where

)(ty is the actual output and )(tu is the desired output ( )(tu is the input). The expression called the performance index can take on various forms, typically among them are:

(1) integral of error squared (IES) ∫∞

0

2 )( dtte

(2) integral of absolute error (IAS) ∫∞

0)( dtte

(3) integral of time multiplied absolute error criterion (ITAE)

∫∞

0)( dttet

Having chosen an appropriate performance index, the system which minimizes the integral is called optimal. The object of modern control theory is to design a system so that it is optimal with respect to a performance index and will be discussed in the part II of this course. 4.4 Response of higher order systems We can write the transfer function of an thn - order system in the form

nnn

mmm

asasbsbsK

sG+++

+++=

−

−

K

K1

1

11 )(

)( (4.14)

Example 4.1________________________________________

With reference to Fig. 2.11, calculate the close loop transfer

function )(sG given the transfer functions3

1)(+

=s

sA and

ssB /2)( =


___________________________________________________________________________________________________________


19

)(sR )(sC

A(s)

B(s)

)(sE

)(sR )(sC

)()(1)()(

sBsAsAsG

+=

Fig. 2.11

We obtain

)2)(1(23)( 2 ++

=++

=ss

sss

ssG

__________________________________________________________________________________________ The response of the system having the transfer function (4.14) to a unit step input can be written in the form

)())(()())((

)(21

21

n

mpspspsszszszsK

sY++++++

=L

L (4.15)

where mzzz ,,, 21 L : the zeros of the numerator

nppp ,,, 21 L : the zeros of the denominator We first assume that mn ≥ in equation (4.14); we then have two cases to consider: Case 1: nppp ,,, 21 L are all distinct numbers. The partial fraction expansion of equation (4.15) has the form

n

nps

Kps

Ks

KsY+

+++

+= +1

1

21)( L (4.16)

121 ,,, +nKKK L are called the residues of the expansions. The

response has the form

tpn

tp neKeKKty −+

− +++= 1211)( K

Case 2: nppp ,,, 21 L are not distinct any more. Here at least one of the roots, say 1p , is of multiplicity r , that is

)()()()())((

)(21

21

nr

m

pspspsszszszsK

sY+++

+++=

L

L (4.17)

The partial fraction expansion of equation (4.17) has the form

1

2

1

2

1

211

)()(

+−

+−+

+++

+++

+=rn

rnr

rps

Kps

Kps

Ks

KsY LL (4.18)

Since ptjj et

jK

psK −−−

−=

⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

+11

)!1()(L , ),,2,1( rj L= , the

response has the form

+−

++++= −−−− tprrtptp etrKeKeKKty 111 12

22211 )!1()( K

tprn

tp rneKeK 1223

+−−+−

− ++L (4.19) We now consider that mn < in equation (4.14); which is the case when the system is improper; that is, it can happen when we consider idealized and physically non-realisable systems,

such as resistanceless circuits. We then divide the numerator until we obtain a proper fraction so that when applying a unit step input, we can write )(sY as

)()()()( 1

1

1111

21n

nnn

nnmn

asassdsdsKscsccKsY

+++

+++++++=

−

−−−

K

KK

(4.20) where Kdc ss ,, and 1K are all constants. The inverse Laplace transform of the first right term of (4.20) involves the impulse function and various derivatives of it. The second term of (4.20) is treated as in Case 1 or Case 2 above. Example 4.2________________________________________

Find the response of the system having a transfer function

8106)65(5)( 23

2

+++

++=

ssssssG

to a unit step input. In this case,

)]1()][1()[4()3)(2(5

8106)65(5)( 23

2

isisssss

ssssssY

−++++++

=+++

++=

The partial fraction expansion as

)1()1(4

)( 4321is

Kis

KsK

sKsY

−++

+++

++=

where4

151 =K ,

41

2 −=K , 4

73

iK +−= ,

47

4iK −−

= .

Hence

)352sin(4

200041

415

)sin2cos14(41

41

415)(

4

4

++−=

+−+−=

−−

−−

tee

tteety

tt

tt

__________________________________________________________________________________________ 4.5 Steady state error Consider a unity feedback system as in Fig. 4.8

)(sR )(sCA(s)

)(sE

Fig. 4.8 where

)(tr : reference input )(tc : system output )(te : error

We define the error function as

)()()( tctrte −= (4.21)


___________________________________________________________________________________________________________


20

hence, )(lim teet

ss→∞

= . Since )()()()( sEsAsRsE −= , it

follows that )(1

)()(sA

sRsE+

= and by the final value theorem

)(1)(lim)(lim

00 sAssRssEe

ssss +

==→→

(4.22)

We now define three error coefficients which indicate the steady state error when the system is subjected to three different standard reference inputs )(sr . (1) step input, )()( tuktr = ( k is a constant)

)(lim1)(1/lim

00 sA

ksAskse

ss

ss

→→ +

=+

=

p

sKsA =

→)(lim

0, called the position error constant, then

pss K

ke+

=1

or ss

ssp e

ekK

−= (4.23)

t

)(tc

k

steady-stateerror ess

)(tuk

Fig. 4.9 (2) Ram input, )()( tutktr = ( k is a constant)

In this case, 2)(sksR = , so that

vss K

ke = or ss

v ekK = , where

)(lim0

ssAKs

v→

= is called the velocity error constant.

t

)(tc steady-stateerror ess

)(tuk

Fig. 4.10

(3) Parabolic input, )(21)( 2 tutktr = ( k is a constant)

In this case, 3)(sksR = , so that

ass K

ke = or ss

a ekK = , where

)(lim 20

sAsKs

a→

= is called the acceleration error constant.

t

)(tc steady-stateerror ess

)(tuk

Fig. 4.11

Example 4.3________________________________________ Find the (static) error coefficients for the system having a

open loop transfer function )24(

8)(+

=ss

sA

∞==

→)(lim

0sAK

sp

4)(lim0

==→

ssAKs

v

0)(lim 20

==→

sAsKs

a __________________________________________________________________________________________ From the definition of the error coefficients, it is seen that sse depends on the number of poles at 0=s of the transfer function. This leads to the following classification. A transfer function is said to be of type N if it has N poles at the origin. Thus if

)()()()()(

1

1

nj

m

pspsszszsKsA

−−

−−=

L

L (4.24)

At 0=s , js sKsA 1

0lim)(→

= where )()()()(

1

11

n

mppzzK

K−−−−

=L

L (4.25)

1K is called the gain of the transfer function. Hence the steady state error sse depends on j and )(tr as summarized in Table 4.1

Table 4.1

sse j System

r(t)=ku(t) r(t)=ktu(t) r(t)=½kt2u(t)

0 1 2

Type 1 Type 2 Type 3

Finite 0 0

∞ finite

0

∞ ∞

finite

4.6 Feedback Control Consider a negative feedback system in Fig. 4.12

)(sR )(sCA(s)

B(s)

)(sE

Fig. 4.12

The close loop transfer function is related to the feed-forward transfer function )(sA and feedback transfer function )(sB by


___________________________________________________________________________________________________________


21

)()(1)()(

sBsAsAsG

+= (4.26)

We consider a simple example of a first order system for

which1

)(+

=as

KsA and csB =)( , so that

aKcs

aKKcasKsG

1/

1)(

++

=++

=

On taking Laplace inverse transform, we obtain the impulse response of the system, where

(1) 0=c (response of open loop system): ateaKtg /)( −=

(2) 0≠c : αt

acK

eaKe

aKtg t −− ==

+1

)( , where 1+

=cKaα

a andα are respectively the time-constants of the open loop and closed loop systems. a is always positive, butα can be either positive or negative. Fig. 4.13 shows how the time responses vary with different values of cK .

t

)(tg

stable region 4=Kc

0=Kc1−=Kc

3−=Kc5−=Kc1−≤Kc

aK

Fig. 4.13

If the impulse response does not decay to zero as t increase, the system is unstable. From the Fig. 4.13, the instability region is defined by 1−≤Kc . In many applications, the control system consists basically of a plant having the transfer function )(sA and a controller having a transfer function )(sB , as in Fig. 4.14.

)(sR )(sCA(s)

)(sEB(s)

)(sQ

controller plant

Fig. 4.14

With the closed loop transfer function

)()(1)()()(sBsA

sBsAsG+

= (4.27)

The controllers can be of various types. (1) The on-off Controller

The action of such a controller is very simple.

⎩⎨⎧

<>

=0)(0)(

)(2

1teifQteifQ

tq

where )(tq is output signal from the controller 1Q , 2Q are some constants The on-off controller is obviously a nonlinear device and it cannot be described by a transfer function.

(2) Proportional Controller For this control action

)()( teKtq p=

where pK is a constant, called the controller gain. The transfer function of this controller is

pKsEsQsB ==)()()( (4.28)

(3) Integral Controller

In this case ∫=t

dtteKtq0

)()( , hence

sKsB /)( = (4.29)

(4) Derivative Controller

In this case dtdeKtq =)( , hence

KssB =)( (4.30)

(5) Proportional-Derivative Controller (PD)

In this case dtdeKteKtq p 1)()( += , hence

)1(1)( 1 sKKsKKKsB p

pp +=⎟

⎟⎠

⎞⎜⎜⎝

⎛+= (4.30)

(6) Proportional-Integral Controller (PI)

In this case ∫+=t

p dtteKteKtq0

1 )()()( , hence

⎟⎠⎞

⎜⎝⎛ +=⎟

⎟⎠

⎞⎜⎜⎝

⎛+=

sKK

sKKKsB p

pp 111)( 1 (4.31)

(7) Proportional-Derivative-Integral Controller (PID)

In this case ∫++=t

p dtteKdtdeKteKtq

021 )()()( , hence

)/1(11)( 2121 skskK

sKKs

KKKsB p

ppp ++=⎟

⎟⎠

⎞⎜⎜⎝

⎛++=

Example 4.4________________________________________ Design a controller for a plant having the transfer function

)2/(1)( += ssA so that the resulting closed loop system has a zero steady state error to a reference ramp input. For zero steady state error to a ramp input, the system must be of type 2. Hence if we choose an integral controller with

sKsB /)( = then the transfer function of the closed loop system including the plant and the controller is

KssK

sBsAsBsA

++=

+ 23 2)()(1)()(


___________________________________________________________________________________________________________


22

The response of this control system depends on the roots of the denominator polynomial 02 23 =++ Kss . If we use PI controller, )/1()( sKKsB p += the system is of type 2 and response of the system depends on the roots of

02 23 =+++ pp KKsKss __________________________________________________________________________________________


___________________________________________________________________________________________________________

Chapter 5 Stability

23

C.5 Stability 5.1 Introduction 5.2 The Concept of Stability The properties of a system are characterized by its weighting function )(tg , that is, its response to a unit impulse or equivalently by the Laplace transform of the weighting function – the transfer function. A system is said to be asymptotically stable if its weighting function response decays to zero as t tends to infinity. If the response of the system is indefinitely oscillatory, the system is said to be marginally stable. Back to the system transfer function has the form

nnn

mmm

asasbsbsK

sG+++

+++=

−

−

K

K1

1

11 )(

)( (5.1)

and the system response is determined, except for the residues, by the poles of )(sG , that is by the solution of

011 =+++ −

nnn asas K (5.2)

Eq. (5.2) is called characteristic equation of the system. It follows that the system is asymptotically stable if and only if the zeros of the characteristic equation (that is the finite poles of the transfer function) nppp ,,, 21 L are negative or have negative real parts. Fig. 5.1 illustrates graphically the various cases of stability.

Type of root s-plane graph ( ωσ js += ) Response graph Remark

Real and negative

σ

ω

t

y

Asymptotically stable

Real and positive

σ

ω

t

y

Unstable

Zero

σ

ω

t

y

Marginally stable

Conjugate complex with negative real

part σ

ω

t

y

Asymptotically stable

Conjugate imaginary (multiplicity r=1) σ

ω

t

y

Marginally stable

Conjugate imaginary (multiplicity r=2) σ

ω

t

y

Unstable

Conjugate with positive real part σ

ω

t

y

Unstable

Roots of multiplicity r=2 at the origin

σ

ω

ty =

t

y

Unstable

Fig. 5.2

5.3 Routh Stability Criterion 5.4 Introduction to Lyapunov’s Method

Lyapunov’s method is the most general method we know for system stability analysis. It is applicable for both linear and nonlinear systems of any order. For linear system, it provides both the necessary and sufficient conditions, whereas for


___________________________________________________________________________________________________________

Chapter 5 Stability

24

nonlinear systems it provides only the sufficient conditions for asymptotically stable. Lyapunov considered the solution )(tx of the state equations to trace out a trajectory (or an orbit or a path) which, for an asymptotically stable system, terminates at the origin of the n-dimensional state-space, known as the phase plane when n=2. Take a simple example: consider the motion of an undamped pendulum. The equation of motion is

0sin =+ θθlg&& (5.4)

which is nonlinear system.

θ l

mg

Fig. 5.2 Assume that the displacements are small, we have θθ ≈sin and (5.4) can be rewritten as

0=+ θθlg&& (5.5)

This is well known linear equation of simple harmonic motion, and has the solution

)/sin( εθ += tlgA (5.6) where A and ε are constants determined by the initial conditions. Let define the state variables: θ=1x , θ&=2x . (5.5) can be written as

⎪⎩

⎪⎨⎧

−=

=

12

21

xlgx

xx

&

&

(5.7)

and the solution

⎪⎩

⎪⎨⎧

+=

+=

)/cos(/

)/sin(

2

1

ε

ε

tlglgAx

tlgAx (5.8)

and obtain the trajectory by eliminating t in (5.8)

222

21

/1 Ax

lgx =+ (5.9)

The trajectories (for various values of A and ε ) are ellipses as shown in Fig. 5.3

1x

2x

Fig. 5.3

Every trajectory shown in Fig. 5.3 is closed, showing that the pendulum, assumed to have no resistance to its motion, will swing indefinitely. A closed trajectories is typical of a periodic motion, that is, one for which the solution )(tx has the property )()( tTt xx =+ , where T is the period of the motion. When the oscillations are damped, the trajectories would terminate at the point 0x = so that it may have the form shown in Fig. 5.4.

1x

2x0x

Fig. 5.4

In general autonomous system, the state equation correspond-ding to equation (5.7) takes the form

⎩⎨⎧

==

),(),(

212

211xxQxxxPx

&

& (5.9)

The solutions corresponding to Eq. (5.8) are

⎩⎨⎧

==

)()(

2

1txtx

ψφ

(5.10)

and can be plotted to give the trajectory. For an n-dimensional state equation, the algebra is similar. Definition A point ),( 0

201 xx , for which ),(0),( 0

201

02

01 xxQxxP == is

called a critical point. For a linear system, there is only one equilibrium point and it is the point 0x = . A system is asymptotically stable when it is stable and the trajectory eventually approaches 0x = as ∞→t . In mathematical terminology, these definitions take the form shown in Fig. 5.5


___________________________________________________________________________________________________________

Chapter 5 Stability

25

The equilibrium point 0x = (Fig. 5.5) is said to be (1) stable if , given any 0>t , there exists 0>δ so that

every solution of (5.10) which at 0=t is such that 2222

221 )0()0( δψϕ <+=+ xx

implies 222 )0()0( εψϕ <+ for 0≥t

(2) asymptotically stable if, (a) it is stable, and

(b) ⎪⎩

⎪⎨⎧

=

=

∞→

∞→

0)(lim

0)(lim

t

t

t

t

ψ

ψ or 0)]()([lim 22 =+

∞→tt

tψψ

(3) unstable if it is not stable

0

)0(x

unstable

δ

1x

2x

stable

ε

asymptoticallystable

Fig. 5.5

In the definition, starting ‘near’ the equilibrium point is defined by the ‘δ -neighborhood’, whereas the ‘reasonable’ distance is the ‘ ε -neighborhood’. If we consider an unforced system, having the state equation

xx A=& (5.11) we have found the solution

)0()(

)0(21

21 x

xxt

ntt

At

neAeAeA

eλλλ +++=

=

K (5.12)

For asymptotic stability, 0)(lim =

→∞t

tx , where )(tx is the

norm of the (solution) vector x defined by

222

21),( nxxx +++== Kxxx

Hence for asymptotic stability all the state variables

nxxx ,,, 21 L must decrease to zero as ∞→t . Form (5.12), we see that this is the case when ),,2,1(0Re nii L=<λ , where iλ is the eigenvalues of A , that is, the solution of the characteristic equation 0)det( =− AsI .

Lyapunov’s method depends on the state trajectory – but not directly. We illustrate his method using the pendulum example. The potential energy U of the system

221)cos1( θθ mglmglU ≈−= (for smallθ )

The kinetic energy of the system

2221 θ&mlK =

Hence the system energy is

),( 21

22

2212

121

22212

21

xxV

xmlmglx

mlmglV

=

+=

+= θθ &

(5.13)

Notice that 0)0,0( =V . It follows that

12

22

2

21 =+

Vbx

Vax , where

mgla 22 = and 2

2 2ml

b =

The trajectory is (again) seen to be an ellipse (Fig. 5.6) with major and minor axes having lengths Va22 and Vb22 respectively.

01x

2x

43

42

1

44 344 21

Va2

Vb2

Fig. 5.6

The rate of change of the energy along a trajectory is

dtdx

dxdV

dtdx

dxdV

dtdVV 2

2

1

1

)(+==

x& (5.14)

0)/( 12

221

22

211

=−+=

+=

xlgxmlxmglxdt

dxxmldtdxmglx

Hence the total energy is a constant along any trajectory. If

0/ <dtdV , the energy is decreasing monotonically along every trajectory. This means that every trajectory moves toward the point of minimum energy, that is the equilibrium point (0,0). This is an essence the Lyapunov’s criterion for asymptotic stability.


___________________________________________________________________________________________________________

Chapter 5 Stability

26

Lyapunov’s function )(xV is similar to the energy function, it has the following properties: (1) )(xV and its partial derivatives ),,2,1(/ nixV i L=∂∂ ,

are continuous. (2) 0)( >xV for 0x ≠ in the neighborhood of 0x = , and

0)( =0V

(3) <)(xV& for 0x ≠ (in the neighborhood of 0x = ), and

0)( =0V&

)(xV satisfying the above properties is called a Lyapunov function. Definition

)(xV is • positive-definite if 0)( >xV for 0x ≠ , and 0)( =0V • positive-semi-definite if 0)( ≥xV for all x • negative-definite if )(xV− is positive-definite • negative-semi-definite if )(xV− is positive-semi-definite

Example 5.5_______________________________________

Classify the Lyapunov function a. 2

221 3)( xxV +=x

0xx ≠∀> ,0)(V and 0)( =0V ⇒ )(xV is positive-definite b. 2

12

21 2)()( xxxV −+−=x 0xx ≠∀< ,0)(V and 0)( =0V ⇒ )(xV is negative-definite c. 2

21 )2()( xxV +=x Whenever 12 2xx −= , 0)( =0V , otherwise 0)( >0V . Hence

0)( ≥xV ⇒ )(xV is positive-semi-definite d. 21

22

21 42)( xxxxV −+=x

0)1,1( <V and 0)3,1( >V . )(xV assumes both positive and negative values⇒ )(xV is indefinite

_________________________________________________________________________________________ Lyapunov’s Theorem The origin (that is the equilibrium point) is asymptotically stable if there exists a Lyapunov function in the neighbor-hood of the origin. If 0)( ≤xV& , then the origin is a stable point. This is sometimes referred to as stability in the sense of Lyapunov. 5.5 Quadratic Forms Definition A quadratic form is a scalar function )(xV of variables

[ ]Tnxxxx L21= defined by

[ ]

∑∑= =

=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=

′=

n

i

n

jjiij

nnnnn

nn

xxp

x

xx

ppp

pppppp

xxx

PV

1 1

21

21

22221121211

21

)(

ML

MOMMLL

L

xxx

(5.15)

The matrix P is of order nn× and is symmetric. Sylvester’s Criterion: the necessary and sufficient conditions for a quadratic form xxx PV ′=)( to be positive-definite is that all the successive principal minors of the symmetric matrix P are positive, that is

011 >p , 022211211 >pp

pp , 0333231232221131211

>ppppppppp

, etc.

Example 5.6_______________________________________

Consider the function

32312123

22

21 22434)( xxxxxxxxxV −−+++=x

Rewrite )(xV in the form

[ ]⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−−−

=

−−+++=

321

321

32312123

22

21

111132124

22434)(

xxx

xxx

xxxxxxxxxV x

Since 4>0, 083224 >= , 05

111132124

>=−−

−−

, Sylvester’s

criterion assures us that )(xV is positive-definite. _________________________________________________________________________________________ 5.6 Determination of Lyapunov’ Functions Definition The matrix A is said to be stable, if the corresponding system defined by equation xx A=& is stable. As explained above, to determine whether A is stable we seek a symmetric matrix P so that xx PxV ′=)( is a positive-definite function, and

xxxxxx

xxxx

][)(

)(

PAPAPAPA

PPxV

+′′=

′+′=

′+′= &&&

(5.16)

is negative. Since P is symmetric, ][ PAPA +′ is symmetric.

Hence )(xV& is a quadratic form. Let PAPAQ +′=− (5.17) for )(xV& to be negative-definite, xx Q′ must be positive-definite.


___________________________________________________________________________________________________________

Chapter 5 Stability

27

Lyapunov’s theorem can now be expressed in the following form:

Given a positive-definite matrix Q , the matrix A is stable if the solution to the equation (5.17) results in a positive-definite matrix P .

Example 5.7_______________________________________

Use Lyapunov’s direct method to determine whether the following systems are asymptotically stable:

a. ⎩⎨⎧

−==

212

212 xxxxx

&

&

Solving IPAPA −=+′ , we obtain

⎥⎦⎤

⎢⎣⎡−

−−= 1113

41P

which, by Sylvester’s criterion, is not positive definite. Hence the system is not stable.

b. ⎩⎨⎧

−−==

212

2125 xxx

xx&

&

We obtain ⎥⎦⎤

⎢⎣⎡= 31

117101P which is positive definite. Hence

the system is asymptotically stable. _________________________________________________________________________________________ Example 5.8_______________________________________

Determine using Lyapunov’s method, the range of K for the system in Fig. 5.8 to be asymptotically stable

)(sU )(1 sX1+s

K2

3+s

)(2 sX

Fig. 5.8

From Fig. 5.8

21 23 X

sX

+= ⇒ 211 32 xxx +−=&

)(1 12 UX

sKX +−+

= ⇒ KuxKxx +−−= 212&

The state equation is

uKxx

Kxx

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−−=⎥⎦

⎤⎢⎣⎡ 0

132

21

21&&

Solving IPAPA −=+′

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦

⎤⎢⎣⎡

−−−

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−−

1001

132

132

22211211

22211211

Kpppp

ppppK

We obtain ⎥⎦⎤

⎢⎣⎡

+−−++

+=

153232333

12181 2

KKKKK

KP

Sylvester’s criterion leads us to conclude that 3/2−>K . _________________________________________________________________________________________

5.7 The Nyquist Stability Criterion If the transfer function of a system is not known, it is often possible to obtain it experimentally by exciting the system with a range of sinusoidal inputs of fixed amplitude by varying frequencies, and observing the resulting changes in amplitude and phase shift at the output. The information obtained in this way, known as the system frequency response, can be used in various ways for system analysis and design. 5.8 The Frequency Response Consider the system )()()( sUsGsY = . For a sinusoidal input

of unit amplitude ttu ωsin)( = so that )/()( 22 ωω += ssU and

22)(0(

ωω+

=s

sGsY

Assuming that )(sG has poles at mppps −−−= ,,, 21 L ,

)(sY can be expanded in partial fractions as

ωω isQ

isQ

psK

sYn

j j

j

−−

++

+=∑

=

21

1

)( (5.18)

and

jps

jj

s

sGpsK

−=+

+=

22

)()(

ω

ω

)(21)()(

221 ωω

ω

ω

iGis

sGpsQ

is

j −−=+

+=

−=

)(21)()(

222 ωω

ω

ω

iGis

sGpsQ

is

j =+

+=

=

On taking the inverse Laplace transform of equation (5.18), we obtain

444 3444 214434421formstatesteady

titi

formtransient

n

j

tpj eQeQeKty j

−

−

=

− ++=∑ ωω21

1

)( (5.19)

The steady-state term can be written as

})(Im{])()([21 tititi eiGeiGeiGi

ωωω ωωω =−− − (5.20)

In polar form we can express φωω ieiGiG )()( = , where

)}(Re{)}(Im{tan 1

ωωφ

iGiG−= , and Eq. (5.12) becomes

)sin()(})(Im{ )( φωωω φω +=+ tiGeiG ti (5.21)


___________________________________________________________________________________________________________

Chapter 5 Stability

28

)sin()(})(Im{ )( φωωω φω +=+ tiGeiG ti (5.21) Eq. (5.21) shows that the steady-state output is also sinusoidal, and relative to the input the amplitude is multiplied by )( ωiG . The frequency diagram is the plot of )( ωiG in the complex plane as the frequency varies form −∞ to +∞ . Since for the systems considered the transfer function )(sG are the ratios of two polynomials with real coefficients, it follows that

)( ωiG = the complex conjugate of )( ωiG − and it is sufficient to plot )( ωiG for ∞<≤ω0 ; the remainder of the locus is symmetrical about the real axis to this plot. Example 5.9_______________________________________

Determine the frequency diagrams for the following transfer functions a. KssG =)(

ωω iKiG =)( , hence ωω KiG =)( and 090)( =ωϕ ⇒ the

plot is the positive imaginary axis (for 0≥ω ) b. sKsG /)( =

)/()( ωω iKiG = , hence ωω /)( KiG = and 090)( −=ωϕ

⇒ the plot is the negative imaginary axis (for 0≥ω ) c. )/()( saKsG +=

)/()()/()( 22 ωωωω +−=+= aiaKiaKiG , hence 22/)( ωω += aKiG and )/(tan)( 1 aωωϕ −−=

⇒ the plot for 0≥ω is given in Fig. 5.9

0 ϕ)( ωiG

+∞=ω

−∞=ω

ωincreasing

aK2 a

K

0=ω

Imaginaryaxis

Real axis

Locus correspondingto negative frequency

Fig. 5.9

_________________________________________________________________________________________ If the transfer function involves several factors, the above procedure can be used for each factor individually, and the overall result is obtained by the following rules If )()()()( 21 sGsGsGsG kL= , then

)()()()( 21 ωωωω iGiGiGiG kL= , and

)()()()( 21 ωφωφωφωφ k+++= K ,

where

)}(Re{)}(Im{

tan)( 1ωω

φiGiG

kj

jj

−= ),,2,1( kj L=

Example 5.10______________________________________

Construct the frequency diagram for the system having the transfer function

)1()(

+=

ssKsG ( K is constant)

Since )()(1

1)( 21 ωωωω

ω iGiGi

Ki

iG =⎟⎠⎞

⎜⎝⎛

+⎟⎠⎞

⎜⎝⎛= , we obtain

21

1

11)(ωω

ω+

=iG and ωπφ 1tan2/ −−−= . The plot is

given in Fig. 5.10 Imaginary

axis

Real axis

G

0=ω

0=

ω

ϕ

−∞=ω

+∞=ω

Fig. 5.10

________________________________________________________________________________________ To use the Nyquyist criterion we must plot the frequency response of the system’s open-loop transfer function. Consider the closed loop system in the Fig. 5.11

)(sR )(sCG(s)

H(s)

)(sE

)()()(1 sCsHsC =

Fig. 5.11

The closed loop transfer function

)()(1)(

)()(

sHsGsG

sGsC

+= (5.22)

The ratio of the feedback signal )(1 sC to the actuating error

signal )(sE is )()()()(1 sHsG

sEsC

= and is called the system

open-loop transfer function. If we consider the unity negative feedback system shown in

Fig. 5.12 we again find that )()()()(1 sHsG

sEsC

=


___________________________________________________________________________________________________________

Chapter 5 Stability

29

)(sR )(sCG(s)H(s)

)(sE

)(1 sC

Fig. 5.12

And both of the systems in Fig. 5.11 and Fig. 5.12 have the same open-loop transfer functions. Assume that if there are any factors which cancel in the product )()( sHsG they involve only poles and zeros in the left haft s-plane. We will consider the system to be stable so long as the close loop transfer function equation (5.22) has no poles in the RHP. We note that 1ss = is a pole of equation (5.22) if (1) 0)()(1 11 =+ sHsG and 0)( 1 ≠sG

(2) nsssAsG

)()()(

1−= and mss

sBsHsG)()()()(1

1−=+ , where

mn > 0)( 1 ≠sA and 0)( 1 ≠sB . From (1) and (2), we conclude that the system having the transfer function equation (5.22) is stable if and only if

)]()(1[ sHsG+ has no zeros in RHP. 5.9 An Introduction to Conformal Mappings To establish the Nyquist criterion we regard the open loop transfer function )()()( sFsHsG = of a feedback system as a mapping from the s-plane to the )(sF -plane. As an example, consider )4/(1)( += ssF . We consider two planes: (1) the s-plane, having a real axis, σ=}Re{s and imaginary

axis ω=}Im{s , where ωσ is += (2) the )(sF -plane which has similarly a real axis

)}(Re{ sF and imaginary axis )}(Im{ sF Consider corresponding paths in the two planes as in Fig. 5.13

1.0)}(Re{ sF

)}(Im{ sF

0σ

ωi

D

AB

C

i

-i

-11

2.0 3.0

1.0

1.0−

A’

D’

C’

B’path−γ path−Γ

planes − planesF −)(

Fig. 5.13

In Fig. 5.13 it is shown that )(sF traces out circular arcs as s moves along the insides of the square. This can of course be proved analytically. For example, along DA

ωis +=1 11 ≤≤− ω and corresponding along '' AD

ivuii

sF +=+

−=

+= 225

55

1)(ωω

ω

where 2255ω+

=u and 225 ωω+

−=v . They relationship is

222 )10/1()10/1( =+− vu

This is the equation of a circle, center )0,10/1( and radius 10/1 . In Fig. 5.13 we note that (1) the closed contour γ in the s -plane is mapped into a

closed contour Γ in the )(sF -plane (2) Any point inside γ maps into a point inside Γ (3) We can define a positive direction in which s traces out

its path. It does not matter which direction is chosen to be positive.

It is interesting to consider the direction in which )(sF sweeps out its path, as s moves along a contour which encloses the pole )4( −=s of )(sF . This is shown in Fig. 5.14.

)}(Re{ sF

)}(Im{ sF

σ

ωi

D

AB

C A’

D’C’

B’

path−γ path−Γ

planes − planesF −)(

-5 -4 -3 -2 0

i

-i

Fig. 5.14

In this case we note that as s traces the γ -contour in the positive direction, )(sF traces the corresponding Γ -contour in the opposite, that is, negative direction.. Cauchy’s Theorem Let γ be a closed contour in the s -plane enclosing P poles and Z zeros of )(sF , but not passing through any poles or zeros of )(sF . As s traces out γ in the positive direction,

)(sF traces out a contour Γ in the −)(sF plane, which encircles the origin )( PZ − times in the positive direction. In Fig. 5.13, γ does not enclose any poles or zeros of )(sF , so that ZP == 0 ⇒ the origin of the )(sF -plane is not encircled by the Γ -contour. In Fig. 5.14, γ encircles a pole of )(sF , so that 1=P ,

0=Z , hence 1−=− PZ ⇒ Γ -contour encircles the origin (-1) times in the positive direction, that is, Γ encircles the origin once in the negative direction.


___________________________________________________________________________________________________________

Chapter 5 Stability

30

5.10 Application of Conformal Mappings to the Frequency Response In section 5.8 we discussed the frequency response, which required s to trace out the imaginary axis in the s-plane. In section 5.9, Cauchy’s theorem calls for a closed contour in the s-plane avoiding the poles and zeros of )(sF . We now combine these two contours and define the Nyquist path. Nyquist path consist of the entire imaginary axis, and a semicircular path of large radius R in the right half of s-plane, as shown in Fig. 5.15

)}(Re{ sF

)}(Im{ sF

σ

ωi

Γ

planesF −)(

0

γ

∞→R

Nyquistpath

planes −

Fig. 5.15

The small semicircular indentations are used to avoid poles and zeros of )(sF which lie along the imaginary axis. The semi circle in the s-plane is assumed large enough to enclose all the poles and zeros of )(sF which may lie in the RHP. The Nyquist Criterion To determine the stability of a unity feedback system, we consider the map Γ in the )(sF -plane of the Nyquist contour traversed in a positive direction. The system is stable if (1) Γ does not encircle the point (-1,0) when the open-loop

transfer function )()( sHsG has no poles in the RHP. (2) Γ encircles the point (-1,0) P times in the positive

direction when the open-loop transfer function )()( sHsG has P poles in the RHP.

Otherwise, the system is unstable. Fig. 5.7 shows examples of a stable and an unstable system for the case where 0=P .

planesF −)(

0

planesF −)(

0-1 -1

unstable systemstable system Fig. 5.17

Example 5.11______________________________________

Use the Nyquist criterion to determine the stability of the closed-loop system having an open-loop transfer function

)2)(1()(

++=

sssKsF

when (1) 1=K and (2) 10=K Since )(sF has a pole at the origin, the Nyquist contour has a semicircular indentation as shown in Fig. 5.18(a)

σ

ωiplanesF −)(

0

planes −

R

rα

θ

∞i

∞−i

+0i

−0i

0.10.2

+∞=ω

−∞=ω

0−=ω)0( −iF

2=ω

0+=ω )0( +iF(a) (b)

Fig. 5.18 The )(sF contour for 1=K corresponding to the Nyquist path is shown in Fig. 5.18(b). (1) For 1=K , the point ),1( +∞− is not encircled by the )(sF contour. Since )(sF has no poles in the RHP, we can conclude that the system is stable. (2) For 10=K , the magnitude )( ωiF is 10 times larger than

when 1=K for each finite value of ω while the phase φ is unchanged. This means that Fig. 5.18 is valid for 10=K if we change the coordinate scales by a factor of 10. This time the critical point ),1( +∞− is not encircled twice. We can conclude that there are two closed-loop poles in the RHP, and the system is unstable. ________________________________________________________________________________________


___________________________________________________________________________________________________________

Chapter 6 Controllability and Observability

31

C.6 Controllability and Observability 6.1 Introduction 6.2 Controllability We consider a system described by the state equations

⎩⎨⎧

=+=

xyxx

CuBA&

(6.1)

where A is nn× , B is mn× and C is nr × . With the transformation

zx P= (6.2) we can transform equation (6.1) into the form

⎩⎨⎧

=+=

xyzx

1

11C

uBA& (6.3)

where PAPA 1

1−= , BPB 1

1−= and PCC =1 . Assuming

that A has distinct eigenvalues nλλλ ,,, 21 L we can choose P so that 1A is a diagonal matrix, that is,

},,,{ 211 ndiagA λλλ L= If 2=== rmn the first order of (6.3) has the form

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡=⎥⎦

⎤⎢⎣⎡

21

22211211

21

21

21

00

uu

bbbb

zz

zz

λλ

&&

which is written as

⎩⎨⎧

′+=

′+=ubzz

ubzz

2222

1111λλ

&

& (6.4)

where 1b′ and 2b′ are the row vectors of the matrix 1B . The output equation is

22221

12111

21

22211211

21 zc

czcc

zz

cccc

yy

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡=⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡=⎥⎦

⎤⎢⎣⎡

or 2211 zczcy += (6.5)

where 21,cc are the vectors of 1C . So in general, equation (6.3) can be written in the form

⎪⎪⎩

⎪⎪⎨

⎧

=

=′+=

∑=

n

iii

iiii

y

ni

1

),,2,1(

zc

ubzz L& λ

(6.6)

It is seen from equation (6.6) that if ib′ , the thi row of 1B , has all zero components, then 0+= iii zz λ& , and the input

)(tu has no influence on the thi mode (that is tieλ ) of the system. The mode is said to be uncontrollable, and a system having one or more such modes is uncontrollable. Otherwise, where all the modes are controllable the system is said to be completely state controllable, or just controllable. By definition, the system equation (6.6) is controllable if only if a control function )(tu can be found to transform the initial

state [ ]Tnzzzz 002010 L= to a specifies state

[ ]Tnzzzz 112111 L= . Example 6.1_______________________________________

Check whether the system having the state-space representation

[ ]xy

xx

2164

4321

−=

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡−= u&

is controllable. The eigenvalues 11 =λ , 22 =λ and the corresponding

eigenvectors [ ]T211 =x , [ ]T322 =x , so that the modal matrix is

⎥⎦⎤

⎢⎣⎡= 31

21P and ⎥⎦⎤

⎢⎣⎡−

−=−11231P

Using transformation zx P= , the sate equation becomes

[ ]zy

zz

4120

2001

−−=

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡= u&

This equation shows that the first mode is uncontrollable and so the system is uncontrollable. ________________________________________________________________________________________ Definition The matrix ]|||[ 1BABABQ n−= L is called the system controllability matrix. The Controllability Criterion

nQrank =}{ ⇔ the system is controllable Example 6.2_______________________________________

Using the controllability criterion verify that the system in example 6.1 is uncontrollable.


___________________________________________________________________________________________________________


32

For the system

⎥⎦⎤

⎢⎣⎡= 64b and ⎥⎦

⎤⎢⎣⎡=⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡−−= 12

864

4321bA ,

so that

[ ] 1}12684{}{}{ =⎥⎦⎤

⎢⎣⎡== rbAbrQr

Since the rank of Q is less than 2, the system is uncontrollable. ________________________________________________________________________________________ Example 6.3_______________________________________

Determine whether the system, governed by the equation

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−+⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦⎤

⎢⎣⎡

21

21

21

2163

5160

uu

xx

xx&&

is controllable. Using the controllability criterion

[ ] }42126

2163{}{}{ ⎥⎦

⎤⎢⎣⎡ −−

−−== rBABrQr

It is obvious that the rank of this matrix is 1. The system is therefore uncontrollable. ________________________________________________________________________________________ 6.3 Observability

Consider the system in the form⎩⎨⎧

=+=

xyuzz

1

11

CBA&

. If a row of

the matrix 1C is zero, the corresponding mode of the system will not appear in the output y . In this case the system is unobservable, since we cannot determine the state variable corresponding to the row of zeros in 1C from y . Example 6.4_______________________________________

Determine whether the system have the state equations

[ ] ⎥⎦⎤

⎢⎣⎡−=

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡−−=⎥⎦

⎤⎢⎣⎡

21

21

21

23

21

5645

xxy

uxx

xx&&

is observable.

Using the modal matrix ⎥⎦⎤

⎢⎣⎡= 31

21P , the transform matrix

zx P= , transform the state-equations into

[ ] ⎥⎦⎤

⎢⎣⎡=

⎥⎦⎤

⎢⎣⎡−+⎥⎦

⎤⎢⎣⎡⎥⎦⎤

⎢⎣⎡−=⎥⎦

⎤⎢⎣⎡

21

21

21

01

11

1001

zzy

uzz

zz&&

This results shows that the system is unobservable because the output y does not influenced by the state variable 2z . ________________________________________________________________________________________

The above example illustrates the importance of the observability concept. In this case we have a non-stable system, whose instability is not observed in the output measurement. Definition The matrix TnTT ACACCR ]|||[ 1−= L is called the system observability matrix. The Observability Criterion

nRrank =}{ ⇔ the system is observable Example 6.5_______________________________________

Using observability criterion, verify that the system examined in example 6.4 is unobservable For this system

[ ]23 −=′c and ⎥⎦⎤

⎢⎣⎡−−= 56

45A ,

hence

⎥⎦⎤

⎢⎣⎡−

−= 2323R , so that 1}{ =Rr

Since the rank of R is less than 2, the system is unobservable. ________________________________________________________________________________________ 6.4 Decomposition of the system state From the discussion in the two previous sections, the general state variables of a linear system can be classified into four exclusive groups as follows:

• controllable and observable • controllable and unobservable • uncontrollable and observable • uncontrollable and unobservable

Assuming that the system has distinct eigenvalues, by appropriate transforms the state equations can be reduced to the form below

[ ]⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

4321

31

21

4321

43

21

4321

00

00

000000000000

xxxx

xxxx

xxxx

CCy

uBB

AA

AA

&&&&

(6.11)

The (transformed) systems matrix A has been put into a “block-diagonal” form where each )4,3,2,1( =iAi is in diagonal form. The suffix i of the state-variable vector ix implies that the elements of this vector are the state

variables corresponding to the thi group defined above.


___________________________________________________________________________________________________________


33

Example 6.6______________________________________

Consider the system

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡ −

+

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

−

−

−−

=

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

654321

21

21

654321

654321

101101102100

200100110011

400000000000003000000100000010000002

xxxxxx

yy

uu

xxxxxx

xxxxxx

&&&&&&

By inspection we can rewritten the above system in the following form

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡ −

+

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

−−

−

−

=

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

245631

21

21

245631

245631

010111020110

000001201111

100000030000000000000400000010000002

xxxxxx

yy

uu

xxxxxx

xxxxxx

&&&&&&

hence

• controllable and observable 63,1 , xxx

• controllable and unobservable 5x • uncontrollable and observable 4x • uncontrollable and unobservable 2x

________________________________________________________________________________________ We can represent the decompositions of the state variables into four groups by a diagram

S1

S2

S3

S4

uy

Fig. 6.1

In general, a transfer function )(sG represents only the subsystem 1S of the system considered, and indeed on adding to 1S the subsystem 432 ,, SSS will not effect )(sG .

Example 6.7_______________________________________

Illustrate the above discussion with the systems considered in examples 6.1 and 6.4 In example 6.1, the state equations were transferred into the diagonal form

[ ] ⎥⎦⎤

⎢⎣⎡−−=

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡=⎥⎦

⎤⎢⎣⎡

21

21

21

41

20

2001

zzy

uzz

zz&&

and can be represented by Fig.6.2

uzz 22 22 +=&

11 zz =& -1

-4u

y

Fig. 6.2

The system has two modes corresponding to the poles 1=λ and 2=λ . The transfer function

[ ]2

820

110

02

1

41][)( 1−−

=⎥⎦⎤

⎢⎣⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

−−−=−= −

ss

sBAsICsG

It is seen that the transfer function involves only the mode corresponding to 2=λ - the controllable and observable one. The uncontrollable mode 1=λ does not appear in the transfer function. In example 6.4 the transformed state equations are

[ ] ⎥⎦⎤

⎢⎣⎡=

⎥⎦⎤

⎢⎣⎡−+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡−=⎥⎦

⎤⎢⎣⎡

21

21

21

01

11

1001

zzy

uzz

zz&&

and can be represented in Fig.6.3

uzz 22 22 +=&

uy

uzz += 11&

Fig. 6.3

In this case

[ ]1

111

110

01

1

01][)( 1+−

=⎥⎦⎤

⎢⎣⎡−

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

+=−= −

ss

sBAsICsG

This result shows that the unobservable mode 1=λ does not involved in the transfer function. ________________________________________________________________________________________


___________________________________________________________________________________________________________


34

Definition

The state equation⎩⎨⎧

=+=

xyxx

CuBA&

are said to be a realization

of the transfer function )(sG if BAsICsG 1][)( −−= . Definition

A realization equation⎩⎨⎧

=+=

xyxx

CuBA&

of a transfer function

)(sG is said to be minimal if it is both controllable and observable. Example 6.8_______________________________________ Obtain a minimal realization of a system having the transfer function

⎥⎦⎤

⎢⎣⎡

+++−−

++= )1(223

)1()2)(1(

1)( ssss

sssG

The system has two modes 1−=λ and 2−=λ . Hence we can write

⎥⎦⎤

⎢⎣⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+

+⎥⎦⎤

⎢⎣⎡=

−=

⎥⎦⎤

⎢⎣⎡

+++−−

++=

−

22211211

22211211

1

210

01

1][

)1(223)1(

)2)(1(1)(

bbbb

s

scccc

BAsIC

ssss

sssG

And the minimal realization is therefore

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡−

−=

⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

−−=⎥⎦

⎤⎢⎣⎡

21

21

21

2111

1201

2001

xxy

uxx

xx&&

________________________________________________________________________________________

6.5 A Transformation into the Companion Form There exists a transformation

zx T= (6.12) transforming the system

uA bxx +=& (6.13) where A is nn× matrix assumed to have n distinct eigenvalues, into

uC dzz +=& (6.14) where ATTC 1−= is in a companion form, that is

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−−−−

=

−1210

1000

001000010

naaaa

C

L

L

MOMMM

L

and

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

== −

1

000

1

M

bd T

(6.15)

Also the characteristic equation of A yields 00

11 =−=+++=− −− CIaaAI n

nn λλλλ L

We construct the transformation matrix T in the following way: Let p be a vector of order 1×n . Then 12 ,,, −′′′ nAAA ppp L are n row vectors, which we collect to make up the n rows of the

matrix 1−T , that is

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

′

′′

=

−

−

1

1

nA

AT

p

pp

M (6.16)

Assume for the moment that 1−T is a non-singular matrix, so that T exists and has the portioned form

[ ]nT qqq L11= (6.17) so that ITT =−1 , hence

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

′′′

′′′′′′

−−− 100

010001

12

11

1

21

21

L

MOMM

L

L

L

MOMM

L

L

nnnn

n

n

AAA

AAA

qpqpqp

qpqpqpqpqpqp

(6.18) On taking the product

[ ]n

n

A

A

AATT qqq

p

pp

LM

11

1

1

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

′

′′

=

−

−

we obtain the matrix

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

′′′

′′′′′′

nnnn

n

n

AAA

AAAAAA

qpqpqp

qpqpqpqpqpqp

L

MOMM

L

L

21

22

21

221

(6.19)

Note that the first )1( −n rows of equations (6.19) are the same (row by row) as the last )1( −n rows of the matrix equation (6.18) ⇒ the matrix equation (6.19) is in the companion form, and


___________________________________________________________________________________________________________


35

⎪⎪

⎩

⎪⎪

⎨

⎧

′=−

′=−

′=−

− nn

n

n

n

Aa

Aa

Aa

qp

qp

qp

1

21

10

(6.20)

We must also have

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

′

′′

− 1

00

1MM

b

p

pp

nA

A or

⎪⎪

⎩

⎪⎪

⎨

⎧

=′

=′=′

− 0

00

1bp

bpbp

nA

A

that is

[ ] dbbbp ′=′ −1nAA L (6.21) Eq. (6.21) is a system of n simultaneous equations in n unknowns (the components of p ) and will have a unique solution if and only if the rank of the system controllability matrix is n , that is if

[ ] nAArank n =− bbb 1L (6.22) (then the system is controllable). Example 6.9_______________________________________ The state equation of a system is

u⎥⎦

⎤⎢⎣

⎡−+⎥

⎦

⎤⎢⎣

⎡−−

=12

1663513

xx&

Find the transformation so that the state matrix of the transformed system is in a companion form.

Using ⎩⎨⎧

=′=′

00

bpbp

A, we obtain

[ ] 012

21 =⎥⎦

⎤⎢⎣

⎡−pp and [ ] 1

12

1663513

21 =⎥⎦

⎤⎢⎣

⎡−⎥⎦

⎤⎢⎣

⎡−−

pp

that is

⎩⎨⎧

=−=+−

14902

21

21pppp

⇒ ⎩⎨⎧

==

21

2

1pp

and the transformation matrix

⎥⎦

⎤⎢⎣

⎡−

−=

1123

T and ⎥⎦

⎤⎢⎣

⎡=−

31211T

The required transformation is

zx ⎥⎦

⎤⎢⎣

⎡−

−=

1123

&

________________________________________________________________________________________


___________________________________________________________________________________________________________

Chapter 7 Multivariable Feedback and Pole Location

36

C.7 Multivariable Feedback and Pole Location

7.1 Introduction 7.2 State Feedback of a SISO System Consider a SISO system described by the state equations

uA bxx +=& (7.1) The dynamics of this system are determined by the poles, that is, the eigenvalues nλλλ ,,, 21 L (assumed distinct) of the matrix A . If system is controllable, there exists a matrix T such that

zx T= transforms the system defined by equation (7.1) to

udC += zz& (7.2) where

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

== −

naaaa

TATC

LL

MOMMMLL

321

1

1000

01000010

,

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

== −

10

00

1 MbTd

Now apply the feedback control of the form

zk ′=)(tu , [ ]nkkk ,,, 21 L=′k (7.3) yields

zkz ][ ′+= dC& (7.4) which is the system whose dynamics are determined by the eigenvalues of ][ k ′+ dC , that is by the eigenvalues of

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

n

nn

rrrr

kkkkaaaa

LL

MOMMMLL

LL

MOMMMLL

LL

MOMMMLL

321

321321

1000

01000010

0000

00000000

1000

01000010

(7.5) where iii akr −=− , ),,2,1( ni L= The characteristic equation of ][ k ′+ dC is

0121 =++++ − rsrsrs n

nn K

By appropriate choice of the components nkkk ,,, 21 L of k in equation (7.5) we can assign arbitrary values to the coefficients nrrr ,,, 21 L and so achieve any desired pole

configurations. Since xz 1−= T it follows that

xkzk 1)( −′=′= Ttu (7.6)

Example 7.1_______________________________________

Given the system u⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡−−= 2

11525813 xx& , find the feedback

control law so that the resulting closed loop system has poles at 2−=λ and 3−=λ .

With the transformation matrix ⎥⎦⎤

⎢⎣⎡= 21

11T and

⎥⎦⎤

⎢⎣⎡−

−=−11121T the state equation becomes

uuTTAT ⎥⎦⎤

⎢⎣⎡+⎥⎦

⎤⎢⎣⎡−=+= −−

10

251011 zbzz&

the characteristic equation in this case is 0522

1 =+− λλ and the poles are at i21±=λ so that the system is unstable.

Apply feedback law in the form [ ] ⎥⎦⎤

⎢⎣⎡=

21

21 zzkku and using

equation (7.5)

752

165

222

111−=−−=−=

−=−=−=rakrak

and the control law is

[ ] [ ]xxxkzk 65111271)( 1 −=⎥⎦⎤

⎢⎣⎡−

−−−=′=′= −Ttu

• Checking the poles of the closed loop system After assigned poles

xkbbxx )( 1−′+=+= TAuA& and

[ ] ⎥⎦

⎤⎢⎣

⎡−−

=−⎥⎦

⎤⎢⎣

⎡+⎥

⎦

⎤⎢⎣

⎡−−

=′+ −

31528

6521

15258131TA kb

The corresponding characteristic equation is

0)3)(2(652 =++=++ λλλλ hence the closed loop poles were assigned at 2−=λ and

3−=λ as desired. ________________________________________________________________________________________ 7.3 Multivariable Systems The pole assignment for a multivariable system is more complicated than for a scalar system since a multitude of inputs are to be controlled, and not just one ! We will discuss a relatively simple method dealing with this problem.


___________________________________________________________________________________________________________


37

• Matrix property If P and Q are matrices of orders nm× and mn × respectively, then

|||| PQIQPI nm −=− (7.7) where the notation || A stands for the determinant of A .

mI and nI are the unit matrices of order nm× and mn × respectively. • Proof Firstly, take a notice about some matrix properties For any BA, : |||| BAAB = For any square matrix A

||0

00

0A

IA

AI

==

10

0==

IAI

IAI

When mn =

|||)(||)(||| 11 QPIPQPQPPPQI −=−=−=− −− When mn ≠

||0

0

PQIIPPQI

IQI

IQPI

IQPI

m

n

m

n

m

n

m

n

m

−=

−=

−=

(7.9)

||0

0

QPIQPI

PI

IQPI

IQI

IQPI

n

n

m

n

m

n

m

n

m

−=

−=

−=

(7.8)

⇒ |||| PQIQPI nm −=− (7.7) Example 7.2_______________________________________

Given [ ]21 ppP = and show that |||| PQIQPI nm −=−

[ ]21 ppP = and ⎥⎦

⎤⎢⎣

⎡=

2

1qq

Q ⇒ 2,1 == nm

Hence )(1|| 2211 qpqpPQI m +−=− Also

⎥⎦

⎤⎢⎣

⎡−−−−

=⎥⎦

⎤⎢⎣

⎡−=−

2212

2111

2212

21112 1

1||

pqpqpqpq

pqpqpqpq

IPQI n

Hence 21122211 )1)(1(|| pqpqpqpqPQI n −−−=− )(1 2211 qpqp +−= ________________________________________________________________________________________

Now we consider a multivariable system having n state variables and m inputs. To simplify the mathematical manipulations we shall initially assume that the state matrix is in a diagonal form, that is the state equation has the form

uzz 1B+Λ=& (7.10) where },,,{ 21 ndiag λλλ L=Λ 111 ,, ××× ∈∈∈ mnmn RRRB uz The system open loop characteristic polynomial is

)())((|| 21 nssssI λλλ −−−=Λ− L (7.11) where the system poles are assumed to be distinct. To correspond to equation (7.2) we assume a feedback law of the form

zu 1K= (7.12)

where nmRK ×∈1 is a proportional gain matrix. The closed loop system dynamics

zz )( 11KB+Λ=& (7.13)

The objective of this procedure is to choose 1K so that the closed loop system poles nρρρ ,,, 21 L have prescribed values. These poles are the solution of the closed loop characteristic equation, that is the roots of

0)())((|| 2111 =−−−=+Λ− nsssKBsI ρρρ L (7.14) Select 1K to have the so-called dyadic form

df ′=1K (7.15)

where [ ]′= mfff L21f and [ ]nddd L21=′d with this choice, (7.14) becomes

|)(||||| 11

1 dfdf ′Λ−−Λ−=′+Λ− − BsIIsIBsI Taking the determinant of both sides and using (7.11) and (7.14) we obtain

|)(|)()( 11

11df ′−−−=− −

==BsIIss i

n

ii

n

iλλπρπ (7.17)

Let

[ ]′== nrrrrB L211f (7.19)

111)( ×− ∈Λ−= nRBsIP f

nRQ ×∈′= 1d , It follows that

|)(1||| 11

1 fddf BsIBsI −Λ−′−=′+Λ− (7.18) we then have


___________________________________________________________________________________________________________


38

fd 11)( BsI −Λ−′

[ ]

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

−

−

−

=

n

n

n

r

rr

s

s

s

dddM

L

MOMM

L

L

L 2

1

2

1

21

100

10

001

λ

λ

λ

n

nns

rds

rds

rdλλλ −

++−

+−

= L2

22

1

11

∑=

−=

n

i i

iis

rd

1λ

(7.20)

Substituting Eqs. (7.18) and (7.20) into Eq. (7.17) we obtain

⎟⎟

⎠

⎞

⎜⎜

⎝

⎛

−−−=− ∑

===

n

i i

iii

n

ii

n

i srd

ss1

111)()(

λλπρπ

Hence

∑=

=

==

−=

−

−−− n

i i

ii

in

i

in

ii

n

is

rd

s

ss

11

11

)(

)()(

λλπ

ρπλπ (7.21)

On taking partial fraction

∑=

=

==

−=

−

−−− n

i i

i

in

i

in

ii

n

is

R

s

ss

11

11

)(

)()(

λλπ

ρπλπ (7.22)

where the residues iR at the eigenvalue ),,2,1( njj L=λ can be evaluated as

)(

)()()(lim

1

11

in

i

in

ii

n

ij

sj

s

sssR

j λπ

ρπλπλ

λ−

⎥⎦

⎤⎢⎣

⎡−−−−

=

=

==

→ (7.23)

The procedure for the pole assignment can be summarized as follows: • Calculate the residues ),,2,1( njR j L= from Eq. (7.23)

)(

)()()(lim

1

11

in

i

in

ii

n

ij

sj

s

sssR

j λπ

ρπλπλ

λ−

⎥⎦

⎤⎢⎣

⎡−−−−

=

=

==

→

• Calculate ir after choosing if using Eq. (19)

[ ]′== nrrrrB L211f • Calculate id from the relation between (7.21) and (7.22) jjj Rrd = (7.24)

• Calculate gain 1K from Eq. (7.15) df ′=1K

Example 7.3_______________________________________

Given a system defined by the state equation

u⎥⎦

⎤⎢⎣

⎡+⎥

⎦

⎤⎢⎣

⎡−

−=

12

2001

xx&

find a proportional gain matrix 1K such that the closed loop system poles are both at 3−=λ . In this case 1,2 == mn so that df ′=1K is a one-row matrix, and f is one-element matrix The open loop characteristic polynomial is

23)2)(1())(( 221 ++=++=−− ssssss λλ

The closed loop characteristic polynomial is

96)3())(( 2221 ++=+=−− sssss ρρ

The residuals are

4)2)(1(

)73)(1(lim1

1 −=++−−+

=−→ ss

ssRs

and

1)2)(1(

)73)(2(lim2

2 =++−−+

=−→ ss

ssRs

Hence (Eq. (7.24)): 411 −=dr and 122 =dr From Eq. (7.19)

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡

2

112

rr

f so that ⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡

2

12rr

ff

Choose 1=f , ⎩⎨⎧

==

⇒12

2

1rr

and ⎩⎨⎧

=−=1

2

2

1dd

and [ ]121 −=K .

Check The closed loop characteristic polynomial is

[ ]1212

2001

00

|| 11 −⎥⎦

⎤⎢⎣

⎡−⎥

⎦

⎤⎢⎣

⎡−

−−⎥

⎦

⎤⎢⎣

⎡=−Λ−

ss

KBsI

2)3(

1225

1224

2001

00

+=

+−+

=

⎥⎦

⎤⎢⎣

⎡−−

−⎥⎦

⎤⎢⎣

⎡−

−−⎥

⎦

⎤⎢⎣

⎡=

s

ss

ss

________________________________________________________________________________________ In general case, if the state matrix is not necessarily in a diagonal form. Assume that

uxx BA +=& (7.25) Using the transformation zx T= , Eq. (7.25) becomes


___________________________________________________________________________________________________________


39

uzuzz

1

11

BBTTAT

+Λ=+= −−&

(7.26)

where ATT 1−=Λ and BTB 1

1−= .

Applying the inverse transformation, xz 1−= T to the resulting state equation, we obtain

xxxxx

BKATKTBTT

+=+Λ= −− 1

111&

(7.27)

where

11

−= TKK (7.28) is the proportional gain matrix associated with the state of the system specified by Eq. (7.25). Example 7.4_______________________________________

Given

u⎥⎦

⎤⎢⎣

⎡−

+⎥⎦

⎤⎢⎣

⎡−−

=5

973

104xx&

Find the proportional gain matrix K such that the closed loop poles are both at 3−=λ . On applying the transformation

zzx ⎥⎦

⎤⎢⎣

⎡−−

==31

52T

for which

⎥⎦

⎤⎢⎣

⎡−−

=−

21531T

It is found that the system is the same as the one considered in example 7.3. Hence

[ ]121 −=K and

[ ] [ ]12721

53121

1 −−=⎥⎦

⎤⎢⎣

⎡−−

−== −TKK

so the feedback control

[ ]x127−=u which achieve the desired pole assignment. Check The closed loop system yields

⎥⎦

⎤⎢⎣

⎡ −−=⎥

⎦

⎤⎢⎣

⎡ −−+⎥

⎦

⎤⎢⎣

⎡−−

=+53329859

603510863

73104

BKA

And the closed loop system poles are the roots of the characteristic equation

0965332

9859 2 =++=−−

+ss

ss

as required. ________________________________________________________________________________________ 7.4 Observers In may control system it is not possible to measure entire state vector, although the measurement of some of the state variables is practical. Fortunately, Luenberger has shown that a satisfactory estimate of the state vector can be obtained by using the available system inputs and outputs to derive a second linear system known as an observer. The observer is a dynamic system whose state vector, say )(tz is an approximation to a linear transformation of the state )(tx , that is

)()( tTt xz ≈ (7.29) The entire state vector z is of course, available for feed back. Consider the system

xuxx

cyBA

′=+=&

(7.30)

where nnRA ×∈ , mnRB ×∈ and nRc ×∈′ 1 . To simplify the mathematical manipulation, the system has been assumed to have a single output. But the results are applicable to multi-output systems.

uoutputsystem

uxx BA +=&

observeruhzz NyF ++=&

x y

z

Fig. 7.1 If the dynamics of the observer when regarded as a free system (that is when the input is zero) are defined by zz F=& then, when the inputs are applied, using equation

uhzz NyF ++=& (7.31) where h is an arbitrary vector (of order 1×n ). Using (7.30) and (7.31), we form the difference

uxchxzux-uhzx-z

)()()( TBNFTTATF-TBTANyFT

−++−′+−=++=&&

(7.32)

If we choose

TBN

FTTA=

−=′ch (7.33)


___________________________________________________________________________________________________________


40

Eq. (7.32) becomes

)()( xzxz TFTdtd

−=−

which has the solution

)]0()0([ xzxz TeT Ft −=− or

)]0()0([ xzxz TeT Ft −+= (7.34) If it is possible to chose )0()0( xz T= then xz T= for all

0≥t . Generally it is more realistic to assume that )0(z is only approximately equal to )0(xT , in which case it is

required that Fte approaches the zeros matrix as rapidly as possible. This can be achieved by choosing the eigenvalues of F to have enough large negative real parts. Having obtained an estimate of xT , now we construct a

system having the transfer function 1−T to obtain x . This can be avoided if we choose T to be the identity transformation I . The resulting system is called an identity observer. In this case equations (7.33) becomes

BNFA

=−=′ch (7.35)

From Eq. (7.35) we obtain

ch ′−= AF (7.36) so that the observer eigenvalues are seen to be determined by the choice of h . Example 7.7_______________________________________

Design an identity observer having both eigenvalues at 3−=λ for the system defined by the following state

equations

u⎥⎦

⎤⎢⎣

⎡ −+⎥

⎦

⎤⎢⎣

⎡−

−=

1011

2011

xx&

[ ]x01=y

[ ]x01=y ⇒ only the state variable 1x is available at the output.

[ ]

⎥⎦

⎤⎢⎣

⎡−−

−−=

⎥⎦

⎤⎢⎣

⎡−⎥

⎦

⎤⎢⎣

⎡−

−=

′−=

211

0120

11

2

1

2

1

hh

hh

AF ch

The observer characteristic equation is

0|| =− FIλ

that is

0)22()3( 2112 =+++++ hhh λλ

Since the observer eigenvalues are at 3−=λ , the above equation is identical to

0962 =++ λλ hence

92263

21

1=++=+

hhh

so that

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡=

13

2

1hh

h

It follows that the state matrix of the required observer is

⎥⎦

⎤⎢⎣

⎡−−

−=

2114

F

Check On computing the difference xz && − , we find

)(21

14

1011

2011

1011

0103

2114

xz

ux

uxzxz

−⎥⎦

⎤⎢⎣

⎡−−

−=

⎥⎦

⎤⎢⎣

⎡ −−⎥

⎦

⎤⎢⎣

⎡−

−−

⎥⎦

⎤⎢⎣

⎡ −+⎥

⎦

⎤⎢⎣

⎡+⎥

⎦

⎤⎢⎣

⎡−−

−=− &&

as required. ________________________________________________________________________________________


___________________________________________________________________________________________________________

Chapter 8 Introduction to Optimal Control

41

C.8 Introduction to Optimal Control

8.1 Control and Optimal Control Take an example: the problem of rocket launching a satellite into an orbit about the earth.

• A control problem would be that of choosing the thrust angle and rate of emission of the exhaust gases so that the rocket takes the satellite into its prescribed orbit. • An associated optimal control problem is to choose the controls to affect the transfer with, for example, minimum expenditure of fuel, or in minimum time.

8.2 Examples 8.2.1 Economic Growth 8.2.2 Resource Depletion 8.2.3 Exploited Population 8.2.4 Advertising Policies 8.2.5 Rocket Trajectories The governing equation of rocket motion is

extmtd

dm FF

v+= (8.22)

where m : rocket’s mass v : rocket’s velocity F : thrust produced by the rocket motor

extF : external force It can be seen that

βmc

=F (8.23)

where c : relative exhaust speed

dtdm /−=β : burning rate Let φ be the thrust attitude angle, that is the angle between the rocket axis and the horizontal, as in the Fig. 8.4, then the equations of motion are

gmc

dtdv

mc

dtdv

−=

=

φβ

φβ

sin

cos

2

1

(8.24)

vr

x

v

φ

path of rocket

Fig. 8.4 Rocket Flight Path

Here ),( 21 vv=v and the external force is the gravity force only ),0( mgext −=F . The minimum fuel problem is to choose the controls, β and φ , so as to take the rocket from initial position to a prescribed height, say, y , in such a way as to minimize the fuel used. The fuel consumed is

∫T

odtβ (8.25)

where T is the time at which y is reached. 8.2.6 Servo Problem A control surface on an aircraft is to be kept at rest at a position. Disturbances move the surface and if not corrected it would behave as a damped harmonic oscillator, for example

02 =++ θθθ wa &&& (8.26) where θ is the angle from the desired position (that is, 0=θ ). The disturbance gives initial values 00 , θθθθ ′== & , but a servomechanism applies a restoring torque, so that (8.26) is modified to

uwa =++ θθθ 2&&& (8.27) The problem is to choose )(tu , which will be bounded by

ctu ≤)( , so as to bring the system to 0,0 == θθ & in minimum time. We can write this time as

∫T

dt0

.1 (8.28)

and so this integral is required to be minimized. 8.3 Functionals All the above examples involve finding extremum values of integrals, subject to varying constraints. The integrals are all of the form

∫=1

0

),,(t

tdttxxFJ & (8.29)

where F is a given function of function )(tx , its derivative

)(tx& and the independent variable t . The path )(tx is defined for 10 ttt ≤≤

• given a path )(1 txx = ⇒ give the value 1JJ = • given a path )(2 txx = ⇒ give the value 2JJ =

in general, 21 JJ ≠ , and we call integrals of the form (8.29) functional.


___________________________________________________________________________________________________________

Chapter 8 Introduction to Optimal Control

42

8.4 The Basic Optimal Control Problem Consider the system of the general form ),( t,f uxx =& , where

),,,( 21 ′= nffff L and if the system is linear

uxx BA +=& (8.33) The basic control is to choose the control vector U∈u such that the state vector x is transfer from 0x to a terminal point at time T where some of the state variables are specified. The region U is called the admissible control region. If the transfer can be accomplished, the problem in optimal control is to effect the transfer so that the functional

∫=T

dttfJ0

0 ),,( ux (8.34)

is maximized (or minimized).


___________________________________________________________________________________________________________

Chapter 9 Variational Calculus

43

C.9 Variational Calculus

9.1 The Brachistochrone Problem Consider the Johann Bernoulli problem: Find the trajectory from point A to point B in a vertical plane such that a body slides down the trajectory (under constant gravity and no friction) in minimum time.

vr

x

A y

),( baBa

b

gr

Fig.9.1 The Brachistochrone Problem

Introduce coordinates as in Fig.9.1, so that A is the point

)0,0(A and B is ),( baB . Assuming that the particle is released from rest at A , conservation of energy gives

0221 =− xgmvm (9.1)

The particle speed is 22 yxv && += . From Fig.9.2, we can see that an element of arc length, sδ , is given approximately

by 22 yxs δδδ +≈

yδsδ

xδ

Fig.9.2 Arc Length Element Hence,

22

0lim ⎟

⎠

⎞⎜⎝

⎛+⎟⎠⎞

⎜⎝⎛===

→ dtdy

dtdx

dtds

ts

vt δ

δδ

(9.2)

From (9.1) and (9.2), 21

)2(/ −= gxdsdt so that on integrating, the time of descent is given by

∫∫ +==

adx

xdxdy

ggx

dst0

2)/(1

)2(

1

)2( 21

21 (9.3)

Our problem is to find the path )(xyy = such that ,0)0( =y

bay =)( and which minimizes (9.3). 9.2 Euler Equation

Suppose we are looking for the path )(xyy = between A and B (Fig.9.3) which gives an extremum value for the functional

∫=1

0

),/,(t

tdxxdxdyyFJ (9.5)

x

y

Aα

a b

Bβ

ηεεC

0C

Fig.9.3 Possible paths between A and B

Let 0C be the required curve, say )(0 xy , and let εC be a neighboring curves defined by

)()()( 0 xxyxy ηεε += (9.6) where,

ε : small parameter )(xη : arbitrary differential function of x ,

0)()( == ba ηη . Thus equation (9.6) generates a whole class of neighboring curve εC , depending on the value of ε . The value 0=ε gives the optimal curve 0C . The value of J for the path εC for the general problem is given by

∫ ′+′+=b

adxxyyFJ ),,( 00 ηεηε ,

dxdyy =′ ,

dxdηη =′

For a given function )(xy , )(εJJ = . For extremum values of J , we must have

00=

=εεddJ (9.7)

Taking the differentiation yields

[]dxxyyFy

xyyFddJ

y

b

ay

),,(

),,(

00

00

ηεηε

ηεηεηε

′+′+′

+′+′+=

′

∫

where

dydFFy = ,

yddFFy ′

=′


___________________________________________________________________________________________________________


44

yy FFdyd

yddF

ddy

dydF

ddF

′′+=′

′+= ηη

εεε

ηε+= 0yy ηε ′+′=′ 0yy

Applying condition (9.7) we obtain

[ ] 0),,(),,( 0000 =′′+′ ′∫ dxxyyFxyyF yb

ay ηη

From now on all the partial derivations of F are evaluated on the optimal path 0C , so we will leave off the ),,( 00 xyy ′ dependence. With this notation, on the optimal path,

0)( =′+∫ ′b

ayy dxFF ηη

and the integrating the second term by parts gives

0)( =⎥⎦⎤

⎢⎣⎡ −+ ∫ ′′ dxF

dxdFF

b

a

yyb

ay ηηη

or

0)()()( =⎥⎦⎤

⎢⎣⎡ −+− ∫ ′=′=′ dxF

dxdFFaFb

b

a

yyaxybxy ηηηη

(9.8) But 0)()( == ba ηη , so we are left with

0)( =⎥⎦⎤

⎢⎣⎡ −∫ ′ dxF

dxdF

b

a

yy ηη (9.9)

This must hold for all differentiable functions )(xη such that

0)()( == ba ηη We now require the following Lemma. Lemma Let )(xη be a differentiable function for bxa ≤≤ such that

0)()( == ba ηη . If f is continuous on bxa ≤≤ and

0)()( =∫ dxxxfb

a

η

for all such )(xη , then 0=f on bxa ≤≤ . Applying the result of this Lemma to (9.9), immediately gives

0)( =− ′yy FdxdF (9.10)

on the optimal path. This is known as Euler’s equation (or the Euler-Lagrange equation) and must be satisfied by paths )(xy which yield extremum values of the functional J .

Example 9.1_______________________________________

Find the curve )(xy which gives an extremum value to the functional

∫ +′=1

0

2 )1( dxyJ

with 2)1(,1)0( == yy Here, 12 +′= yF ⇒ 0=yF . Hence the Euler equation (9.10) becomes

0)( =′yFdxd

and on integrating, AFy =′ constant, that is

AyFy =′=′ 2 Integrating again,

BAxy += 2/ From boundary conditions 1,2 == BA , and so the optimal curve is the straight line

1+= xy as illustrated in Fig. 9.5

y

x

2

1

10 Fig. 9.5 Optimal Path

The corresponding extremum value of J is

22)1(1

0

1

0

2 ∫∫ ==+′= dxdxyJ

_________________________________________________________________________________________ Example 9.2_______________________________________

Find the curve )(xy which has minimum length between (0,0) and (1,1) Let 10),( ≤≤= xxyy be any curve between (0,0) and (1,1) . The length, L , is given by


___________________________________________________________________________________________________________


45

∫ ′+=1

0

21 dxyL

The integrand 21 yF ′+= is independent of y , that is, 0=yF . Hence (9.10) becomes

0)( =′yFdxd

and on the integrating AFy =′ , constant. Thus

Ay

y=

′+

′21

which implies that By =′ = constant. Hence CxBy += , where C is a constant. To pass through (0,0) and (1,1), we obtain xy = . ________________________________________________________________________________________ Example 9.3 The Brachristochrone problem_____________

Determine the minimum value of the functional (9.3), that is

∫′+

=a

dxxy

gt

0

2121

such that 0)0( =y and bay =)( .

xyF

21 ′+=

⇒ 0)( ==′ yy FFdxd

⇒ cyy

yFy =′+

′=′

)1( 2, constant.

⇒ xc

xcy 2

2

1−=′

and dxxcx

xcy ∫−

=22

Let define )cos1(2

12 θ−≡

cx ⇒ )2/(sin/ 2cddx θθ = ,

therefore

Ac

y +−= )sin(2

12

θθ , A is a constant

Initial condition: 0)0,0(),( =≡= θyx ⇒ 0=A . Hence

)sin(2

1

)1(2

1

2

2

θθ

θ

−=

−=

cy

xosc

x

(9.11)

This is a question of a cycloid, the constant c being chosen so that the curve passes through the point ),( ba . A typical example of such a curve is shown in Fig. 9.6

)0,0(A

),( baB

x

y Fig. 9.6 (a,b) Cycloid from A and B

⊗ Note that: Our theory has only shown us that we have a stationary value of t on this path. The path found does in fact corresponding to minimum t . ________________________________________________________________________________________ The Euler equation, (9.9), takes special forms when the integrand F is independent of either y or x : 1) F independent of y

From (9.10): 0)( =− ′yy FdxdF ⇒ 0)( =′yF

dxd , hence

=′yF constant (9.12) 2) F independent of x In general

{

yy FyFyxF

dxyd

yF

dxdy

yF

dxdF ′′+′=

∂∂

+′

′∂∂

+∂∂

=

=0

Using (9.10): 0)( =− ′yy FdxdF ⇒ )( yy F

dxdF ′= , give

)()( yyy FydxdFyF

dxdy

dxdF

′′ ′=′′+′= , therefore

yFyF ′′− = constant (9.13) Example 9.4 Minimum Surface Area___________________

A plane curve, )(xyy = passes through the points ),0( b and )(),,( cbca ≠ in the yx − plane ( cba ,, are positive constants)

and the curve lies above the x-axis as shown in Fig. 9.7.

x

y ),( ca

b

c

)(xyy =

a Fig. 9.7 Curve rotates about x-axis


___________________________________________________________________________________________________________


46

Determine the curve )(xy which, when rotated about the x-axis, yields a surface of revolution with minimum surface area. The area of the surface of revolution is given by

∫∫ ⎟⎠⎞

⎜⎝⎛+==

adx

dxdyydsyA

0

2122 ππ

so we wish to find the curve )(xy to minimizes this integral.

The integrand 21 yyF ′+= . This is independent of x so we can apply (9.13)

AFyF y =′− ′ , constant

⇒ Ay

yyyy =′+

′−′+

2

22

11

⇔ )1( 222 yAy ′+=

⇔ 221 AyAdx

dyy −==′

⇔ ∫∫ =−

dxAAy

dy 122

⇔ BAxAy +=− /)/(cosh 1 , B: constant

Hyperbolic cosine of2

coshxx eexx

−+≡=

so that )/cosh( BAxAy += (9.14) The constants are chosen so that the curve passes through (0,b) and (a,c). _________________________________________________________________________________________ 9.3 Free End Conditions This section generalizes the result obtained in section 9.2 for finding extremum values of functionals. Find the path bxaxyy ≤≤= ),( which gives extremum values to the functional

∫ ′=b

adxxyyFJ ),,( (9.15)

with no restriction on the value of y at either end point. The analysis used in 9.2 follows through directly, except that we no longer have the restriction 0)()( == ba ηη . The optimal curve, ∈C , are illustrated in Fig.9.8. Form Eq. (9.8)

0)()()( =⎟⎠⎞

⎜⎝⎛ −+− ∫ ′=′=′ dxF

dxdFFaFb

b

a

yyaxybxy ηηη

(9.16) The free end point also satisfy the Euler equation

x

y

a

A

B

b

0C

Fig. 9.8 Optimal and Neighboring Curves

0)( =− ′yy FdxdF (9.17)

Eq. (9.6) becomes 0)()( =−

=′=′ axybxy FaFb ηη . This result

must be true for all differentiable functions )(xη . In fact for the class of all differentiable functions )(xη with the restriction

0)( =aη which means that

0)( ==′ bxyFbη

Since the value of )(bη is arbitrary, we conclude that

0=′∂

∂yF at bx = (9.18)

Similarly

0=′∂

∂yF at ax = (9.19)

and if only free at one end point, then it can be shown that

0=′∂

∂yF at that end point. We can summarize these results by

0=′∂

∂yF at end point at which y is not specified. (9.20)

Example 9.5 ______________________________________

Find the curve )(xyy = which minimizes the functional

∫ +′=1

0

2 )1( dxyJ

where 1)0( =y and y is free at 1=x . We have already seen, from the example 9.1, that the Euler equation gives

BxAy +=2


___________________________________________________________________________________________________________


47

Apply Eq. (9.20) to give

0=′∂

∂yF at 1=x that is, 02 =′y at 1=x .

The boundary conditions⎩⎨⎧

==

⇒⎩⎨⎧

=′=

10

0)1(1)0(

BA

yy

. Hence the

optimal path is

10,1)( ≤≤= xxy and the optimum value of J is 1=J , which is clearly a minimum. _________________________________________________________________________________________ 9.4 Constrains The results obtained in the last two sections can be further generalized

- firstly to the case where the integrand of more than one independent variable,

- secondly to the case of optimization with constraints. If we wish to find extremum values of the functional

∫=b

ann dttxxxxxxFJ ),,,,,,,,( 2121 &L&&L (9.21)

where, nxxx ,,, 21 L are independent functions of t nxxx &L&& ,,, 21 are differentiation with respect to t We obtain n Euler equations to be satisfied by the optimum path, that is

),,2,1(0 nixF

dtd

xF

iiL

&==⎟⎟

⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂ (9.22)

Also at any end point where ix is free

0=∂∂

ixF&

(9.23)

Example 9.6 ______________________________________

Find the extremals of the functional

∫ ++=2/

021

22

21 )2(

πdtxxxxJ &&

subject to boundary conditions 0)0(1 =x , 1)2/(1 =πx ,

1)2/(2 =πx . Using Eq. (9.22) to give

⎜⎜⎜⎜

⎝

⎛

=+

=+⇒

⎪⎪

⎩

⎪⎪

⎨

⎧

=⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂

=⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂

0)2(2

0)2(2

0

0

21

12

22

11

xdtdx

xdtdx

xF

dtd

xF

xF

dtd

xF

&

&

&

&

Hence

Hence 012 =+ xx && and 021 =+ xx && . Eliminating 2x gives

⎪⎩

⎪⎨⎧

−−+=

+++=⇒=−

−

−

tDtCBeAex

tDtCBeAexx

dtxd

tt

tt

cossin

cossin0

2

114

14

Applying the boundary conditions gives

⎪⎪⎪

⎩

⎪⎪⎪

⎨

⎧

==−=−

=−

−

00

1

2/

DC

ABe

eAπ

π

,

Hence the optimum path is

2/01

2/21 π

ππ ≤≤

−

−==

−

−− t

eeeexx

tt

_________________________________________________________________________________________ In many optimization problems, we have constraints on the relevant variables. We first deal with extremum values of a function, say ),( 21 xxf subject to a constraint of the form

0),( 21 =xxg . Assuming that it is not possible to eliminate one of 1x or 2x using the constraint, we introduce a Lagrange multiplier, λ , and form the augmented function

),(),(* 2121 xxgxxff λ+= (9.24) We now look for extremum values of *f , that is

0**

21=

∂∂

=∂∂

xf

xf (9.25)

Solving these equations, 21, xx will be functions of the parameter λ , that is, )(),( 21 λλ xx . And appropriate value of λ is found by satisfying 0))(),(( 21 =λλ xxg . Since

ff ≡* , we have the condition for the extremum values for f . Example 9.7 ______________________________________

Find the extremum value of

22

2121 ),( xxxxf +=

subject to 43 21 =+ xx . We first note that we can easily eliminate 1x from the constraint equation to give

22

22 )34( xxf +−=

and we can then solve the problem by evaluating 0/ 2 =dxdf . Using a Lagrange multiplier, we consider the augmented function

)43(* 2122

21 −+++= xxxxf λ


___________________________________________________________________________________________________________


48

For extremum value of *f ,

⎩⎨⎧

=+=+

⇒

⎪⎪⎩

⎪⎪⎨

⎧

=∂∂

=∂∂

03202

0*

0*

2

1

2

1λλ

xx

xfxf

Hence 2/3,2/ 21 λλ −== xx and to satisfy the constraint

5/44)]2/3(3[)2/( −=⇒=−+− λλλ and the extremum value of f occurs at

5/6,5/2 21 == xx _________________________________________________________________________________________ We can now return to the main problem: Suppose we wish for extremum values of (9.21)

∫ ∫==b

a

b

ann dttFdttxxxxxxFJ ),,(),,,,,,,,( 2121 xx &&L&&L

(9.21) subject to a constraint between txxxxxx nn ,,,,,,,, 2121 &L&&L which we write as

0),,( =tg xx & (9.26) then we introduce a Lagrange multiplier, λ , and form the augmented functional

∫ +=b

adtgFJ )(* λ (9.27)

We now find the extremum values of *J in the usual way, regarding the variables as independent. The Lagrange multiplier can then be eliminated to yield the required optimum path. Example 9.8 ______________________________________

Minimize the cost functional

dxx

zy

gJ

a

∫+

=0

1

21 &&

where the variables )(),( xzzxyy == are subject to the constraint

1+= zy and where bayy == )(,0)0( Introducing the Lagrange multiplier λ , the augmented functional is

∫ ⎥⎥⎦

⎤

⎢⎢⎣

⎡−−+

+=

b

adxzy

xzy

gJ )1(

121* λ

&&

Euler equation yields

0)1(22

: =⎟⎟

⎠

⎞

⎜⎜

⎝

⎛

+−

zyxgz

dxdy

&&

&λ

0)1(22

: =⎟⎟

⎠

⎞

⎜⎜

⎝

⎛

+−−

zyxgy

dxdz

&&

&λ

Eliminating λ , we obtain

0)1(22

=⎟⎟

⎠

⎞

⎜⎜

⎝

⎛

+

+

zyxgzy

dxd

&&

&&

⇒ Azyx

zy=

+

+

)1( &&

&&

using the constraint 1−= yz , we obtain for the optimum path

2)1( 2

A

yx

y=

+ &

&

_________________________________________________________________________________________ Example 9.9 ______________________________________

Minimize the cost functional

dtxJ ∫=2

0

221

&&

where 0)2(,0)2(;1)0(,1)0( ==== xxxx && To use Lagrange multiplier, we introduce the state variable

xxxx &== 21 , . The functional is now

∫=2

0

221 dtxJ &

and we have the constraint

012 =− xx & The augmented functional is

dtxxxJ ∫ ⎟⎠⎞

⎜⎝⎛ −+=

2

012

22 )(

21* && λ

Euler equation yields

0)(:

0)(0:

2

1

=−

=−−

xdtdx

dtdx

&λ

λ ⇒

00

2 =−=

x&&

&

λλ

Eliminating λ , we obtain 032

3=

dt

xd, and

CtBtAx

DtCtBtAx

++=

+++=

22

23

1 23


___________________________________________________________________________________________________________


49

Applying the end conditions implies

127

23

147

21

22

231

++=

+++=

ttx

tttx

⇒ Azyx

zy=

+

+

)1( &&

&&

and we obtain for the optimum path

4/13)2/73(21 2

0

2 =−= ∫ dttJ _________________________________________________________________________________________


___________________________________________________________________________________________________________

Chapter 10 Optimal Control with Unbounded Continuous Controls

50

C.10 Optimal Control with Unbounded Continuous Controls

10.1 Introduction The methods appropriate for variational calculus can readily extended to optimal control problems, where the control vector u is unbounded. Technique for dealing with the general case U∈u , a bounded admissible control region, will be discussed in the next chapter. We start with a simple example. Suppose we wish to find extremum values of the functional

∫ +=T

odtuxJ )( 22 (10.1)

where x : the state variable u : the control variable satisfy the differential equation

uxx =+& (10.2) with the boundary conditions 0)(,)0( 0 == Txxx . A direct method of solving this problem would be to eliminate u from (10.2) and (10.1), which reduces it to a straightforward calculus of variations problem. In general, it will not able to eliminate the control, so we will develop another method based on the Lagrange multiplier technique for dealing with constraints. We introduce the Lagrange multiplier, λ , and from the augmented functional

∫ −+++=T

dtuxxuxJ0

22 )]([* &λ (10.3)

The integrand )(22 uxxuxF −+++= &λ is a function of two variables x and u. We can evaluate Euler’s equation for x and u, that is

0=⎟⎠⎞

⎜⎝⎛∂∂

−∂∂

xF

dtd

xF

& ⇒ 02 =−+ λλ &x (10.4)

0=⎟⎠⎞

⎜⎝⎛∂∂

−∂∂

uF

dtd

uF

& ⇒ 02 =− λu (10.5)

From (10.2), (10.4) and (10.5), we obtain

02 =− xx&&

⇒ tt BeAex 22 −+= or, alternatively TbTax 2sinh2cosh += .With the boundary conditions, we obtain

T

tTxx

2sinh

)(2sinh0 −=

and the corresponding optimal control is recovered from (10.2), giving

TtTtTxu

2sinh)(2cosh2)(2sinh

0−−−

=

10.2 The Hamiltonian Consider the general problem of minimizing

∫=T

odttuxfJ ),,(0 (10.6)

subject to the differential equation

),,( tuxfx =& (10.7) and boundary condition ax =)0( . With Lagrange multiplier, λ , we form the augmented functional

∫ −+=T

odtxtuxftuxfJ ]}),,([),,({* 0 &λ

The integrand ]),,([),,(0 xtuxftuxfF &−+= λ is a function of the two variables x and u, and so we have two Euler equations

⎪⎪⎩

⎪⎪⎨

⎧

=∂∂

+∂∂

=+∂∂

+∂∂

⇒

⎪⎪⎩

⎪⎪⎨

⎧

=⎟⎠⎞

⎜⎝⎛∂∂

−∂∂

=⎟⎠⎞

⎜⎝⎛∂∂

−∂∂

0

0

0

0

0

0

uf

uf

xf

xf

uF

dtd

uF

xF

dtd

xF

λ

λλ &

&

& (10.8-9)

Now defining the Hamintonian

ffH λ+= 0 (10.10) (10.8) and (10.9) can be rewritten as

xH∂∂

−=λ& (10.11)

uH∂∂

=0 (10.12)

These are the equations, together with (10.2), which govern the optimal paths.

Example 10.1______________________________________

Using Hamintonian method, find the optimal control u which yields a stationary value for the functional

∫ +=1

0

22 )( dtuxJ where ux =&

with 1)0( =x and not fixed at 1=t .


___________________________________________________________________________________________________________


51

Here ufuxf =+= ,220 and the Hamintonian is

uuxH λ++= 22

(10.11): xH∂∂

−=λ& ⇒ x2−=λ&

(10.12): uH∂∂

=0 ⇒ 02 =+ λu

Hence we obtain 0=− xx&& with the solution is

tbtax coshsinh += From the boundary conditions

1cosh)1cosh( tx −

= and 1cosh

)1sinh( tu −−=

_________________________________________________________________________________________ 10.3 Extension to Higher Order Systems The method described in the last section can be readily extended to higher order systems. For example, we wish to minimize

∫∞

+=0

22 )(21 dtuxJ α (10.13)

where,

uxx +−= &&& , α is constant (10.14) and initially ax = , bx =& , and as ∞→t , both x and 0→x& . Following the usual technique, we define xxxx &≡≡ 21 , so that we have two constraints 012 =− xx & (10.15)

022 =−+− xux & (10.16) The augmented function is

∫∞

⎥⎦⎤

⎢⎣⎡ −+−+−++=

0222121

221 )()()(

21* dtxuxxxuxJ && λλα

We now have three Euler equations, namely,

011

=⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂

xF

dtd

xF

& ⇒ 011 =+ λ&x (10.17)

022

=⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂

xF

dtd

xF

& ⇒ 0221 =+− λλλ & (10.18)

0=⎟⎠⎞

⎜⎝⎛∂∂

−∂∂

uF

dtd

uF

& ⇒ 02 =+ λα u (10.19)

Eliminating iλ , we obtain

0121

2

41

4=+−

αx

dtxd

dtxd

(10.20)

with 4=α , (10.20) has solutions of the form mtex =1 , where m satisfies

021

21 2

224 =⎟⎠⎞

⎜⎝⎛ −=+− mmm

with the boundary conditions, we get

2/1 ])2/([ tetabax −++= (10.23)

2/2 ]2/)2/([ tetabbx −+−= (10.24)

2/]2/)2/11)(2/()12(2/[ tetabbau −−+−−−−= (10.25) General Problem The basic problem is to find the optimal controls u which yield extremal values of

∫=T

dttfJ0

0 ),,( ux (10.27)

subject to the constraints

),,2,1(),,( nitfx ii L& == ux (10.28)

where, [ ]Tnxxxx L21= and [ ]Tnuuuu L21= We introduce the Lagrange multipliers, ),,2,1( nipi L= usually called the adjoint variables and form the augmented functional

dtxfpfJT n

iiii∫ ∑ ⎥⎥⎦

⎤

⎢⎢⎣

⎡−+=

=0 1

0 )(* &

We also define the Hamintonian, H, is

∑=

+=n

iii fpfH

10 (10.29)

so that

dtxpHJT n

iii∫ ∑ ⎥⎥⎦

⎤

⎢⎢⎣

⎡−=

=0

1

* &

The integrand, ∑=

−=n

iii xpHF

1

& depends on x, u, t, and we

form (n+m) Euler equations, namely

),,2,1(0 nixF

dtd

xF

iiL

&==⎟⎟

⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂ that is,

ii x

Hp∂∂

−=&

(10.30)

which are known as the adjoint equations; and

),,2,1(0 mjuF

dtd

uF

jjL

&==⎟

⎟⎠

⎞⎜⎜⎝

⎛

∂∂

−∂∂ that is, 0=

∂∂

juH

(10.31)


___________________________________________________________________________________________________________


52

The optimal solutions are determined from (10.28,30,31). Using boundary conditions given in section 8.4, we have

)0(ix given ),,2,1( ni L= and ),,2,1(),( qlTxl L= are given. The remaining values )(,),(1 TxTx nq L+ are free, and so we must apply the free end condition (9.23):

0=∂∂

ixF&

),,1( nqk L+= at Tt =

This gives

),,1(0)( nqkTpk L+== (10.32) which is known as the transversality condition. In general

0)( =Tpi for any ix not specified at Tt = . Example 10.2______________________________________

Find the control u which minimizes

∫=1

0

2dtuJ where axux +=& , a is constant, and

(i) 1)0( =x (ii) 1)0( =x , 0)1( =x The Hamintonian is )(2 axupuH ++= where p is the adjoint variable. (10.31): 02 =+ pu (10.30): app −=&

⇒ ⎪⎩

⎪⎨

⎧

−=

=

−

−

at

at

eAu

Aep

2

⇒ ateAaxx −−=−2

&

and atat eaABex −+=

4, the constants A and B will be

calculated from boundary conditions. (i). 1)0( =x and )1(x is not specified Here 14/ =+ aAB and since )1(x is not specified, we

apply the transversality condition (10.32) at 1=t , which is 0)1( =p , that is 0=A . The optimal control

10,0 ≤≤= tu . This gives 0=J , which is clearly a minimum as 0≥J .

(ii) 1)0( =x , 0)1( =x Now, we have 14/ =+ aAB

04/ =+ − aAeBe aa Hence, )1/(4 2aeaA −−= and the optimal control is 10),1/(2 2 ≤≤−−= −− teaeu aat _________________________________________________________________________________________

Example 10.3______________________________________

Find the optimal control u which gives an extremum value of the functional

∫=1

0

2dtuJ where uxxx == 221 , &&

with 1)0(1 =x , 0)1(1 =x , 1)0(2 =x and )1(2x is not specified. The Hamintonian is

upxpuH 2212 ++=

(10.31): 220 puuH

+==∂∂ ⇒ 22

1 pu −=

(10.30): App

p−=−=

=

12

1 0&

& ⇒

BAtpAp

+−==

2

1

Hence 22BtAu −=

222BtAx −=& ⇒

⎪⎪⎩

⎪⎪⎨

⎧

++−=

+−=

DCttBtAx

CtBtAx

231

22

412

24

Boundary conditions give 1,1,12,12 ==== DCBA . The optimal control is

10),1(6 ≤≤−= ttu _________________________________________________________________________________________

Introduction to Control Theory Including Optimal Control Nguyen Tan Tien - 2002.5 ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Chapter 11 Bang-bang Control 53

C.11 Bang-bang Control 11.1 Introduction This chapter deals with the control with restrictions: is bounded and might well be possible to have discontinuities. To illustrate some of the basic concepts involved when controls are bounded and allowed to have discontinuities we start with a simple physical problem: Derive a controller such that a car move a distance a with minimum time. The motion equation of the car

udt

xd=

2

2 (11.1)

where

βα ≤≤−= utuu ),( (11.2) represents the applied acceleration or deceleration (braking) and x the distance traveled. The problem can be stated as minimize

∫=T

odtT 1 (11.3)

subject to (10.11) and (10.12) and boundary conditions

0)(,)(,0)0(,0)0( ==== TxaTxxx && (11.4) The methods we developed in the last chapter would be appropriate for this problem except that they cannot cope with inequality constraints of the form (11.2). We can change this constraint into an equality constraint by introducing another control variable, v, where

))((2 uuv −+= βα (11.5) Since v is real, u must satisfy (11.2). We introduce th usual state variable notation xx =1 so that

aTxxxx === )(,0)0( 2121& (11.6) 0)(,0)0( 222 === Txxux& (11.7)

We now form the augmented functional

∫ +−+−+=T

vxupxxpT0

222121 [)()(1{* µ&&

dtuu )]})(( −+− βα (11.8) where η,, 21 pp are Lagrange multipliers associated with the constraints (11.6), (11.7) and (11.5) respectively. The Euler equations for the state variables 21, xx and control variables u and v are

011

=⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂

xF

dtd

xF

& ⇒ 01 =p& (11.9)

022

=⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

−∂∂

xF

dtd

xF

& ⇒ 12 pp −=& (11.10)

0=⎟⎠⎞

⎜⎝⎛∂∂

−∂∂

uF

dtd

uF

& ⇒ )2(2 up −−= αβµ (11.11)

0=⎟⎠⎞

⎜⎝⎛∂∂

−∂∂

vF

dtd

vF

& 02 =µv (11.12)

(11.12) ⇒ 0=v or 0=µ . We will consider these two cases. (i) 0=µ

⎩⎨⎧

==

⇒00

2

1

pp

⇒ be impossible.

(ii) 0=v

(11.5): ))((2 uuv −+= βα ⇒ α−=u or β=u Hence

⎩⎨⎧

≤<−≤≤

=Tt

tx

τατβ 0

2&

the switch taking place at time τ . Integrating using

boundary conditions on 2x

⎩⎨⎧

≤<−−≤≤

==TtTt

ttxx

τατβ

)(0

12 & (11.13)

Integrating using boundary conditions on 1x

⎪⎪⎩

⎪⎪⎨

⎧

≤<+−−

≤≤=

TtaTt

ttx

τα

τβ

2

2

1)(

21

021

(11.14)

Both distance, 1x , and velocity, 2x , are continuous at

τ=t , we must have (11.14) ⇒ )( T−−= ταβτ

(11.15) ⇒ 22 )(21

21 Ta −−= ταβτ

Eliminating T gives the switching time as

)(

2βαβ

ατ+

=a (11.15)

and the final time is

αβ

βα )(2 +=

aT (11.16)

The problem now is completely solved and the optimal control is specified by



⎩⎨⎧

≤<−≤≤

=Tt

tu

τατβ 0

(11.17)

this is illustrated in Fig. 11.1

0

α−

βu control

τ T t time

Fig. 11.1 Optimal Control

This figure shows that the control: - has a switch (discontinuity) at time τ=t - only take its maximum and minimum values This type of control is called bang-bang control. 11.2 Pontryagin’s Principle (early 1960s) Problem: We are seeking extremum values of the functional

dttfJT

∫= 00 ),,( ux (11.18)

subject to state equations

),,2,1(),,( nitfx ii L& == ux (11.19) initial conditions 0xx = and final conditions on qxxx ,,, 21 L

)( nq ≤ and subject to U∈u , the admissible control region. For example, in the previous problem, the admissible control region is defined by

}:{ βα ≤≤−= uuU As in section 10.4, we form the augmented functional

dtxfpfJT n

iiii∫ ∑ ⎥⎥⎦

⎤

⎢⎢⎣

⎡−+=

=0 1

0 )(* & (11.20)

and define the Hamintonian

∑=

+=n

iii fpfH

10 (11.21)

For simplicity, we consider that the Hamintonian is a function of the state vector x, control vector u, and adjoint vector p, that is, ),,( puxHH = . We can express *J as

dtxpHJT n

iii∫ ∑ ⎟⎟

⎠

⎞

⎜⎜

⎝

⎛−=

=0

1

* & (11.22)

and evaluating the Euler equations for ix , we obtain as in section 10.4 the adjoint equations

),,2,1( nixHp

ii L& =

∂∂

−= (11.23)

The Euler equations for the control variables, iu , do not follow as in section 10.4 as it is possible that there are discontinuities in iu , and so we cannot assume that the partial derivaties iuH ∂∂ / exist. On the other hand, we can apply the

free end point condition (9.23): 0=∂∂

ixF&

to obtain

0)( =Tpk nqk ,,1L+= (11.24) that is, the adjoint variable is zero at every end point where the corresponding state variable is not specified. As before, we refer to (11.24) as tranversality conditions. Our difficulty now lies in obtaining the analogous equation to

0/ =∂∂ iuH for continuous controls. For the moment, let us assume that we can differentiate H with respect to u, and consider a small variation uδ in the control u such that uu δ+ still belong to U , the admissible control region. Corresponding to the small change in u , there will be small change in x , say xδ , and in p , say pδ . The change in the value of *J will be *Jδ , where

dtxpHJT

o

n

iii∫ ∑

=

−= }{*1

&δδ

The small change operator, δ , obeys the same sort of properties as the differential operator dxd / . Assuming we can interchange the small change operator,δ , and integral sign, we obtain

∫ ∑∑

∫ ∑

∫ ∑

==

=

=

−−=

−=

−=

T n

iii

n

iii

T n

iii

T n

iii

dtxppxH

dtxpH

dtxpHJ

011

01

01

][

)]([

)]([*

&&

&

&

δδδ

δδ

δδ

Using chain rule for partial differentiation

∑∑∑===

∂∂

+∂∂

+∂∂

=n

ii

i

n

ii

i

m

jj

jp

pHx

xHu

uHH

111

δδδδ

so that

dtxppxxpppH

uuHJ

n

iiiiiiii

i

T m

jj

j

⎥⎥⎦

⎤⎟⎟⎠

⎞⎜⎜⎝

⎛−−−

∂∂

+

⎢⎢

⎣

⎡

∂∂

=

∑

∫ ∑

=

=

1

01

*

&& δδδδ

δδ

since ii xHp ∂−∂= /& . Also, from (11.21) iii

xfpH

&==∂∂



using (11.19): ),,2,1(),,,( nitfx ii L& == ux . Thus

( )

( )∫ ∑∑

∫ ∑∑

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡−

∂∂

=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡+−

∂∂

=

==

==

T n

iii

m

jj

j

T n

iiiii

m

jj

j

dtxpdtdu

uH

dtxpxpuuHJ

0 11

0 11

*

δδ

δδδδ &&

We can now integrate the second part of the integrand to yield

( ) ∫ ∑∑ ⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

∂∂

+−===

T m

jj

j

Tn

iii dtu

uHxpJ

0 101

* δδδ (11.25)

At 0=t : ),,1( nixi L= are specified ⇒ 0)0( =ixδ At Tt = : ),,1( qixi L= are fixed ⇒ 0)( =Txiδ For ,,,1 nqi L+= from the transversality conditions, (11,24)

0)()( =TxTp ii δ for nqqi ,,1,,,2,1 LL += . We now have

∫ ∑ ⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

∂∂

==

T m

jj

jdtu

uHJ

0 1

* δδ

where juδ is the small variation in the thj component of the control vector u . Since all these variations are independent, and we require 0* =Jδ for a turning point when the controls are continuous, we conclude that

),,2,1(0 mjuH

jL==

∂∂ (11.26)

But this is only valid when the controls are continuous and not constrained. In our present case when U∈u , the admissible control region and discontinuities in u are allowed. The arguments presented above follow through in the same way, except that jj duuH )/( ∂∂ must be replaced by

),,();,,,,,;( 21 puxpx HuuuuuH mjj −+ LL δ We thus obtain

∫ ∑=

−+=T m

jmjj dtHuuuuHJ

0 11 )],,();,,,,;([* puxpx LL δδ

In order for u to be a minimizing control, we must have 0* ≥Jδ for all admissible controls uu δ+ . This implies that

),,();,,,,;( 1 puxpx HuuuuH mjj ≥+ LL δ (11.27) for all admissible juδ and for mj ,,1L= . So we have established that on the optimal control H is minimized with respect to the control variables, muuu ,,, 21 L . This is known as Pontryagin’s minimum principle.

We first illustrate its use by examining a simple problem. We required to minimize

∫=T

dtJ01

subject to uxxx == 221 , && where βα ≤≤− u and aTx =)(1 ,

0)(2 =Tx , 0)0()0( 21 == xx . Introducing adjoint variables 1p and 2p , the Hamiltonian is given by

upxpH 2211 ++= We must minimize H with respect to u, and where

],[ βα−=∈Uu , the admissible control region.. Since H is linear in u, it clearly attains its minimum on the boundary of the control region, that is, either at α−=u or β=u . This illustrated in Fig.11.3. In fact we can write the optimal control as

⎩⎨⎧

<>−

=00

2

2pifpif

uβα

α− β

u control

H

Fig. 11.3 The case 02 >p

But 2p will vary in time, and satisfies the adjoint equations,

12

2

11 0

pxHp

xHp

−=∂∂

−=

=∂∂

−=

&

&

Thus Ap =1 , a constant, and BAtp +−=2 , where B is constant. Since 2p is a linear function of t, there will at most be one switch in the control, since 2p has at most one zero, and from the physical situation there must be at least one switch. So we conclude that (i) the control α−=u or β=u , that is, bang-bang control; (ii) there is one and only one switch in the control. Again, it is clear from the basic problem that initially β=u , followed by α−=u at the appropriate time. 11.3 Switching Curves In the last section we met the idea of a switch in a control. The time (and position) of switching from one extremum value of the control to another does of course depend on the initial starting point in the phase plane. Byconsidering a specific example we shall show how these switching positions define a switching curve.



Suppose a system is described by the state variables 21, xx where

uxx +−= 11& (11.28) ux =2& (11.29)

Here u is the control variable which is subject to the constraints 11 ≤≤− u . Given that at bxaxt === 21 ,,0 , we wish to find the optimal control which takes the system to

0,0 21 == xx in minimum time; that is, we wish to minimize

∫=T

dtJ01 (11.30)

while moving from ),( ba to )0,0( in the 21 xx − phase plane and subject to (11.28), (11.29) and

11 ≤≤− u (11.31) Following the procedure outline in section 11.2, we introduce the adjoint variables 1p and 2p and the Hamiltonian

1121

2111)()(1

xpppuupuxpH

−++=++−+=

Since H is linear in u, and 1|| ≤u , H is minimized with respect to u by taking

⎩⎨⎧

>+−<++

=0101

21

21ppifppif

u

So the control is bang-bang and the number of switches will depend on the sign changes in 21 pp + . As the adjoint equations, (11.23), are

02

2

11

1

=∂∂

−=

=∂∂

−=

xHp

pxHp

&

&

⇒ BpAep t

==

2

1 , A and B are constant

and BAepp t +=+ 21 , and this function has at most one sign change. So we know that from any initial point ),( ba , the optimal control will be bang-bang, that is, 1±=u , with at most one switch in the control. Now suppose ku = , when 1±=k , then the state equations for the system are

kxkxx

=+−=

2

11&

&

We can integrate each equation to give

BtkxkAex t

+=+= −

2

1 , A and B are constants

The 21 xx − plane trajectories are found by eliminating t , giving

|/)(|log 12 AkxkBx −−=− Now if 1=u , that is, 1=k , then the trajectories are of the form

|/)1(|log 12 AxBx −−=− that is

Cxx +−−= |1|log 12 (11.32) where C is constant. The curves for different values of C are illustrated in Fig.11.4

1x

2x1=u

0

1

A

Fig. 11.4 Trajectories for 1=u Follow the same procedure for 1−=u , giving

Cxx +−= |1|log 12 (11.33) and the curves are illustrated in Fig. 11.5.

01x

2x

1−

1−=u

B

Fig. 11.4 Trajectories for 1−=u



The basic problem is to reach the origin from an arbitrary initial point. All the possible trajectories are illustrated in Figs. 11.4 and 11.5, and we can see that these trajectories are only two possible paths which reach the origin, namely AO in Fig. 11.4 and BO in Fig. 11.5.

A

B

1x

2x

0

1=u

1−=u

1− 1

Fig. 11.4 Trajectories for 1−=u Combining the two diagrams we develop the Fig. 11.6. The curve AOB is called switching curve. For initial points below AOB, we take 1+=u until the switching curve is reached, followed by 1−=u until the origin is reached. Similarly for the points above AOB, 1−=u until the switching curve is reached, followed by 1+=u until the origin is reached. So we have solved the problem of finding the optimal trajectory from an arbitrary starting point. Thus the switching curve has equation

⎩⎨⎧

<+−>+

=0)1log(0)1log(

11

112 xforx

xforxx (11.34)

11.4 Transversarlity conditions To illustrate how the transversality conditions ( 0)( =Tpi if

ix is not specified) are used, we consider the problem of finding the optimum control u when 1|| ≤u for the system described by

21 xx =& (11.35) ux =2& (11.36)

which takes the system from an arbitrary initial point

bxax == )0(,)0( 21 to any point on the 2x axis, that is, 0)(1 =Tx but )(2 Tx is not given, and minimize

∫=T

dtJ0

.1 (11.37)

subject to (11.35), (11.36), the above boundary conditions on 1x and 2x , and such that

11 ≤≤− u (11.38)

Following the usual procedure, we form the Hamiltonian

upxpH 2211 ++= (11.39) We minimize H with respect to u, where 11 ≤≤− u , which gives

⎩⎨⎧

>−<+

=0101

2

2pifpif

u

The adjoint variables satisfy

⎪⎪⎩

⎪⎪⎨

⎧

−=∂∂

−=

=∂∂

−=

12

2

11 0

pxHp

xHp

&

&

⇒⎩⎨⎧

+−==

BAtpAp

2

1

Since )(2 Tx is not specified, the transversality condition becomes 0)(2 =Tp . Hence BAt +−=0 and )(2 tTAp −= . For Tt <<0 , there is no change in the sign of 2p , and hence no switch in u. Thus either 1+=u or 1−=u , but with no switches. We have

)(2 122 Bxx −= when 1=u (11.40)

)(2 122 Bxx −−= when 1−=u (11.41)

These trajectories are illustrated in Fig. 11.7, the direction of the arrows being determined from ux =2&

1x

2x1+=u

0

1−=u2x

1x0

Fig. 11.7 Possible Optimal Trajectories

1x2x

0

),( ba

+T

−TA

{{

),( ba

Fig. 11.8 Initial point below OA

We first consider initial points ),( ba for which 0>a . For points above the curve OA, there is only one trajectory,



1−=u which reaches the 2x -axis, and this must be the optimal curve. For points below the curve OA, there are two possible curves, as shown in Fig. 11.8. From (11.36): ux =2& , that is 12 ±=x& , and the integrating between 0 and T gives

TxTx ±=− )0()( 22 that is

|)0()(| 22 xTxT −= (11.42) Hence the modulus of the difference in final and initial values of 2x given the time taken. This is shown in the diagram as

+T for 1+=u and −T for 1−=u . The complete set of optimal trajectories is illustrated in Fig. 11.9.

1x

2x

0

1−=u1+=u

Fig. 11.9 Optimal Trajectories to reach 2x -axis in minimum

time 11.5 Extension to the Boltza problem


Chapter 12 Applications of Optimal Control 59

C.12 Applications of Optimal Control 12.6 Rocket Trajectories The governing equation of rocket motion is

extmtd

dm FF

v+= (8.22)

where

m : rocket’s mass v : rocket’s velocity F : thrust produced by the rocket motor

extF : external force It can be seen that

βmc

=F (8.23)

where

c : relative exhaust speed dtdm /−=β : burning rate

Let φ be the thrust attitude angle, that is the angle between the rocket axis and the horizontal, as in the Fig. 8.4, then the equations of motion are

gmc

dtdv

mc

dtdv

−=

=

φβ

φβ

sin

cos

2

1

(8.24)

Here ),( 21 vv=v and the external force is the gravity force only ),0( mgext −=F .

vr

x

y

φ

path of rocket

Fig. 8.4 Rocket Flight Path

The minimum fuel problem is to choose the controls, β and φ , so as to take the rocket from initial position to a prescribed height, say, y , in such a way as to minimize the fuel used. The fuel consumed is

∫T

odtβ (8.25)

where T is the time at which y is reached.

Problem Minimizing the fuel (cost function)

∫=T

odtJ β (12.41)

subject to the differential constraints

φβ cos1mc

dtdv

= (12.42)

gmc

dtdv

−= φβ sin2 (12.43)

and also

1vdtdx

= (12.44)

2vdtdy

= (12.45)

The boundary conditions 0=t : 0,0,0,0 21 ==== vvyx Tt = : x not specified, 0,, 21 === vvvyy Thus we have four state variables, namely 21,,, vvyx and two controls β (the rate at which the exhaust gases are emitted) and φ (the thrust attitude angle). In practice we must have bounds on β , that is,

ββ ≤≤0 (12.46) so that 0=β corresponds to the rocket motor being shut

down and ββ = corresponds to the motor at full power. The Hamilton for the system is

⎟⎠

⎞⎜⎝

⎛ −++++= gmcp

mcpvpvpH φβφββ sincos 432211

(12.47) where 4321 ,,, pppp are the adjoint variables associated with 21,,, vvyx respectively. (12.47)⇒

⎟⎠⎞

⎜⎝⎛ +++−+= φφβ sincos1 4342211 m

cpmcpgpvpvpH

If we assuming that 0sincos1 43 ≠⎟⎠⎞

⎜⎝⎛ ++ φφ

mcp

mcp , we

see that H is linear in the control β , so that β is bang-bang.

That is 0=β or ββ = . We must clearly start with ββ = so that, with one switch in β , we have


Chapter 12 Applications of Optimal Control 59

⎪⎩

⎪⎨⎧

≤<<≤

=Tttforttfor

1

10

0ββ (12.48)

Now H must also be maximized with respect to the second control,φ , that is, 0/ =∂∂ φH giving

0cossin 43 =+− φβφβmcp

mcp

which yields 34 /tan pp=φ . The adjoint variables satisfy the equations

01 =∂∂

−=xHp& ⇒ Ap =1

01 =∂∂

−=yHp& ⇒ Bp =2

ApvHp −=−=∂∂

−= 11

1& ⇒ AtCp −=3

22

1 pvHp −=

∂∂

−=& ⇒ BtDp −=4

where, A, B, C, D are constant. Thus

AtCBtD

−−

=φtan

Since x is not specified at Tt = , the transversality condition is 0)(1 =Tp , that is, 0=A , and so

bta −=φtan (12.49) where CBbCDa /,/ == . The problem, in principle, I now solved. To complete it requires just integration and algebraic manipulation. Withφ given by (12.49) and β given by (12.48), we can integrate (12.42) to (12.45). This will bring four further constants of integration; together with a, b and the switchover time t1 we have seven unknown constants. These are determined from the seven end-point conditions at 0=t and Tt = . A typical trajectory is shown in Fig. 12.5.

vr

x

y

1tt =ββ =

0=β

maximum thrust arc

null thrust arc

yy =

0

Fig. 12.5 Typical rocket trajectory 12.7 Servo Problem The problem here is to minimize

∫=T

dtT01 (12.50)

subject to

udtd

dtd

=++ θωθαθ 22

2 (12.51)

Boundary conditions 0=t : 00 , θθθθ ′== &

Tt = : 0,0 == θθ & Constraint on control: 1|| ≤u . Introduce the state variables: θθ &== 21 , xx . Then (12.51) becomes

uxxax

xx

+−=

=

12

22

21

ω&

& (12.52)

As usual, we form the Hamiltonian

)(1 12

2221 uxxapypH +−−++= ω (12.53) where 21, pp are adjoint variables satisfying

22

1 pxHp ω=∂∂

−=& (12.54)

211 appyHp +−=∂∂

−=& (12.55)

Since H is linear in u, and 1|| ≤u , we again immediately see that the control u is bang-bang, that is, 1±=u , in fact

⎩⎨⎧

>−<+

=0101

2

2pifpif

u

To ascertain the number of switches, we must solve for 2p . From (12.54) and (12.55), we see that

222

212 pappapp &&&&& +−=+−= ω that is,

022

22 =+− ppap ω&&& (12.56) The solution of (12.56) gives us the switching time.

introduction to control theory including optimal control

Documents