16.513 control systems, lecture note #4 -...

1

1

Last Time: Modeling of Selected Systems (§2.5, §2.6):

– Electrical circuits, Mechanical systems, simple financial systems

Linear Algebra,

– Linear spaces over a field, subspace

– Linear dependence and Linear independence

16.513 Control Systems, Lecture Note #4

2

Linear Independence

A set of vectors {x1, x2, .., xm} in Rn is LD if {1, 2, .., m} in R, not all zero, s.t.

1x1 + 2x2 + .. + m xm = 0 (*) If the only set of {i}i=1 to m s.t. the above holds is

1 = 2 = .. = m = 0then {xi}i=1 to m is said to be LI

Given {x1, x2, .., xm}, form 1 2 mA x x ... x

If A=0 has a unique solution, LI;If A=0 has nonunique solution, LD.

If rank(A)=m, the solution is unique LIIf rank(A)<m, the solution is not unique LD.

2

3

Today:

The base of a linear space: Basis– Representations of a vector in terms of a basis

– Relationship among representations for different bases

Generalization of the idea of length: Norms

A sense of orientation: Inner Product– The concept of perpendicularity: Orthogonality

– Gram-Schmidt Process to obtain orthonormal vectors

Linear Operators and Representations

4

Basis and Representations• Basis: The basic elements from which everything can be

constructed.

• A set of vectors {e1, e2, .., en} of Rn is said to be a basis of Rn

if every vector in Rn can be uniquely expressed as a linear combination of them

– They span Rn

– For any x Rn, there exist {1, 2, .., n} s.t.

n

1iiinn2211 ee..eex

n

2

1

n21 :e...eex

n21 e...eex

– : Representation of x with respect to the basis

– x is uniquely identified with

~

3

5

Y is a subset of Rn (Y Rn), a subspace, to be precise . If {e1, e2, .., en} is a basis, we need to have Rn Y. To ensurethis,

11

1

21 2 1

Let : , ,

... : , ,:

n

i i i ni

n i n

n

Y e R

e e e R

– For every x Rn, there exist {1, 2, .., n} s.t.

n

1iiinn2211 ee..eex

Now given a set of vectors, e1, e2, …, en. What is the conditionfor the set to be a basis for Rn?

6

A particular basis -- The orthonormal basis:

,

1

0

0

0

e,

0

1

0

0

e,,

0

0

1

0

e,

0

0

0

1

e n1n21

For all xRn, we have

nn2211n21

n

2

1

exexex

1

0

0

x

0

1

0

x

0

0

1

x

x

x

x

x

And the representation is unique.

4

7

Q. What else qualifies to be a basis?

Theorem: In an n-dimensional vector space, any set of n LI vectors qualifies as a basis

Proof:– Let {e1, e2, .., en} be linearly independent

– For any x Rn, {x, e1, e2, .., en} are linearly dependent

– {0, 1, 2, .., n}, not all zero, such that

0e...eex nn22110

1 1 2 210

1... ,

n

n n i ii

x e e e e

– 0 0. Otherwise, {e1, e2, .., en} are not linearly independent. Thus

– Any x can be expressed as a linear combination of them

8

• What can be said now?

0e~n

1iiii

– The linear combination is unique

– {e1, e2, .., en} is a basis, and the proof is completed

iallfor~

or0~

iiii

• Is the combination unique here?– Suppose that another linear combination

n

1iiie

~x

n

1iiie

~ since {e1, e2, .., en} are LI

5

9

Another explanation: Let {e1, e2, .., en} be linearly independent. Form a square matrix A=[e1 e2 .. en]. Then A is nonsingular,i.e., det A 0

Theorem: In an n-dimensional vector space, any set of nLI vectors qualifies as a basis

Consider the equation

A x

There exists a unique solution =A-1x. For any x Rn, there exists a unique such that

1 1 2 2 n ne e e x

{e1, e2, .., en} is a basis.

xeee nn ....2211

10

Change of Basis• Any set of n LI vectors qualify as a basis

;e...ee n21 n21 e...ee

– For a particular x, the representation is unique for each basis

;eβx;eβxn

1iii

n

1iii

Example: Consider R2 ,

1

0e,

0

1e 21

1

1e,

1

1e 21

Given , what are

2

1x ?β and β

2

1β ,

2

1ee2eex 2121

– Given , how to find?

6

11

;eβx;eβxn

1iii

n

1iii

nj

2j

1j

n21

n

1iiijj

p

:

p

p

e...eeepe

Or the other way,

nj

2j

1j

n21

n

1i

iijj

q:

qq

e...eeeqe

Problem: Given , find ; or the otherwise

We need to express one basis in terms of the other

12

1 2 n 1 2 n 1 2 n

1 2 n

e e ... e e e ... e p p ... p

e e ... e P

P ~ nn

nj

2j

1j

n21

n

1iiijj

p

:

p

p

e...eeepe

pj

jn21 pe...ee

Pβe...eeβe...eex n21n21

n

1iiieβxGiven

βe...ee n21

Pββ A new representation:

n

n

pppE

EpEpEp

21

21

... Recall

7

13

βQβ

– Conversely, we can express in terms of

– What is the relationship between P and Q?

,IPQPQP 1QPor

;e...ee n21

Qe...ee

q...qqe...eee...ee

n21

n21n21n21

βQe...eeβe...eex n21n21

je

14

;e...ee n21

n21 e...ee

In summary: Suppose we have an old basis:

And a new basis

n

i i 1 2 ni 1

Suppose x β e = e e e β;

n

i ii 1

How to find such that x β e ?

Express the old basis in terms of the new basis

1j

2jj 1 2 n 1 2 n j

nj

ppe e e ... e e e ... e p for each j:

p

Form P = [p1 p2 … pn]. Then Pββ

8

15

Express the new basis in terms of the old basis

1j

2jj 1 2 n 1 2 n j

nj

qqe e e ... e e e ... e q for each j:

q

Form Q = [q1 q2 … qn]. Then -1β Q β

1 2 n 1 2 nAn old base: e e ... e ; A new base e e ... e

n

i i 1 2 ni 1

Suppose x β e = e e e β;

Alternative approach:

n

i ii 1

is the representaion in terms of the new basis: x β e .

16

Example: Consider R2 ,

1

0e,

0

1e 21

1

1eee,

1

1eee 212211

11

11ee)ee( 2121 ,

11

11Q

11

11

2

1P

2

1β ,

2

1ee2eex 2121

• Given , what are

2

1x ?β and β

1.5

0.5

2

1

11

11

2

1Pββ

• Verify: 1 21 1 1e e β ( 0.5) 1.5 x1 1 2

√

1

1e,

1

1e 21

9

17

1

0e,

0

1e 21Example. Consider R2 with

1 1 2e = cos ψ e + sin ψ e 1 2 1cosψ= e e ~ qsinψ

212 ecosesine 221 q~cos

sinee

-1cos ψ -sin ψ cos ψ sin ψQ= , P=Q =sin ψ cos ψ -sin ψ cos ψ

QandP

– For = 45 and

0

2

1

1

21

21

21

21

e1

e2 x_

e1e2

_

e1

_e2

_

e1

e2

12 ex

? is what ,1

1

1

121

eex

18

In summary:

1 2 n Given a basis e e ... e ;

1 2 n 1 2 n Let the new basis be e e ... e e e ... e Q

1 2 n For x such that x e e ... e β

11 2

-1nWe have x e e ... e Q β β=Q β-

-11 2 n 1 2 nThen e e ... e e e ... e Q

1 2 n For x such that x e e ... e β

1 2 nWe have x e e ... e Qβ Qβ β=

10

19

For a block diagonal matrix

B

CAS 0

BCAS 0

1

1111

0 BCBAAS

111

11 0

BCABAS

Exercise: Compute the inverse for

1

1 0 01 1 0 , 0 1 1

S

Inverse for block diagonal matrices:

20

Exercise:

The new basis is 1 1 2 2 2 3 3 3e e + e ; e e + e ; e e

1) For x =e1+e2+e3, find a,b,c such that1 2 3x=ae +be +ce ,

What is the representation of the new basis in terms of the old one?

1 2 3 1 2 3 1 2 3

1 0 0 1 0 0e e e = e e e 1 1 0 = e e e Q, Q= 1 1 0

0 1 1 0 1 1

1 2 3

1 0 0The old basis: e 0 , e 1 ,e 0

0 0 1

-11 2 3 1 2 3

1 0 0 1 0 0P=Q 1 1 0 , e e e = e e e -1 1 0

1 1 1 1 -1 1

1 2 3 1 2 3 1 2 3

1 1 1x= e e e 1 = e e e P 1 = e e e 0

1 1 1

abc

The representation of the old basis in terms of the new basis:

11

21

Exercise:

The new basis is 1 1 2 2 2 3 3 3e e + e ; e e + e ; e e

1 2 3

1 0 0The old basis: e 0 , e 1 ,e 0

0 0 1

1 2 3 1 2 3 1 2 3

1 1 1x= e e e -1 = e e e -1 = e e e 0

1 1 0Q

abc

1 2 3 1 2 32) Given x=e e +e , what are a,b,c such that x=ae be +ce ?

1 2 3 1 2 3 1 2 3

1 0 0 1 0 0e e e = e e e 1 1 0 = e e e Q, Q= 1 1 0

0 1 1 0 1 1

Need the representation of the new basis in terms of the old one:

22

Today:







12

23

Norms • Want to add more structures to a linear space

• Norms: Generalization of the idea of length

x1

x x2

– Key points?

– From here, we can define distance, convergence, derivative, etc.

Norm. ||x||: Rn R (or Cn R) such that– ||x|| 0 and ||x|| = 0 iff x = 0

– ||x|| = || ||x|| for all R (or C)

– ||x1 + x2|| ||x1|| + ||x2|| ~ Triangular inequality

~ (x12+x2

2)1/2

24

• Some norms for Rn or Cn

– ||x||1– ||x||2– ||x||p– ||x||

|xi|, 1-norm

[|xi|2]1/2, 2-norm, Euclidean norm, spectral norm

[|xi|p]1/p, p-norm

maxi |xi|, -norm

• Do they satisfy the conditions in the definition?

• Find the various norms for x = (2, -3, 1)T

– ||x||1– ||x||2– ||x||3– ||x||

|xi| = 2 + 3 + 1 = 6

[|xi|2]1/2 = (4 + 9 + 1)1/2 = 141/2 = 3.74

[|xi|3]1/3 = (8 + 27 + 1)1/3 = 361/3 = 3.302

maxi |xi| = 3

13

25

-4 -2 0 2 4-4

-2

0

2

4

-4 -2 0 2 4-4

-2

0

2

4

-4 -2 0 2 4-4

-2

0

2

4

-4 -2 0 2 4-4

-2

0

2

4

Geometric interpretation of norms: Level sets

3,2,11x 3,2,1

2x

3,2,1

x3,2,13x

3x:2 Rx

2x:2 Rx

1x:2 Rx

Large:

Medium:

Small:

14

27

Inner Product

• More structure: A sense of orientation

x

yz

x’y = (1 0) (4 3)T = 4 ~ Projection of y onto x

x’z= (1 0) (0 2)T = 0, as x and z are "orthogonal"

2

0z,

3

4y,

0

1x

The inner product between two vectors x and y is denoted as x, y

For xRn, x, y = x’y. Given x,y,zRn,

What are x, y and x, z?

28

Example. x = (1 j, 2)T, y = (3, 1 + 2j)T

x*y = ? x*x = ?

nn

1iii

* Cforyxyxyx,

*: Complex conjugate transpose

Q. Why defined in this way?

– Want x, x R and x, x > 0 when x 0

– The 2-norm is induced by the inner product, i.e.,

||x||2= x, x1/2

7j52j1

32j1yxyx

n

1iii

*

62

j12j1xxxx

n

1iii

*

~ Always real,

non negative

For xCn, the inner product is

15

29

Some properties of inner product: Consider x,yCn

x,yy,x

y,xαy,xαy,xαxα 22112211

22112211 y x,αy x,αyαyαx,

Bar ~ Complex conjugate

yx,A* Ayx,AyxyxA ***

• We also have

2121yy,xx,yx,

• The Cauchy-Schwarz Inequality

Or, 22

* yx|yx|

30

2121 y,yx,xy,x

Proof. If y = 0, the above is obviously satisfied. Now assume y 0. Then for an arbitrary scalar C,

yx,yx0 y,yx,yy,xx,x

yy

yxSet

yy

xy

yy

yxthen

yy

yx

yy

xy

yy

yxx,x0and

222

2yxyyx,x0or

2121 y,yx,xy,xor

16

31

• The concept of perpendicularity and unit size

• In Rn or Cn, x and y are orthogonal (written as x y) iff x,y= 0; x is normalized if x,x=1. (i.e., ||x||2=1)– {x1, x2, .., xm} is an orthogonal set iff xi xj i j

Orthogonality and normality

– x is orthogonal to a set S if x s s S

2

0

2

x,

0

3

0

x,

1

0

1

x 321

Rba,,

b

b

a

S,

1

1

0

x

32

Can we have m>n? No. Because the dimension is n, andAn orthogonal set of nonzero vectors is a linearly

independent set. How to show this?

Consider an orthogonal set of nonzero vectors {x1,x2,…,xm} Rn? What can we say about it?

17

33

• Suppose that {x1, x2, .., xm} is an orthogonal set of nonzero vectors and iixi = 0, we can show that

i must be zero for all i=1,2,…m. How?

<xk, iixi> = <xk, 0> = 0

= ii <xk, xi> = k <xk, xk>

k = 0 k {x1, x2, .., xm} are linearly independent

– If m = n, then {x1, x2, .., xn} is good candidate for a basis in view of its orthogonality

34

What can be said about an orthonormal set of nonzero vectors {x1,x2,…,xm} Rn? Let X=[x1 x2 … xm], what is X’X?

mm2m1m

m22212

m12111

m21

m

2

1

x'xx'xx'x

x'xx'xx'x

x'xx'xx'x

xxx

'x

'x

'x

XX'

{x1,x2,…,xm} is an orthonormal set if it is orthogonal and xi,xi=||xi||22 =1 for each i.

mI

100

010

001

x1

x2

x3

An even better choice for a basis

18

35

Example: θany for ,cosθ

sinθ,

sinθ

cosθ)x,(x 21

2Icosθsinθ

sinθcosθ

cosθsinθ

sinθcosθXX'

How about XX’? Do we have XX’=I? Need to be careful about this. If m=n, then from

X’X=I, we have X=(X’)1. Hence XX’=I. If m<n, then XX’ I , e.g.,

I

100

000

001

100

001

10

00

01

XX',

10

00

01

X

I10

01

10

00

01

100

001XX'

36

Let {x1,x2,…,xn} be an orthonormal set in Rn.• They are linearly independent to each other;• ||xk||2 = <x,x> =x’x=1;• Every yRn can be uniquely expressed as

1 1 2 2 n n

k k k

y=a x +a x + +a x

where a x , , the projection of y to x .y

19

37

Today:







38

Consider Rn, – Given a set of LI vectors {ei}i=1 to m, ei Rn.

– Form the set of linear combinations

Rα :eαY i

m

1iii

Then Y is a linear space, and is a subspace of Rn.

It is the space spanned by {ei}i=1 to m

The dimension of Y is m

A basis of Y is (e1, e2, …, em)

Subspace spanned by a set of vectors

How can we obtain an orthonormal basis of Y?

20

39

Orthonormal basis for a subspace

Given a basis (e1,e2,…em) for a subspace Y. Suppose that the vectors are not orthogonal to each other. How to get an equivalent basis (ê1,ê2,…,êm)

which also span Y and in addition ||êi||2=1 for all i, êi êj for i j? (*)

Basic requirement, êi has to be a linear combination of (e1,e2,…em), i.e.,

êi = pi1e1+pi2e2+…+pimem

(ê1,ê2,…,êm) = (e1,e2,…em) P

Problem: find pij, i,j =1,2,…m such that êi’s satisfy (*)

40

Gram-Schmidt Process

– Example: Consider R2

x1

x2

e1 = (2, 1) T

e2 = (1, 2) T

111

2122 v

vv

evev

v2

Take v1 = e1

– The component of e2 that is to v1:

2.1

6.0

1

2

14

22

2

1

The projection of e2 onto e1

21

41

• {v1 , v2} is an orthogonal set:

<v1 , v2> = -1.2 + 1.2 = 0– What is next?

– Normalization. How?

x1

x2

515

2

vv

e1

11

5251

vv

e2

22 e2

_e1_

42

• General Procedure:– Let {e1, e2, .., en} be a linearly independent set

– Form an orthogonal set by subtracting from each vector the components that align with previous vectors

11 ev 111

2122 v

vv

evev

222

321

11

3133 v

vv

evv

vv

evev

1n

1kk

kk

nknn v

vv

evev

– Normalize the new set

ivv

ei

ii

22

43

– Relation between vectors: LD/LI, inner product

– Length: norms, normalized vectors

– Representation: basis, orthonormalizations

• So far, we have addressed a few issues on linearvector spaces:

Next we will discuss operations between vector spaces linear operations and matrices

44

Linear Operations

– Range: {y Y | x X, s.t. f(x) = y} Y

• Functions– A function f is a mapping from

domain X to codomain Y that assigns

each x X one and only one element of Y

• What is a "linear function"?

x y

fX Y

– A function L that maps from X to Y is said to be a linear operator (linear function, linear mapping, or linear transformation) iff

L(1x1 + 2x2) = 1L(x1) + 2L(x2)

1, 2 R (or C ), and x1, x2 X

23

45

Linear Operations Associated with Matrices

• Consider X=Rn and Y=Rm

• Given an mn real matrix S;• For xRn, define

y=L(x)=Sx– You can show that L is a linear map

• Thus we can use a matrix to define a linear map.• What if we define y=f(x)=Sx+c for a nozero cRm?

– Of course not a linear map

• More to say? Is every linear map from Rm to Rn

associated with a matrix? • Yes !

x y

LX=Rn Y=Rm

46

Matrix Representation of Linear Operators

• Still consider X=Rn to Y=Rm. • A linear operator is uniquely

determined by how the basis are mapped

Theorem. Suppose that– {e1, e2, .., en} is a basis of Rn

– Then L: X Y is uniquely determined by n pairs of mapping

ei → yi , i = 1, 2, .., n

x y

LRn Rm

24

47

• Let the basis of X be {e1, e2, .., en} • Let the basis of Y be {w1, w2, .., wm}• Suppose that

mi

2i

1i

iim21ii

s

ss

s,swwwy)L(ee

i

• Then by linearity, for any xX, with representation i.e., x=i=1

nieiwe have

n

1i

n

1iiim21ii sαwwwyαL(x)yx

11 12 1n 1

n21 22 2n 2

1 2 m i i 1 2 mi 1

m1 m2 mn n

s s s α

s s s αw w w α s w w w

s s s α

• Representation of y: S

The linear operator is determined by how the basis are mapped

S

48

Example. Rotating counter-clock-wise in R2 by

How to proceed?

1

0e,

0

1e 21

1

0w,

0

1w 21

– What is y1? What is y2?

sin

coswwwsinwcosy 21211

cos

sinwwwcoswsiny 21212

s1

s2

cossinsincosS

– If x=ee2, e.g., = (1, 1)T, and = 90, then y=w1+w2, with

1

111

0110Sαβ e1

e2 x y

e1→ y1, e2→ y2

Le1Le2

e1

e2y2= = y1

25

49

1.5,2121S2,11

21S1.5,2131S 321

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

-5 0 5-5

0

5

-5 0 5-5

0

5

-5 0 5-5

0

5

-5 0 5-5

0

5

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

-20 0 20-20

0

20

Given 3 matrices:

Subsets in R2

{y=S1x: xDi}

D1 D2 D3 D4

{y=S2x: xDi}

{y=S3x: xDi}

50

Change of Basis– L: x y ~ The mapping is independent of bases

– = A ~ The ith column of A is the representation of

Lei (= yi) w.r.t. {w1, w2, .., wm}• The representation A depends on the bases for X and Y

– Consider a special case where L: Rn Rn ( or Cn to Cn)

– With {e1, e2, .., en} as a basis, under operator L

x → y: [e1 e2 .. en] → [e1 e2 …en] A– For simplicity, we denote L: =A– Suppose the basis is changed to {ê1, ê2,…,ên} for X,Y.

– What would be the new rep. of L?

– Suppose the new rep. is Ā. How are A and Ā related?

26

51

– Still consider the map

x → y: [e1 e2 .. en] → [e1 e2 …en] A

Under the new basis, we have

x → y: [ê1 ê2 .. ên] P → [ê1 ê2 …ên] PA

Let the new basis be

[ê1 ê2…ên]= [e1 e2 .. en] Q

Then equivalently,

[ê1 ê2…ên] P= [e1 e2 .. en], where P=Q-1

If we let a=Pb=PAthen a → b=PAP-1a

The new rep. for the operator is Ā=PAP-1= Q-1AQ

52

1PAPA A

11 QAQPAPAor

– This is called the similar transformation, and A and are similar matricesA

27

53

1PAPAwith,xAx

Example. Consider

32

10A,Axx

20

01

21

11

32

10

11

12PAPA 1

x2001x

What is the dynamics for a new representation with

x1 112Pxx

A much simpler structure

Later on we will discuss how to find such a P matrix

The diagonal elements capture the maincharacteristic of the system

54

Today:







28

55

• Next Time: 3.3 to 3.6– Norm of a Linear Operator– Systems of linear algebraic equations– Orthogonal complement– Similarity transformation: the Companion form– Eigenvalues and Eigenvectors– Diagonal form and Jordan form– Functions of a square matrix

56

Homework Set #4

Problem 1: Given a basis for R3,1 0 0

2 , 1 , 03 2 1

1 1Express x= 1 and y= 0 in terms of the basis.

1 1

Problem 2: Let the old basis be

100

e,010

e,001

e 321

The new basis is 1 1 2 1 2 3 1 2 3e e ; e 2e +e ; e e -2e +e

1) For 1 2 3x 2e 3e e , find a,b,c such that x=ae1+be2+ce3

2) For x=e1-e2+e3, find such that ,eγeβeαx 321

29

57

Problem 3:

1) Use Gram-Schmidt procedure to perform orthonomalization on the basis

1 2

1 10 , 11 1

e e

2) Express the new basis in terms of the old basis3) What is the representation of x=e1+e2 in terms

of the new basis?

Problem 4:0 0 1

x Ax, A 0.5 0 0.50 2 1

1) What is the dynamics for a new representation with

0.5 0 0.5x Px, P 0 1 0

0.5 0 0.5

Consider

Problem 5 : (optional) Reproduce the plots on slide 49.

16.513 control systems, lecture note #4 -...

Documents