16.513 control systems, lecture note #4 -...
Post on 03-Jul-2020
0 Views
Preview:
TRANSCRIPT
1
1
Last Time: Modeling of Selected Systems (§2.5, §2.6):
– Electrical circuits, Mechanical systems, simple financial systems
Linear Algebra,
– Linear spaces over a field, subspace
– Linear dependence and Linear independence
16.513 Control Systems, Lecture Note #4
2
Linear Independence
A set of vectors {x1, x2, .., xm} in Rn is LD if {1, 2, .., m} in R, not all zero, s.t.
1x1 + 2x2 + .. + m xm = 0 (*) If the only set of {i}i=1 to m s.t. the above holds is
1 = 2 = .. = m = 0then {xi}i=1 to m is said to be LI
Given {x1, x2, .., xm}, form 1 2 mA x x ... x
If A=0 has a unique solution, LI;If A=0 has nonunique solution, LD.
If rank(A)=m, the solution is unique LIIf rank(A)<m, the solution is not unique LD.
2
3
Today:
The base of a linear space: Basis– Representations of a vector in terms of a basis
– Relationship among representations for different bases
Generalization of the idea of length: Norms
A sense of orientation: Inner Product– The concept of perpendicularity: Orthogonality
– Gram-Schmidt Process to obtain orthonormal vectors
Linear Operators and Representations
4
Basis and Representations• Basis: The basic elements from which everything can be
constructed.
• A set of vectors {e1, e2, .., en} of Rn is said to be a basis of Rn
if every vector in Rn can be uniquely expressed as a linear combination of them
– They span Rn
– For any x Rn, there exist {1, 2, .., n} s.t.
n
1iiinn2211 ee..eex
n
2
1
n21 :e...eex
n21 e...eex
– : Representation of x with respect to the basis
– x is uniquely identified with
~
3
5
Y is a subset of Rn (Y Rn), a subspace, to be precise . If {e1, e2, .., en} is a basis, we need to have Rn Y. To ensurethis,
11
1
21 2 1
Let : , ,
... : , ,:
n
i i i ni
n i n
n
Y e R
e e e R
– For every x Rn, there exist {1, 2, .., n} s.t.
n
1iiinn2211 ee..eex
Now given a set of vectors, e1, e2, …, en. What is the conditionfor the set to be a basis for Rn?
6
A particular basis -- The orthonormal basis:
,
1
0
0
0
e,
0
1
0
0
e,,
0
0
1
0
e,
0
0
0
1
e n1n21
For all xRn, we have
nn2211n21
n
2
1
exexex
1
0
0
x
0
1
0
x
0
0
1
x
x
x
x
x
And the representation is unique.
4
7
Q. What else qualifies to be a basis?
Theorem: In an n-dimensional vector space, any set of n LI vectors qualifies as a basis
Proof:– Let {e1, e2, .., en} be linearly independent
– For any x Rn, {x, e1, e2, .., en} are linearly dependent
– {0, 1, 2, .., n}, not all zero, such that
0e...eex nn22110
1 1 2 210
1... ,
n
n n i ii
x e e e e
– 0 0. Otherwise, {e1, e2, .., en} are not linearly independent. Thus
– Any x can be expressed as a linear combination of them
8
• What can be said now?
0e~n
1iiii
– The linear combination is unique
– {e1, e2, .., en} is a basis, and the proof is completed
iallfor~
or0~
iiii
• Is the combination unique here?– Suppose that another linear combination
n
1iiie
~x
n
1iiie
~ since {e1, e2, .., en} are LI
5
9
Another explanation: Let {e1, e2, .., en} be linearly independent. Form a square matrix A=[e1 e2 .. en]. Then A is nonsingular,i.e., det A 0
Theorem: In an n-dimensional vector space, any set of nLI vectors qualifies as a basis
Consider the equation
A x
There exists a unique solution =A-1x. For any x Rn, there exists a unique such that
1 1 2 2 n ne e e x
{e1, e2, .., en} is a basis.
xeee nn ....2211
10
Change of Basis• Any set of n LI vectors qualify as a basis
;e...ee n21 n21 e...ee
– For a particular x, the representation is unique for each basis
;eβx;eβxn
1iii
n
1iii
Example: Consider R2 ,
1
0e,
0
1e 21
1
1e,
1
1e 21
Given , what are
2
1x ?β and β
2
1β ,
2
1ee2eex 2121
– Given , how to find?
6
11
;eβx;eβxn
1iii
n
1iii
nj
2j
1j
n21
n
1iiijj
p
:
p
p
e...eeepe
Or the other way,
nj
2j
1j
n21
n
1i
iijj
q:
e...eeeqe
Problem: Given , find ; or the otherwise
We need to express one basis in terms of the other
12
1 2 n 1 2 n 1 2 n
1 2 n
e e ... e e e ... e p p ... p
e e ... e P
P ~ nn
nj
2j
1j
n21
n
1iiijj
p
:
p
p
e...eeepe
pj
jn21 pe...ee
Pβe...eeβe...eex n21n21
n
1iiieβxGiven
βe...ee n21
Pββ A new representation:
n
n
pppE
EpEpEp
21
21
... Recall
7
13
βQβ
– Conversely, we can express in terms of
– What is the relationship between P and Q?
,IPQPQP 1QPor
;e...ee n21
Qe...ee
q...qqe...eee...ee
n21
n21n21n21
βQe...eeβe...eex n21n21
je
14
;e...ee n21
n21 e...ee
In summary: Suppose we have an old basis:
And a new basis
n
i i 1 2 ni 1
Suppose x β e = e e e β;
n
i ii 1
How to find such that x β e ?
Express the old basis in terms of the new basis
1j
2jj 1 2 n 1 2 n j
nj
ppe e e ... e e e ... e p for each j:
p
Form P = [p1 p2 … pn]. Then Pββ
8
15
Express the new basis in terms of the old basis
1j
2jj 1 2 n 1 2 n j
nj
qqe e e ... e e e ... e q for each j:
q
Form Q = [q1 q2 … qn]. Then -1β Q β
1 2 n 1 2 nAn old base: e e ... e ; A new base e e ... e
n
i i 1 2 ni 1
Suppose x β e = e e e β;
Alternative approach:
n
i ii 1
is the representaion in terms of the new basis: x β e .
16
Example: Consider R2 ,
1
0e,
0
1e 21
1
1eee,
1
1eee 212211
11
11ee)ee( 2121 ,
11
11Q
11
11
2
1P
2
1β ,
2
1ee2eex 2121
• Given , what are
2
1x ?β and β
1.5
0.5
2
1
11
11
2
1Pββ
• Verify: 1 21 1 1e e β ( 0.5) 1.5 x1 1 2
√
1
1e,
1
1e 21
9
17
1
0e,
0
1e 21Example. Consider R2 with
1 1 2e = cos ψ e + sin ψ e 1 2 1cosψ= e e ~ qsinψ
212 ecosesine 221 q~cos
sinee
-1cos ψ -sin ψ cos ψ sin ψQ= , P=Q =sin ψ cos ψ -sin ψ cos ψ
QandP
– For = 45 and
0
2
1
1
21
21
21
21
e1
e2 x_
e1e2
_
e1
_e2
_
e1
e2
12 ex
? is what ,1
1
1
121
eex
18
In summary:
1 2 n Given a basis e e ... e ;
1 2 n 1 2 n Let the new basis be e e ... e e e ... e Q
1 2 n For x such that x e e ... e β
11 2
-1nWe have x e e ... e Q β β=Q β-
-11 2 n 1 2 nThen e e ... e e e ... e Q
1 2 n For x such that x e e ... e β
1 2 nWe have x e e ... e Qβ Qβ β=
10
19
For a block diagonal matrix
B
CAS 0
BCAS 0
1
1111
0 BCBAAS
111
11 0
BCABAS
Exercise: Compute the inverse for
1
1 0 01 1 0 , 0 1 1
S
Inverse for block diagonal matrices:
20
Exercise:
The new basis is 1 1 2 2 2 3 3 3e e + e ; e e + e ; e e
1) For x =e1+e2+e3, find a,b,c such that1 2 3x=ae +be +ce ,
What is the representation of the new basis in terms of the old one?
1 2 3 1 2 3 1 2 3
1 0 0 1 0 0e e e = e e e 1 1 0 = e e e Q, Q= 1 1 0
0 1 1 0 1 1
1 2 3
1 0 0The old basis: e 0 , e 1 ,e 0
0 0 1
-11 2 3 1 2 3
1 0 0 1 0 0P=Q 1 1 0 , e e e = e e e -1 1 0
1 1 1 1 -1 1
1 2 3 1 2 3 1 2 3
1 1 1x= e e e 1 = e e e P 1 = e e e 0
1 1 1
abc
The representation of the old basis in terms of the new basis:
11
21
Exercise:
The new basis is 1 1 2 2 2 3 3 3e e + e ; e e + e ; e e
1 2 3
1 0 0The old basis: e 0 , e 1 ,e 0
0 0 1
1 2 3 1 2 3 1 2 3
1 1 1x= e e e -1 = e e e -1 = e e e 0
1 1 0Q
abc
1 2 3 1 2 32) Given x=e e +e , what are a,b,c such that x=ae be +ce ?
1 2 3 1 2 3 1 2 3
1 0 0 1 0 0e e e = e e e 1 1 0 = e e e Q, Q= 1 1 0
0 1 1 0 1 1
Need the representation of the new basis in terms of the old one:
22
Today:
The base of a linear space: Basis– Representations of a vector in terms of a basis
– Relationship among representations for different bases
Generalization of the idea of length: Norms
A sense of orientation: Inner Product– The concept of perpendicularity: Orthogonality
– Gram-Schmidt Process to obtain orthonormal vectors
Linear Operators and Representations
12
23
Norms • Want to add more structures to a linear space
• Norms: Generalization of the idea of length
x1
x x2
– Key points?
– From here, we can define distance, convergence, derivative, etc.
Norm. ||x||: Rn R (or Cn R) such that– ||x|| 0 and ||x|| = 0 iff x = 0
– ||x|| = || ||x|| for all R (or C)
– ||x1 + x2|| ||x1|| + ||x2|| ~ Triangular inequality
~ (x12+x2
2)1/2
24
• Some norms for Rn or Cn
– ||x||1– ||x||2– ||x||p– ||x||
|xi|, 1-norm
[|xi|2]1/2, 2-norm, Euclidean norm, spectral norm
[|xi|p]1/p, p-norm
maxi |xi|, -norm
• Do they satisfy the conditions in the definition?
• Find the various norms for x = (2, -3, 1)T
– ||x||1– ||x||2– ||x||3– ||x||
|xi| = 2 + 3 + 1 = 6
[|xi|2]1/2 = (4 + 9 + 1)1/2 = 141/2 = 3.74
[|xi|3]1/3 = (8 + 27 + 1)1/3 = 361/3 = 3.302
maxi |xi| = 3
13
25
-4 -2 0 2 4-4
-2
0
2
4
-4 -2 0 2 4-4
-2
0
2
4
-4 -2 0 2 4-4
-2
0
2
4
-4 -2 0 2 4-4
-2
0
2
4
Geometric interpretation of norms: Level sets
3,2,11x 3,2,1
2x
3,2,1
x3,2,13x
3x:2 Rx
2x:2 Rx
1x:2 Rx
Large:
Medium:
Small:
14
27
Inner Product
• More structure: A sense of orientation
x
yz
x’y = (1 0) (4 3)T = 4 ~ Projection of y onto x
x’z= (1 0) (0 2)T = 0, as x and z are "orthogonal"
2
0z,
3
4y,
0
1x
The inner product between two vectors x and y is denoted as x, y
For xRn, x, y = x’y. Given x,y,zRn,
What are x, y and x, z?
28
Example. x = (1 j, 2)T, y = (3, 1 + 2j)T
x*y = ? x*x = ?
nn
1iii
* Cforyxyxyx,
*: Complex conjugate transpose
Q. Why defined in this way?
– Want x, x R and x, x > 0 when x 0
– The 2-norm is induced by the inner product, i.e.,
||x||2= x, x1/2
7j52j1
32j1yxyx
n
1iii
*
62
j12j1xxxx
n
1iii
*
~ Always real,
non negative
For xCn, the inner product is
15
29
Some properties of inner product: Consider x,yCn
x,yy,x
y,xαy,xαy,xαxα 22112211
22112211 y x,αy x,αyαyαx,
Bar ~ Complex conjugate
yx,A* Ayx,AyxyxA ***
• We also have
2121yy,xx,yx,
• The Cauchy-Schwarz Inequality
Or, 22
* yx|yx|
30
2121 y,yx,xy,x
Proof. If y = 0, the above is obviously satisfied. Now assume y 0. Then for an arbitrary scalar C,
yx,yx0 y,yx,yy,xx,x
yy
yxSet
yy
xy
yy
yxthen
yy
yx
yy
xy
yy
yxx,x0and
222
2yxyyx,x0or
2121 y,yx,xy,xor
16
31
• The concept of perpendicularity and unit size
• In Rn or Cn, x and y are orthogonal (written as x y) iff x,y= 0; x is normalized if x,x=1. (i.e., ||x||2=1)– {x1, x2, .., xm} is an orthogonal set iff xi xj i j
Orthogonality and normality
– x is orthogonal to a set S if x s s S
2
0
2
x,
0
3
0
x,
1
0
1
x 321
Rba,,
b
b
a
S,
1
1
0
x
32
Can we have m>n? No. Because the dimension is n, andAn orthogonal set of nonzero vectors is a linearly
independent set. How to show this?
Consider an orthogonal set of nonzero vectors {x1,x2,…,xm} Rn? What can we say about it?
17
33
• Suppose that {x1, x2, .., xm} is an orthogonal set of nonzero vectors and iixi = 0, we can show that
i must be zero for all i=1,2,…m. How?
<xk, iixi> = <xk, 0> = 0
= ii <xk, xi> = k <xk, xk>
k = 0 k {x1, x2, .., xm} are linearly independent
– If m = n, then {x1, x2, .., xn} is good candidate for a basis in view of its orthogonality
34
What can be said about an orthonormal set of nonzero vectors {x1,x2,…,xm} Rn? Let X=[x1 x2 … xm], what is X’X?
mm2m1m
m22212
m12111
m21
m
2
1
x'xx'xx'x
x'xx'xx'x
x'xx'xx'x
xxx
'x
'x
'x
XX'
{x1,x2,…,xm} is an orthonormal set if it is orthogonal and xi,xi=||xi||22 =1 for each i.
mI
100
010
001
x1
x2
x3
An even better choice for a basis
18
35
Example: θany for ,cosθ
sinθ,
sinθ
cosθ)x,(x 21
2Icosθsinθ
sinθcosθ
cosθsinθ
sinθcosθXX'
How about XX’? Do we have XX’=I? Need to be careful about this. If m=n, then from
X’X=I, we have X=(X’)1. Hence XX’=I. If m<n, then XX’ I , e.g.,
I
100
000
001
100
001
10
00
01
XX',
10
00
01
X
I10
01
10
00
01
100
001XX'
36
Let {x1,x2,…,xn} be an orthonormal set in Rn.• They are linearly independent to each other;• ||xk||2 = <x,x> =x’x=1;• Every yRn can be uniquely expressed as
1 1 2 2 n n
k k k
y=a x +a x + +a x
where a x , , the projection of y to x .y
19
37
Today:
The base of a linear space: Basis– Representations of a vector in terms of a basis
– Relationship among representations for different bases
Generalization of the idea of length: Norms
A sense of orientation: Inner Product– The concept of perpendicularity: Orthogonality
– Gram-Schmidt Process to obtain orthonormal vectors
Linear Operators and Representations
38
Consider Rn, – Given a set of LI vectors {ei}i=1 to m, ei Rn.
– Form the set of linear combinations
Rα :eαY i
m
1iii
Then Y is a linear space, and is a subspace of Rn.
It is the space spanned by {ei}i=1 to m
The dimension of Y is m
A basis of Y is (e1, e2, …, em)
Subspace spanned by a set of vectors
How can we obtain an orthonormal basis of Y?
20
39
Orthonormal basis for a subspace
Given a basis (e1,e2,…em) for a subspace Y. Suppose that the vectors are not orthogonal to each other. How to get an equivalent basis (ê1,ê2,…,êm)
which also span Y and in addition ||êi||2=1 for all i, êi êj for i j? (*)
Basic requirement, êi has to be a linear combination of (e1,e2,…em), i.e.,
êi = pi1e1+pi2e2+…+pimem
(ê1,ê2,…,êm) = (e1,e2,…em) P
Problem: find pij, i,j =1,2,…m such that êi’s satisfy (*)
40
Gram-Schmidt Process
– Example: Consider R2
x1
x2
e1 = (2, 1) T
e2 = (1, 2) T
111
2122 v
vv
evev
v2
Take v1 = e1
– The component of e2 that is to v1:
2.1
6.0
1
2
14
22
2
1
The projection of e2 onto e1
21
41
• {v1 , v2} is an orthogonal set:
<v1 , v2> = -1.2 + 1.2 = 0– What is next?
– Normalization. How?
x1
x2
515
2
vv
e1
11
5251
vv
e2
22 e2
_e1_
42
• General Procedure:– Let {e1, e2, .., en} be a linearly independent set
– Form an orthogonal set by subtracting from each vector the components that align with previous vectors
11 ev 111
2122 v
vv
evev
222
321
11
3133 v
vv
evv
vv
evev
1n
1kk
kk
nknn v
vv
evev
– Normalize the new set
ivv
ei
ii
22
43
– Relation between vectors: LD/LI, inner product
– Length: norms, normalized vectors
– Representation: basis, orthonormalizations
• So far, we have addressed a few issues on linearvector spaces:
Next we will discuss operations between vector spaces linear operations and matrices
44
Linear Operations
– Range: {y Y | x X, s.t. f(x) = y} Y
• Functions– A function f is a mapping from
domain X to codomain Y that assigns
each x X one and only one element of Y
• What is a "linear function"?
x y
fX Y
– A function L that maps from X to Y is said to be a linear operator (linear function, linear mapping, or linear transformation) iff
L(1x1 + 2x2) = 1L(x1) + 2L(x2)
1, 2 R (or C ), and x1, x2 X
23
45
Linear Operations Associated with Matrices
• Consider X=Rn and Y=Rm
• Given an mn real matrix S;• For xRn, define
y=L(x)=Sx– You can show that L is a linear map
• Thus we can use a matrix to define a linear map.• What if we define y=f(x)=Sx+c for a nozero cRm?
– Of course not a linear map
• More to say? Is every linear map from Rm to Rn
associated with a matrix? • Yes !
x y
LX=Rn Y=Rm
46
Matrix Representation of Linear Operators
• Still consider X=Rn to Y=Rm. • A linear operator is uniquely
determined by how the basis are mapped
Theorem. Suppose that– {e1, e2, .., en} is a basis of Rn
– Then L: X Y is uniquely determined by n pairs of mapping
ei → yi , i = 1, 2, .., n
x y
LRn Rm
24
47
• Let the basis of X be {e1, e2, .., en} • Let the basis of Y be {w1, w2, .., wm}• Suppose that
mi
2i
1i
iim21ii
s
ss
s,swwwy)L(ee
i
• Then by linearity, for any xX, with representation i.e., x=i=1
nieiwe have
n
1i
n
1iiim21ii sαwwwyαL(x)yx
11 12 1n 1
n21 22 2n 2
1 2 m i i 1 2 mi 1
m1 m2 mn n
s s s α
s s s αw w w α s w w w
s s s α
• Representation of y: S
The linear operator is determined by how the basis are mapped
S
48
Example. Rotating counter-clock-wise in R2 by
How to proceed?
1
0e,
0
1e 21
1
0w,
0
1w 21
– What is y1? What is y2?
sin
coswwwsinwcosy 21211
cos
sinwwwcoswsiny 21212
s1
s2
cossinsincosS
– If x=ee2, e.g., = (1, 1)T, and = 90, then y=w1+w2, with
1
111
0110Sαβ e1
e2 x y
e1→ y1, e2→ y2
Le1Le2
e1
e2y2= = y1
25
49
1.5,2121S2,11
21S1.5,2131S 321
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
-5 0 5-5
0
5
-5 0 5-5
0
5
-5 0 5-5
0
5
-5 0 5-5
0
5
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
-20 0 20-20
0
20
Given 3 matrices:
Subsets in R2
{y=S1x: xDi}
D1 D2 D3 D4
{y=S2x: xDi}
{y=S3x: xDi}
50
Change of Basis– L: x y ~ The mapping is independent of bases
– = A ~ The ith column of A is the representation of
Lei (= yi) w.r.t. {w1, w2, .., wm}• The representation A depends on the bases for X and Y
– Consider a special case where L: Rn Rn ( or Cn to Cn)
– With {e1, e2, .., en} as a basis, under operator L
x → y: [e1 e2 .. en] → [e1 e2 …en] A– For simplicity, we denote L: =A– Suppose the basis is changed to {ê1, ê2,…,ên} for X,Y.
– What would be the new rep. of L?
– Suppose the new rep. is Ā. How are A and Ā related?
26
51
– Still consider the map
x → y: [e1 e2 .. en] → [e1 e2 …en] A
Under the new basis, we have
x → y: [ê1 ê2 .. ên] P → [ê1 ê2 …ên] PA
Let the new basis be
[ê1 ê2…ên]= [e1 e2 .. en] Q
Then equivalently,
[ê1 ê2…ên] P= [e1 e2 .. en], where P=Q-1
If we let a=Pb=PAthen a → b=PAP-1a
The new rep. for the operator is Ā=PAP-1= Q-1AQ
52
1PAPA A
11 QAQPAPAor
– This is called the similar transformation, and A and are similar matricesA
27
53
1PAPAwith,xAx
Example. Consider
32
10A,Axx
20
01
21
11
32
10
11
12PAPA 1
x2001x
What is the dynamics for a new representation with
x1 112Pxx
A much simpler structure
Later on we will discuss how to find such a P matrix
The diagonal elements capture the maincharacteristic of the system
54
Today:
The base of a linear space: Basis– Representations of a vector in terms of a basis
– Relationship among representations for different bases
Generalization of the idea of length: Norms
A sense of orientation: Inner Product– The concept of perpendicularity: Orthogonality
– Gram-Schmidt Process to obtain orthonormal vectors
Linear Operators and Representations
28
55
• Next Time: 3.3 to 3.6– Norm of a Linear Operator– Systems of linear algebraic equations– Orthogonal complement– Similarity transformation: the Companion form– Eigenvalues and Eigenvectors– Diagonal form and Jordan form– Functions of a square matrix
56
Homework Set #4
Problem 1: Given a basis for R3,1 0 0
2 , 1 , 03 2 1
1 1Express x= 1 and y= 0 in terms of the basis.
1 1
Problem 2: Let the old basis be
100
e,010
e,001
e 321
The new basis is 1 1 2 1 2 3 1 2 3e e ; e 2e +e ; e e -2e +e
1) For 1 2 3x 2e 3e e , find a,b,c such that x=ae1+be2+ce3
2) For x=e1-e2+e3, find such that ,eγeβeαx 321
29
57
Problem 3:
1) Use Gram-Schmidt procedure to perform orthonomalization on the basis
1 2
1 10 , 11 1
e e
2) Express the new basis in terms of the old basis3) What is the representation of x=e1+e2 in terms
of the new basis?
Problem 4:0 0 1
x Ax, A 0.5 0 0.50 2 1
1) What is the dynamics for a new representation with
0.5 0 0.5x Px, P 0 1 0
0.5 0 0.5
Consider
Problem 5 : (optional) Reproduce the plots on slide 49.
top related