499-413-prac-1 · title: microsoft word - 499-413-prac-1.doc author: svitek created date: 8/16/2005...
TRANSCRIPT
System Representation by a set of Low Dimensional Models Miroslav Svítek
Czech Technical University in Prague, Faculty of Transportation Sciences, Konviktská 20, 110 00 Prague 1
1. Introduction The paper combines the approaches known in signal
processing area with the approaches typical for neural networks. In the signal processing community the signal parameters as mean values, correlation matrix, covariance matrix, regression parameters vector, etc. are estimated through measured data. The knowledge of such parameters results in the identification of the optimal LTI (linear time invariant) model describing the main system property [1]. The model with estimated parameters could be used for time interpolation, filtering, extrapolation, etc.
2. Sylvester's theorems
The decomposition methodology described in this chapter is based on the application of Sylvester's theorems on system transition matrix A of m-dimensional linear time invariant system described by state-space model:
nnn
nn1n
ux yuxx
•+•=•+•=+
DCBA (1)
where nnn y,u,x are m-dimensional state, input and output vectors in time interval n and A,B,C,D are state-space mm ⋅ matrices. Theorem 1: Sylvester theorem for distinct eigenvalues If A is square system transition matrix (1) and if i. represents one of the n distinct eigenvalues of A, and if P(A) is any polynomial of the matrix A, then ∑ .∑
. = .=
.
.−.•.−
•.=.−.
•.−•.=n
1i
n
ij ji
ji
n
1in
ijij
ii
)()(
)(P)(
)(Adj)(P)(PIAIAA
(2) where Adj(A) means the adjoin matrix that is formed by replacing each element of matrix A by its cofactor and then taking the transpose. Theorem 2: Sylvester theorem for repeated eigenvalues If A is square system transition matrix (1) and if i. represents an eigenvalue of A repeated is times, and if k is number of all distinct eigenvalues j. , and if P(A) is any polynomial of the matrix A, then
i
i
ik
)seigenvaluedistinct all(
1i k
ijj
1s
1s
i )(
)(Adj)(Pdd
)!1s()1()(P
.=.
=
.
−
−
∑.
.−.
•.−•..−
−= IAA
(3) where Adj(A) means the adjoin matrix and if all
eigenvalues are equal, then 1)(k
ijj =.−..
.
.
Definition 1: With respect to equation (2) and with assumption of distinct eigenvalues the special matrixes
n21 Z,....,Z,Z could be defined for matrix A:
.. .−.
•.−=
n
ij ji
ji )(
)(Z
IA (4), with following properties:
∑=
=
==•
.=•
n
1ii
iji
ji
.1Z
j,ifor ZZZ
j,ifor 0ZZ (5)
Proof: The proof of theorem 1 and 2 is done in [4] where the proof of matrix components for repeated eigenvalues is also presented. 3. Approximation of derivatives
In equation (3) the derivatives are necessary to be solved. In following theorem the form of derivatives approximation is presented together with the approximation error. Theorem 3: Approximation of derivatives Let f(x) is any function of x and
....dx
)x(fd,dx
)x(fd,dx
)x(df3
3
2
2
first, second, third, etc.
derivatives of function f(x), then the approximate derivatives could be expressed
( )
( )
( ).etc
,)h2x(f)hx(f2)hx(f2)h2x(fh21
dx)x(fd
,)hx(f)x(f2)hx(fh21
dx)x(fd
,)hx(f)hx(fh21
dx)x(df
33
3
22
2
−−−++−+
−+−+
−−+
(6)
Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)
where for all derivatives the approximation error is proportional to ( )2h. . For better precision of approximation other approximate forms exist, e.g. for approximation error proportional to
( )4h. the following equations could be described:
( )
( )etc.
,)h2x(f)hx(f16)x(f30)hx(f16)h2x(fh121
dx)x(fd
,)h2x(f)hx(f8)hx(f8)h2x(fh12
1dx
)x(df
22
2
−−−+−+++−
−+−−+++−
(7) Proof: The proof is given in [4]. Theorem 4: Complex-step approximation of derivatives
Let f(x) is any function of x and dx
)x(df is first
derivative of function f(x), then the approximate derivatives with complex-step approximation could be expressed:
( ) ( )[ ] ( )[ ]
hhjxfIm
hhjxfImlim
dxxdf
0h
•+•+=.
(8) where Im[.] means imaginary part. Then the approximation error is proportional to ( )2h. . Proof: Let us express yjxz •+= and
( ) vjuzf •+= together with Cauchy-Riemann equations [6]:
yv
xu
.
.=..
(9), xv
yu
.
.−=..
(10)
If f is analytic function (satisfy the (9) and (10)) than we can use and rewrite (9) as follows:
( )( ) ( )
hyjxvhyjxvlim
xu
0h
•+−+•+=..
. (11)
Since function f is real function of a real variable than 0y = , ( ) ( )xfxu = and ( ) 0xv = . Equation (11) can be
than rewritten to (8). In order to determine the error involved in this approximation, the derivation based on Taylor series expansion is used (with pure imaginary step equal to hj• ):
( ) ( ) ( ) ( ) ( ) ........dx
xfd!3
1hjdx
xfd!2
1hdx
xdfhjxfhjxf 3
33
2
22 −•••+••−••+=•+
(12) Taking imaginary part of both sides of equation (12) and dividing this equation by h yields:
( ) ( )[ ] ( ) .....
dxxfd
!31h
hhjxfIm
dxxdf
3
32 +••+•+= (13)
Hence the approximation is ( )2h. . The higher order derivatives express in Theorem 3 could be transferred into complex-step method by using approximation of Cauchy's Integral Formula in general form ( Γ is a simple closed positively oriented contour that encloses z):
( ) ( )( ) ∑∫
−
=••π••
•π••
Γ+
•+
••
.−.
.•π•
=1m
0i mni2j
mi2j
1nn
n
e
erzf
rm!nd
zf
j2!n
dzzfd
(14) where m is the number of points used in the integration. the approximate of derivative of order
1m,....,1,0n −= could be fond by using ( 14). From complex variable theory, for a real function of the real variable that is analytic holds:
( ) ( ) vjuyjxfvjuyjxf •−=•−⇒•+=•+ (15).
4. Approximation of derivatives in Sylvester theorem for repeated eigenvalues (transformed eigenvalues method) The approximation of derivatives (6), (7) could be applied to Sylvester theorem for repeated eigenvalues (3) and the following theorem could be defined. Theorem 5: Approximation of P(A) with repeated eigenvalues of matrix A If matrix A has k all distinct eigenvalues (3) where d eigenvalues d. are repeated ds times and ch eigenvalues ch. are poorly distinct, then the polynomial function P(A) could be approximated by set of ch distinct and by set of t transformed distinct eignevalues t. ( hq,....,hq,..,hq,..,hq dddd1111 +.−.+.−. ) with
error proportional at least to )h(O 2 as follows:
∑ ∑∑
∑ ∑ .∑ .
= −=...
=
= −=. .∝ ∝
∝.
= .∝ ∝
∝
••.+.•+•.=
=.−•.+.
•.−••.+.•+
.−.•.−
•.=
d
1f
q
q,ff,fi
ch
1ii
d
1f
q
q
k
f ff,f
ch
1i
k
i ii
f
f
f
f
Z)h(PkZ)(P
)h()(
)h(Pk)()(
)(P)(PIAIA
A
(16) where fq depends on selected approximate form (6), (7), (8) and on the multiplicity of repeated eigenvalue f. ,
.,fk are weight constants of approximation form (6), (7),
(8) ..∝ ∝
∝. .−•.+.
•.−=
k
f f,f )h(
)(Z
IA are transformed
component matrices assigned to transformed distinct
Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)
eigenvalues and h is selected small approximation parameter. Proof: If the Sylvester theorem for repeated eigenvalues (3) is used and if the approximation according to equation (6), (7) or (8) is applied then the part of equation (3) assigned to repeated eigenvalue i. could be approximated with
error proportional at least to )h(O 2 or more (depends on used approximate form, e.g. forms (6), (7) or (8)) as follows:
iiii
i
i
i
i
i
i
q,iiiq0,ii0q,iiiq
1s
1s
ik
ijj
1s
1s
i
Z)hq(Pk..Z)(Pk..Z)hq(Pk
)(Z)(Pdd
)!1s()1(
)(
)(Adj)(Pdd
)!1s()1(
••+.•++•.•++••−.•=
=
.•.•
.•
−−=
.−.
•.−•..−
−
−−
.=.−
−
.=..
−
−
.IA
(17) where hq,...,,..,hq iiiii •+..•−. are transformed eigenvalues assigned to is times repeated eigenvalue i. and
ii q,i0,iq,i Z,...,Z,..,Z − are transformed component matrices assigned to transformed eigenvalues
hq,...,,..,hq iiiii •+..•−. . 5. Approximation based on modified repeated eigenvalues (modified eigenvalues method)
The Sylvester' theorem came out from Lagrange interpolation polynomial with distinct polynomial roots. The repeated roots in Lagrange polynomial could be modified and approximated by distinct roots according to following table with defined approximation error (it is easy to extend the table for higher root multiplicity):
In case the parameter h is small enough the approximation error could be also small because of high power of h. This methodology could be applied to Sylvester's theorem and the modified eigenvalues could
be used instead of repeated ones and then the decomposition could be done with help of equation (2). As an example following table describes the multiplicity of original repeated eigenvalue and its modification to distinct ones: Tab. 2 Original repeated and modified distinct eigenvalues
Multiplicity of original repeated eigenvalues
Modified distinct eigenvalues
2. h ,h +.−. 3. h , ,h +..−. 4. 2h h, h,- ,h2 +.+..−.5. 2h h, , h,- ,h2 +.+...−.
etc. etc. The LTI dynamical system decomposition is based on
the application of Sylvester's theorems or its approximation to state-space matrix A (1) and by using the property of components matrices to calculate the transformed input, output and state-space vectors of one-dimensional models.
6. The LTI system decomposition with distinct eigenvalues of transition matrix
The LTI system decomposition with distinct eigenvalues could be done with help of following fundamental decomposition theorem. Theorem 6: Fundamental decomposition of LTI dynamical systems with distinct eigenvalues of transition matrix A Dynamical m-dimensional LTI dynamical system described by state-space model (1) with transition matrix A with distinct eigenvales could be decomposed into
2m one-dimensional LTI models where the component matrices assigned to transition matrix A are used as transformation matrices of state, input and output vectors (filter banks).
Tab.1: Modified roots in Lagrange polynomial [4]
modified roots/
repeated roots
(a-2h)
(a-h)
a
(a+h)
(a+2h)
approx.
error
2)ax( + 0 ))ha(x( −+ 0 ))ha(x( ++ 0 2h 3)ax( + 0 ))ha(x( −+ )ax( + ))ha(x( ++ 0 )ax(h2 + 4)ax( + ))h2a(x( −+
))ha(x( −+ 0 ))ha(x( ++ ))h2a(x( ++
22
4
)ax(h5h4
+−
−
5)ax( + ))h2a(x( −+
))ha(x( −+ )ax( + ))ha(x( ++ ))h2a(x( ++ 32
4
)ax(h5)ax(h4
+−−+
Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)
Proof: The m-dimensional state-space model (1) with square
mm ⋅ matrices A, B, C, D1 could be decomposed with help of components matrices m21 Z,...,Z,Z into following form
nmn1nmn2n1
nmn2n1n,mn,2n,1
nmn2n1nmmn22n11
1nm1n21n1
uDZ...uDZxCZ..xCZxCZyZ....yZyZy..yy
uBZ..uBZuBZxZ...xZxZxZ...xZxZ
••++••+••++••+••==•++•+•=+++
••++••+••+.++.+.==+++ +++
(18) where form P(A)=A in equation (2) was used and the
matrix components property ∑=
=m
1ii 1Z (5) was taking
into account. By multiplying the equations (18) by component matrix jZ and by taking into account the
property jifor 0ZZ ji .=• and
jifor ZZZ iji ==• (5) the LTI dynamical system could be decomposed into m following sub-systems:
njnjnjn,j
njnjj1nj
uDZxCZyZy
uBZxZxZ
••+••=•=
••+••.=• +
(19) with transition value equal to j. . The transformed m-
dimensional state vectors nj xZ • { }m,...,2,1j . were obtained through filtering of state vector by component matrices (component matrices play the role of filter banks). Each component of transformed state vector
nj xZ • could be easily described as a first-order
dynamical model because the transition value j. is
common for all m transformed states nj xZ • . 7. Identification methods of low dimensional models
The identification task yields to estimation of s-dimensional vector unknown parameters:
[ ]m21m21m21m21 ,...,,,p,...p,p,b,...,b,b,a,...,a,aw ...=r
(20)
based on the knowledge of data vector: ]x,....,x,x,x,...,x,x[x 1i,m1i,21i,1i,mi,2i,1i −−−=r
(21)
where the transformed state-vector fulfill the conditions on m one-dimensional models:
1 for non-square matrices B,C,D, the zero elements could be completed as well as in vectors nn y,u to achieve the form (4.1)
•
•
.
..
=
•
−
−
−
1i,m
1i,2
1i,1
mmm
222
111
m
2
1
i,m
i,2
i,1
mmm
222
111
x..
xx
p..ba........p..bap..ba
0..00..........00..0
x..
xx
p..ba........p..bap..ba
(22) The parameters must be estimated under the pre-
defined composition rule:
•
•
=
i,m
i,1
mm
11
m,m1,m
m,11,1
i,m
i,1
x..
x
p..a......p..a
r..r......
r..r
x..
x
(23) where parameters m,m1,m1,1 r,....,r,...,r are given
beforehand. The (22) could be rewritten into following form:
( )( )
( )
•
•
.
..
−
•
=
−
−
−
1i,m
1i.2
1i,1
mmm
222
111
m
2
1
i,m
i.2
i,1
mmm
222
111
im
i2
i1
x..
xx
p..ba........p..bap..ba
0..00..........00..0
x..
xx
p..ba........p..bap..ba
w,xf..
w,xfw,xf
rr
rr
rr
(24) with the additional functions representing
composition rule: ( )
( )
•
•
−
=
+
i,m
i,1
mm
11
m,m1,m
m,11,1
i,m
i,1
in
i1m
x..
x
p..a......p..a
r..r......
r..r
x..
x
w,xf..
w,xf
rr
rr
(25) The (24) and (25) are set of n non-linear functions
that could be linearized by matrix Taylor series as follows (from (24) and (25) it arises that the functions
( ) ( )ini1 w,xf,....,w,xf rrrrmust converge to zero vector):
( )( )
( )
( )( )
( )
( ) ( ) ( )
( ) ( ) ( )
−
•
..
..
..
..
..
..
+
=
−
−
−
=
−
−
−
−
0..00
w..
ww
w..
ww
ww,xf..
ww,xf
ww,xf
..........w
w,xf..w
w,xfw
w,xf
w,xf..w,xfw,xf
w,xf..
w,xfw,xf
1i,s
1i,2
1i,1
i,s
i,2
i,1
wws
in
2
in
1
in
s
i1
2
i1
1
i1
1iin
1ii2
1ii1
iin
ii2
ii1
1irr
rrrrrr
rrrrrr
rr
rr
rr
rr
rr
rr
(26) In next part we mark the vector of unknown
parameters in i-time step as iwr :
[ ]Tisiii wwww ,,2,1 ...=r
(27)
The measurement vector will be marked as 1iz −r
and transition matrix between measurement and parameter vector as 1iD − . The vector 1iz −
rand matrix 1iD − are
computed from known data vector ixr and last estimate of parameters 1iw −
ras follows:
( )( )
( )
−
•
..
..
..
..
..
..
=
−
−
−
−
−
−
=
−
−
1iin
1ii2
1ii1
1i,s
1i,2
1i,1
wws
in
2
in
1
in
s
i1
2
i1
1
i1
1i
w,xf..w,xfw,xf
w..
ww
w)w,x(f..
w)w,x(f
w)w,x(f
..........w
)w,x(f..w
)w,x(fw
)w,x(f
z
1i
rr
rr
rr
rrrrrr
rrrrrr
r
rr
(28)
Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)
1iwws
in
2
in
1
in
s
i1
2
i1
1
i1
1i
w)w,x(f..
w)w,x(f
w)w,x(f
..........w
)w,x(f..w
)w,x(fw
)w,x(f
D
−=
−
..
..
..
..
..
..
=
rr
rrrrrr
rrrrrr
(29) Based on representation (28) and (29) the
equation for extended Kalman filter [14] could be written:
ii1i1i ewDzrrr +•= −−
(30) where noise vector ie
r is supposed to be Gaussian
with zero mean and covariance matrix Q: [ ] [ ] ji ,cov j,i 0,cov ==.= Qeeee jiji
rrrr
(31) The time evolution of parameters vector is supposed
to be random walk: i,w1ii eww
rrr += − (32),
where noise vector i,wer is supposed to be Gaussian with zero mean and covariance W:
[ ] [ ] ji ,cov j,i 0,cov ,,,, ==.= Weeee jwiwjwiwrrrr
(33). The extended Kalman estimation filter could be for
the studied case written in following form: ( )
( ) 1T1ii1i
T1i1ii
1ii
1i1i1ii1ii
WDSDDSH
QSSwDzHww
−−−−−
−
−−−−
+••••=
+=•−•+= rrrr
(34)
where a priory information 11 S,wr must be known (estimated) in advanced.
8. Examples of low dimensional identification Let us define the LTI system with two repeated eigenvalues 121 == .. as follows with matrix (23)
−
=
+
•
−
=
−
−
25.0
xx
,ee
xx
2110
xx
2,0
1,0
i,2
i,1
1i,2
1i,1
i,2
i,1
(35)
−
=
11
11rrrr
2,21,2
2,11,1 (36)
In this example the parameter vector ),b,a,,b,a(w 222111 ..=r
will be estimated by identification method. The equations (26) could be
rewritten for studied example as follows (in this example m=2, n=4):
−•−+•−−•++•+
•+••.−+•+••.−+
=
−−−−
−−−−
−−−−−−−
−−−−−−−
−
−
−
−
i,1i,21i,21i,1i,11i,21i,1
i,1i,21i,21i,1i,11i,21i,1
1i,21i,21i,11i,21i,2i,21i,2i,11i,2
1i,21i,11i,11i,11i,1i,21i,1i,11i,1
1ii4
1ii3
1ii2
1ii1
xx)bb(x)aa(xx)bb(x)aa(
)xbxa(ˆxbxa)xbxa(ˆxbxa
)w,x(f)w,x(f)w,x(f)w,x(f
rr
rr
rr
rr
(37) where )ˆ,b,a,ˆ,b,a(w 1i,21i,21i,21i,11i,11i,11i −−−−−−− ..=r
is last estimate of parameters vector. In simulation mode the noise covariance was selected:
=
1001
Q (38)
In Fig. 1 the approximate and original evolution of state component i,1x is shown where the evolution of estimated vector parameters is shown in Fig. 2.
0 5 10 15 20 25 30-140
-120
-100
-80
-60
-40
-20
0
20Evolution of real and approximated system
Time interval
Evo
lutio
n of
real
and
app
roxi
mat
ed s
tate
com
pone
nt X
1
Fig.1 Approximate (+) and original evolution of state
components i,1x
0 10 20 30 40 50 60 70 80 90 100-0.5
0
0.5
1
1.5
2Evolution of estimated parameters
Time interval
Evo
lutio
n of
est
imat
ed p
aram
eter
s
Fig. 2 Evolution of estimated parameters with result:
Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)
4457.0a 100,1 = , 5540.0b 100,1 = ,
0064.1ˆ100,1 =. , 4767.0a 100,2 = , 4768.0b 100,2 −= ,
9687.0ˆ100,2 =. The same method was used for system with two
complex conjugate eigenvalues 866.0j5.01 •+=. , 866.0j5.02 •−=. as follows
−
=
+
•
−
=
−
−
25.0
xx
,ee
xx
1110
xx
2,0
1,0
i,2
i,1
1i,2
1i,1
i,2
i,1
(39) Fig. 3 shows the evolution of approximate and original state component i,1x and Fig. 4 shows the evolution of parameters vector.
0 10 20 30 40 50 60 70 80 90 100-10
-8
-6
-4
-2
0
2
4
6
8
10Evolution of real and approximated system
Time interval
Evo
lutio
n of
real
and
app
roxi
mat
ed s
tate
com
pone
nt X
1
Fig.3 Approximate (+) and original
evolution of state components i,1x
0 10 20 30 40 50 60 70 80 90 100-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6Evolution of estimated parameters
Time interval
Evo
lutio
n of
est
imat
ed p
aram
eter
s
Fig.4 Evolution of estimated parameters with result: 3637.0a 100,1 = , 3626.0b 100,1 = ,
5165.0ˆ100,1 =. , 3619.0a 100,2 = ,
3588.0b 100,2 −= , 4923.0ˆ100,2 =.
9. Conclusion The presented results have shown the theory of LTI
systems decomposition for distinct and repeated eigenvalues of transition matrix A of state-space model together with identification algorithm. This theory vindicates much known practice of mixtures of low dimensional dynamical models to approximate the higher order dynamical system. In paper the direct proof of LTI dynamical system decomposition with distinct and repeated eigenvalues of matrix A was presented as fundamental decomposition theorem. The decomposition theory was demonstrated on numerical example together with identification method.
References
[1] Peterka V.: Bayesian Approach to System Identification.
In: Eykfoff P. (ed.): Trends and Progress in System Identification, Pergamon Press Oxford, 239-304.
[2] Function of matrices www.ece.utexas.edu/~ari/Courses/EE380K/Functions_of_Matrices.pdf]
[3] Martins J., Kroo I., Alonso J.: An Automated Method for Sensitivity Analysis Using Complex Variables, American Institute of Aeronautics and Astronautics, AIAA-2000-0689, 2000.
[4] Svítek M: The Decomposition Theory of LTI Systems, Neural Network World, 6/04, pp. 489-506, 2004.
Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)