-
Realization of the Markov parameter sequences using thesingular value decomposition of the Hankel matrixCitation for published version (APA):Hajdasinski, A. K., & Damen, A. A. H. (1979). Realization of the Markov parameter sequences using the singularvalue decomposition of the Hankel matrix. (EUT report. E, Fac. of Electrical Engineering; Vol. 79-E-095).Technische Hogeschool Eindhoven.
Document status and date:Published: 01/01/1979
Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne
Take down policyIf you believe that this document breaches copyright please contact us at:[email protected] details and we will investigate your claim.
Download date: 13. Jun. 2021
https://research.tue.nl/en/publications/realization-of-the-markov-parameter-sequences-using-the-singular-value-decomposition-of-the-hankel-matrix(4d0c695f-1b3c-4669-883b-2d7e043937fc).html
-
Realization of the Markov parameter sequences
using the singular value decomposition of the
Hankel matrix
by
A. K, Hajdasinski and A. A. H. Damen
-
j
E I N D H 0 V E NUN I V E R SIT Y 0 F T E C H N 0 LOG Y
Department of Electrical Engineering
Eindhoven The Netherlands
REALIZATION OF THE MARKOV PARAMETER
SEQUENCES USING THE SINGULAR VALUE
DECOMPOSITION OF THE HANKEL MATRIX
by
A.K. Hajdasinski
and
A.A.H. Damen
TH-Report 79-E-95
ISBN 90-6144-095-5
Eindhoven
May 1979
-
CONTENTS
Abstract
1. Introduction
1.1 Remarks about the desired type of identification
1.2 The degree of complexity of the model:
the order definition of a MIMO-system
1.3 Gauss-Markov estimation of Markov parameters
2. Description and properties of the Ho-Kalman Algorithm
3. Description and properties of the Singular Value Decomposition
3.1 Existence of the S.V.D.
3.2 The least squares fit on a matrix
3.3 Some properties of the S.V.D.
4. Derivation of the realization algorithm using the S.V.D.
4.1 The noise free case
4.2 Estimation of the realization for the noisy case
4.2.1. Estimation of the system order fi 0
4.2.2. Estimation of the Hankel matrix of rank fi 0
4.2.3. Estimation of the shifted Hankel matrix
5. Results of the simulation-examples
6. Conel us ions
Appendix: Some remarks about the limitations of S.V.D. realization
References
2
2
3
12
17
20
20
21
22
23
23
25
25
21)
27
30
43
46
49
-
-1--
REALI ZATION OF THE MAHKOV PAHAMETEH SEQUENCES USING THE SINGULAH VALUE DECOMPOSITION OF THE HANKEL MATRIX
Abstract
Identification of multi input/multi output systems is the topic of this study. A
crucial problem is the degree of complexity of the model used to estimate
tile original system. In mathematical terms tllis is defined by the order or
the dimension of the system and these important notions for multivariable
systems are redefined. Once these notions have been strictly defined and
Markov parameters have heen estimated [rom input/output sequences, it is
shown, that a clear estimate of the system dimension can be obtained by means
of a singular value decomposition of the Hankel matrix. which is built up
by all estimated Markov parameters. With the results of that same singular
value decomposition it is possible to derive a realization, which turns out
to improve the Ho-Kalman algorithm Ln the case of additive independent noise
on the output signals of the system. Especially in low order, long Markov
parameters sequences the presented algorithm seemS to be preferable-It offers
us the possibility to incorporate all estimated Markov parameters into the
estimation of a realization of the system to be identified. In the noisefree
case hoth tllgorithms are equivalent.
Addresses of the Authors:
A. K. Hajdasinski, Central Mining and Designing Office, Plae Grunwaldski 10/8, KATOWICE, Poland
A. A. H. Damen, Group Measurement and Control, Dcpartn.ent of Eledrical En~ineerin~, E indhuvcn L1ni vcrsity uf TcchnuloKY, P.O. Box Gl:l, [,(iOO Mil EINDIIOVEN, The Ndht,,'lands
-
-2-
1. INTRODUCTION
1.1. Remarks about the desired type of identification
In this study we are solely interested in the identification of multi
input / multi output (MIMO) relations in the mathematical sense. No
physical interpretation whatsoever will be pursued and the characte-
ristics of the models will dominantly be determined by our limited,
mathematical ahility, which defines characteristics as linearity, inde-
pendence o[ disturhances, (inite dimensionality etc. Nevertheless the
first aim is application to real processes, which puts strong restric-
tions on possible models and methods. This last
-
-)-
seem to be able now to supply a prac:tiCi"l]]Y relevant algori l:llm, wC'
merely confined ourselves to practical tests by means of model-to-
model adjustments.
The above adstructed degree of complexity corresponds to mathemati-
cal terms as dimension and order. These notions, which are rather
complex in MIMO-systems will be defined and elucidated in the next
paragraph.
2. Once the degree of complexity has been determined or the mathematical
cquivalent,the order estimation has been perforrr:.erl a set of parameters
may he defined, which uniquely describes the model of that special
order. This set of parame_ters may be identified then in the sense, that
a residual (some error between process- and modeloutput) is minimized.
However, various sets of parameters, are possible and one set may be
transformed into another set.
The set of minimal number of parameters, given a certain order of the
model, can be denoted as fundamental, because all other sets necessari-
ly show interdependence of the different parameters.
The interrelations between the different parameter sets will be commen-
ted upon in the next paragraph, especially the "induced sequence" M: ,
denoted as Markov parameters, that can be supposed to be the multidi-
mensional impulse response on the one hand and the "realization" A, B
and C, which is generally known as respectively the system-. input-
and outputmatrix, on the other hand.
1.2. The degree of complexity of the model: the order definition of MIHO-
systems
The structure of the models under discussion ~s defined by the following
adjectives:
- linear
- multivariable (MIMO multi input / multi output)
- time invariant
- finite dynamical
- eli scretc
-
-4-
Onc,e this structure has been given, the complexity has to be limited.
Thi" is known as the model order determination or as the system order
es timation problem. While for S1S0 systems the notion of the system
ord,er is very well defined and extensively worked out, for M1MO-systems
the term "order" causes a number of misunderstandings and ambiguities.
lIowever the order definition of the MIMO-system is even more important
tl!:ll1 [or S[SO-systems. 1\ reasonahle reduction of the state space dimen-
sion, closely related with the order, is extremely important for the
sake of the modelling and computational simplicity.
In the sequel of this report there will be made an attempt towards un~
fication and the precize definition of the multivariable dynamical system
ord,er. Very useful then will occur to be the so called H-model of the
lin,ear multi variable dynamical system, but also some other definitions
must be remembered here in order to make clear all possible equivocal
pas.sages. There will also be printed out the equivalency of different
typ,es of models used for identification of the multi variable system.
See~{ing for thie equivalence was a necessary condition for the develop-
ment of the realization theory i.e. methods of finding the state space
description given a transfer matrix or input/output data.
Definition I: For the mllltivariahlc, linear. dynamical system having p
inputs u 1 (k) '" up (k) and q outputs y I (k) ••. y q (k) there
is defined the qxp matrix K(Z) called the transfer matrix
(being considered the rational matrix of the argument z)
fulfilling the following condition
1. (z)
where y I (z)
1. (z) =
y (z) q
K(z)u(z) :
.':: (z)
u (z) p
and y(z) ,u(z) are the "z" transforms of y(k) and u(k)
under zero initial conditions. 14 I Ii 111121
-
-'1-
iJefinition 2: The characleristic polynomi31 W(?) of the strictly proper
or proper tran.fer matrix K(Z) is defined 35 the least
Conunon iJenominCltor of 311 minors in K(z). having hy the
greatc.t power of Z the coefficient equal to one. I I 11 1121
Definition 3: The degree "IK(z)1 o[ the strictly proper or proper transfer
matrix K(z) is defLned as the degree of its characteristic
polynomial. (Practically it is the smallest number of shifting
elements necessary to model the dynamics of this system).1111
Definition 4: For the multivariable, linear, time invariant, dynamical sys-
tems, the state of the system at an arbitrary time instant
K = ko 1S defined as a minimal set of such numbers YI(ko)'
'J2 (k ) ... x (I< ) the knowledge of which together with the o n 0
knowledge of the system model and inputs for k ). k "
x(k ) = - 0
x (k ) n 0
is sufficient for determination
of the sys tern behaviour for k ~ k o
is called the state vector, and
members xI (ko )" ·yu(ko ) are called state variables.p IIh21
Definition 5: The set of difference equations
~:k+l) = ~(k) + B ~(k)
where x(k) is a (nxl) state vector
~(k) is a (pxl) input vector
is called the state equation, while the set
y(k) = C x(k) - -"here Z(k) 1S a (qxl) output vector
is called the output equation.1 4 11la II~
-
--------------------------------
-h-
Definition 6: The number n of state variables Ln the state equation 1.8
defined as the dimension of the state vector or the state
space and also denoted as the dimension of the complete
system.
Definition 7: The triplet of matrices I A, E, C I is defined as the reali-zation of the dynamical, linear, time invariant, multi va-
riable system.
Definition 8: Any polynomial f(:I.)
Lermna I:
f (z) k k-I 2
~ z + Clz + ... + Ck_2z + Ck_lz + Ck [or which holds
f (A) ~ Ak C Ak- I 2 Ck_1A + C A
O ~ ri + + ... + Ck_2A + I k I
1S called the annihilating polynomial of the A matrix.
The characteristic polynomial of the A matrix - WA(z) is
one of the annihilating polynomials for A (according to
Cayley-Hamilton).
Defl_nition 9: The polynomial f(z) of the smallest, nonequal zero degree k,
fulfilling definition 8 1S called the minimal polynomial
of the A matrix. II IJ 1121 [9]
Definition 10: The matrix coefficient '\ ~ C Ak B for k ~ 0, 1,2, ..... is referred to as the k-th Markov parameter of the system
defined by the realization lA, B, CI .19 III~
(i.e. the inverse liZ" transform of the transfer matrix)
Definition II: The following description of the multivariable dynamical
system is referred to as the H-model of this system.
1711811121
u (i) o for i < 0
where
-
-7-
r'J ~"(O)] y " i (I) ; 1I ':- ~(I) ; /; - properly dimensioned 1. (2) u(2) block vector containing the initial conditions
[MO MI ~-I " " "J Hk~ MI M2 ~ ... Generalized Hankel Matrix M2 M3 ~+I
1 Generalized Toer1itz Matrix
and M - for k = 0, I, 2 ••• are the Markov parameters of k
the considered system.
Now it is necessary to present two theorems being fundamental for fur-
ther considerations.
Theorem I:
Remark:
The sequence of Markov parameters {~} for k = 0, 1, 2, 3 ...
has a finite dimensional realization {A, B, C) if and only if there are an integer r and constants a. such that:
M . r+J
r
= "' /. i=1
a. M . . for all J > 0 L r+]-l
~
where r ~s the degree of the minimal polynomial of the state
matrix A (assuming we consider only minimal realizations).
Theorem 1 is called the realizability criterion and the r
is called the realizability index.
The proof of this theorem ~s to be found in 1 7 1 19 1 II 21
-
Theorem '2:
Remark:
-8-
If the Narkov parallleters sequence {Mk} [or k = 0, 1, 2 ••.
has a finite dimensional realization {A, B, C), \rlith reali-
zability index r,
state space (also
fulfils
then the rr.inimal dimension n of the o
of the realization) for this realization
where
and
H = r
rank III I = n r 0
n - minima] state-space dimension o
I Mil - qxp n r x mln (p,q)
0
M MI M r-I 0
MI M2 M r
LS the Hankel matrix (see definition II).
(fhe proof of this theorem is given in 17119 1112)
From the linear dependence of Markov parameters follows lhat
that rank II N = rank II r+ r
n for all N~O. o
Hith the aid of these II definitions and two theorems it is possible now
to generalize the meaning of the system "order" for different types of
multivariable system descriptions. Before it will be made formally, it
can be of some use to present a simple scheme sharing relations between
the three defined types of models: the transfer matrix, the state equa-
tions and the H-mode 1. (See fig. I)
From this scheme we learn that while from the state space description
there is a straiglltforward way to get the transfer matrix K(z), the re-
verse procedure viz. realization theory is a lot mnTC' complicated. On the
contrary knowing the. Markov narameters it is equally easy to get any re-
quired form of description. For the sake of modelling, Markov parameters
can be derived as easily from the state space description as from the
tranl-lfer matrix. Obviously Markov parameters are also used in the H-rnodel.
-
-')-
other
rC;lli~ations
.. lin-Kalman state space
II, B, e K(z) realization description
+ Markov
parameters
• t II-model
fig. 1. Interdependence of different type models.
considering all pro's and contra's it seems desirable to express .11."1
structural invariants of multivariahle dynamical systems in terms or Markov parameters.
Definition 12: The order of the multivariable system will be defined
as the minimal number of Markov parameters necessary
and sufficient to reconstruct the entire realizable
sequence of Markov parameters according to the theorem l.
In other words the multivariable system order is equal
to the realizability index r.
Alternatively fur the state space description, the multivariahle system
order C;ln he defined ;ts tllC degree of tile minimal polynomial of the
stilte matrix 1\. This follows Jirectly from the proof of theorem I. For
the transfer matrix descri ption llOwcver, the order defini tion in the
general Cilse is not possible.
-
-10-
Defin:ction 13: The dimension of the multivariable dynamical system is
defined as the number n being equal to the rank of the o
H -Hankel matrix for this system, where r is the reali-r
zability index (order of the system).
Alternatively for the state space description it is the dimension of the
state matrix A. And again for the transfer matrix description there does
not exist a unique definition of the system dimension. Only in the case
when
-
L ... _
-11-
Theorclll I mui? lngl'tlier wiLh dl·finitiolls I? and '"3 ;lrc ('ssl'Iltinl for lhe
estimation of the system order. As it was pointed out, the complexity de-
finitiort for the multivariable dynamic system can be most flexible and
most general while using the Markov parameters description. The task of
the complexi ty identification will thus he to determine the order rand
tile (minimal) dimension n of the system being considered. ~~~~~~~~~~~o~~~~~~~~~~~~~~
Such .1. posing of the problem is possible only for some strictly conceptual
systems having both finite order and finite dimension. In such a case
looking for exact rand n makes sense. However in real systems identifi-o cation problem one cannot search for any exact rand n because usually
o those are systems of infinite order and dimension. The only goal .!hich we
can aim at is to find a reasonably simple approximation of the real system
(i.e. estimates of the rand n ). o
As an illustration can be given the following example:
Example I:
K(d +3
-(z-2)
(z+ I) 2
2 (z+4) 2
(2 + I)
The Markov parameters are:
M {)
rank
112
f 112
jll I f
0
3
=f III 2
-I 4
2 4
4 -2 -7
4 -3 -10
defCree I K (z) f (there are no
in K(z».
= 2 " n o distinct poles
-
-12-
So thl~ dimen:-;ion of titP system J S
n = 3 0 One of possible realizations is:
+ 2
-J [: ;} c=[~ A -2 1 ; B = -2 The characteristic polynomial of 1\ ~s:
3 (z+ I)
Ilut the minimal polynomial of /\ is:
"1
:J 0
2 11/\ (z) = (z+l) as can easily be checked, so the order of the system
is, i11deed H = H • 2 r
1.3. Gauss-Markov estimation of Markov parameters
This .mbjeet has b('en broadly treated en I R I , and here will be presented
only main features o[ the method. Starting with the definition 12 of the
II-model it is straightforward to derive the following equation:
T T '1' Y = Hk Sm + M S 00 ( 1 )
where
'! T -"- [ Y (I) Y (I + 1 ) ••• y (I +m) ] (2)
and Z(l) I-th measurement of the output vector
Z (1 +m) I +m-th measurement o[ the output vector
1\ " [M(O) M( I) ••. M(k)] - block vector cont.qining first k+1
Markov parameters of the multi- (3)
variable system.
-
-13-
T N " [H(k+l) ~f(k+:~) ... ~I(I;:-t·llI) ... ~I(I+III-k)]- hlne],; Vt't'lnl" Ill" I il1il4'
2::(1-1) 2:: (1-1 +m)
S " 2::(1-2) 2:: (1-2+m) m=
u(l-I-k) .. 2:: (l-I-k+m)
2:: (1-k-2) .. ~(1-2-k+m)
S A u(1-k-3) .. !!(1-3-k+m) 00
2:: (0)
0
o 0 .•. 0 2::(0)
lec~tll cnntaininJ~ r('-(4 )
maining Markov para-
meters of the considered
system.
finite dimensional matrix
of the input samples
finite dimensional matrix
of the input initial con-
ditions
(5 )
Relation (I) represents the real system equation, while the model equation
IS assumed to be:
(6)
l' -1' N stands for the estimate of the ~ and Y for the estimate of the
Considering that the multivariable dynamical system is corrupted by the
rnultivariable noise E, relation (I) may he written as:
Y (7)
where E is defined as:
(8)
The equation (7) refers to the model shown in fig. 2.
-
-I ',-
noise
fi lter [-
H
y
fig. 2. The block diagram of the multivariable dynamical system described
in terms of Markov parameters.
The multivariable no~se E is considered to be an output of a colouring
filter [or the white multivariable noise H
Assuming that {.':!.(i)} and{~(j)} are for all i and j mutually uncorrelated and that expected value of the {!:(i)}.' \!;.(i)} = Q.iln estimate of the first k+l Markov parameters is found minimizing the following loss function:
v w
whe:re W is a nonnegative weighting matrix and
Solution of this problem results in the following expression for the
Markov parameters estimate:
N T -I
(S W S) S W Y m m m
Expressing N in terms of the E
N
(9)
(10)
(J I)
Conditions under which the N is an asymptotically unbiased estimate of
tlw ~ are discussed in 15 118 I
-
-\5-
UStln11y, due to decre~lsinfl nnttIre of tile {H} sequence [()r staille systems T
for k .1nd m great enough the term S M can be neglected and the part of 'I' -\ ,n tlte hi ns (B W S) S W E assymptotically vanishes when assUming m m '!11
I L~(j)} = 0 and there IS no correlation between samples of E and S . ill
Because those are the only cases to be handled,. the following expression
is ~lssurned to deliver the ;rsymptotical1y umbiased estimate N:
N
-I Choosing as the weighting matrix W _ R , where
it can be shown that such a choice leads to minimization of the
As the following inequal ity
II' 1 (Mk -N) (Mk -N) 'I'f w II :> II, 1 (~-N) (~-N) T f R II
-\ can be proved for all W I R •
At las t
( \2)
( 13)
(I 4)
(15)
and this estimation is equivalent with the Gauss-Markov procedure for the
single input/single output system 1411d. [3]
There remains only to find an appropriat~estimate of the R matrix. This task is completed with the aid of the composite noise notion and again
the realization theory. The estimate achieved on this way is called fur-
ther the Gauss-Markov Estimate with Realization of the Covariance Matrix
(named further C.M.R.E.).
_~Appropriate in the sense, that the profit of the use of R instead of the identity matri~ is not overcompensated in the negative sense by the devia-
tions in the estimate of R.
-
-)(,-
This estimate has hC(.'11 inlrotiuct'd in I H I and also lll(.'rt' are discussed
further properties of the C.M.R.E. estimation.
From this moment, for the sake of the continuation of the re~ort, it can be assumed that we are having the C.M.R.E. estimates of the identified
system Narkov parame ters ~
-
-17-
2. DESCRIPTION AND PROPERTIES OF TIlE HO -KALMAN ALCORITHM
The original version of the lIo-Kalman algorithm has been derived for
noise-free systems. Since the time it was first published r 9 I a number of .1.1gori thros has been proposed basing upon the Ho-Kalman ver-
sion and attempting to give solutions for cases where Markov parameters
were estimated from the operating records. However ever since has not
been proposed any, giving a satisfactory simple approximation of the
realization without a high complexity to attack this problem. So~e
facts about the Realization Algorithm of B.L. Ho and R.E. Kalmanl 9 11121 are reviewed.
Theorem 3: 191 114/
There 1S
El -=-k
For an arbitrary, finite dimensional, linear, dynamical sys-
tem given the input-output map the canonical realization
exists in tIle following form:
-p- number of inputs
-q- number of outputs
-n- dimension of the realization
defined the following matrix
k x 1 . [k matr1x ~ l-~ ~ if k < 1 [II ] k x 1 matrix o~ if k > 1 -'1
-
-18-
In 0 pr-n
-n P II = En E
pr --r !l -qr -n
n pr-n 0 0 -qr-n -qr-n
where H - is the !Iankel matrix for the considered system r
!I -1'
M !'!c l ••• !'!cr-I ] M2 ••• M - -r
• • • • • •• !'!c2r- 2
3. A canonical realization of the considered system is given by:
A = Eqr P (aH ) Q En -n - -r - -pr
B = Eqr P H EP -n - -r -pr
C Eqr H En -q -r -pr
where (YH 1S the shifted Hankel matrix -r
oil -r
MZ ••• M ] - -r • • • • • • •• M I -r+ • • • • • • •• M
-2r-
The .proof of this realization theorem can be found in reference [II.].
Also there can be found the following theorem which will be useful
late:r.
Theol:em 4: The Penrose-Moore pseudoinverse of the Hankel matrix H is ------ -r
r,iven the following:
where H+ stands [or the lJenrose-Moore pseudoinverse of the H -r -r
(without proof here).
-
-19-
Bee,luse for the n01 sy case the realizability criterion (theorem I and 3)
will never be fulfilled, (for the linear dependence is lost in case of
estimated Markov parameters) it is necessary to optimize the multi-
variable system model order r another way. Some solutions of this pro-
blem are suggested 1n 15 II 71. Also thc 1I0-Kalman algorithm does not fit to the no1SY case, because
using not all Markov parameters estimated (only the number which comes
out from the order test 2r-1 see ref. I 7 I), it truncates the informa-tion contained in all noisy data estimated.
.!J Supposing the estimate r of the system order r has been already estima-
ted, it is to be seen that! and Q matrices are to be evaluated basing "-
on the H~ matrix, which is already truncated to qr x pr dimension. Thus -r
if there are estimated L > 2r-1 Markov parameters necessary to produce '" ~ It? and CTHr , then information contained in remaining L-2r+ I Markov para-meters is lost.
*)
Ren~ember , tllat for the noise free case the order r and the dimension n o
have to fulfill the following conditions:
r 1 ) det {H } = 0 M r+j L a. M .. J ). 0 r
i= 1 1 r+J-1
det {H H T}= 0 if p .1< q r r
2) rank{1l } rank {Il } n for all N > o. r r+n 0
3) > n r 0
min (q, p)
In the noisy case
smallest integer,
minant will not lIe
n can he estimated from 2), while r will be the o
that satisfies 3) and 1) approximately, as the deter-
zero exactly due to the noise.
-
-"0-
J. DESCRIPTION AND PROPERTIES OF THE SINCULAR VALUE DECOMPOSITION
The Sjngular Value Decomposition (S.V.D.) will he introduced ill a very
compact form by means of few most important theorems and definitions.
Host vigorous and formal material dealing with this subject is to be
found in121, 16 1,1101.
3. I. Existence of the S.V.D.
Theore:n 5: For any m x n matrix A, the S. V.D. exists, given by:
where
U - 1.5 a m x II matrix consisting of f1 orthonormal colunms U. -J
so:
T U U If'
f' - 1S the rank of the matrix A
D - 1S the f' x f' diagonal matrix:
D diag ("I' "Z' "f')
(11 ~ (fZ';y. •• ~ rTf} > 0
v - ~s a n x I} matrix consisting ofporthonormal columns v. so: -J
The (T. are called singular values. J
The proof of the theorem 5 can be found 1n many references among
them in 1101.
-
-21-
3.~. The le.:lst sqU.:lre_s fit on :l. m~ltrix
For the sake of the realization algorithm it will be desirable to limit
the rank of a Hankel matrix. This also must be done with a minimal effect
on this matrix. Expressing this problem in categories of m x n matrices
(rectangular): if A is the original m x n matrix with the ranklAI =1',
the task will be to find the m x n matrix B with the rank{B I < I' in such
a way that the euklidean norm from A-B is minimal. The norm of this type
is given the definition:
Definition 16: The norm of a matrix A will be defined as:
Theorem·6:
In other words it is necessary to find a B with a limited
rank and minimized norm of difference between A and B. [2] [10]
1611101 Given the S.V.D. for a m x n matrix A:
A diag (aI' a2 ••• al')
IT J ? If 2 ~ (I 3 ~ ... ~ (" > 0
The m x n matrix B of a rank k.,;!' and such that IA-BI m1n, 1S given the following:
f ' 1 Dk I 0
vT vT B U ---:---- Uk Dk k o I 0
I
where Uk contains the first k columns of U V -
k contains the first k columns of V
Die = diag. (lf1 , ('2 '" a k)
Remark: So the B matrix 1S found by setting the smallest
r-k singular values equal to zero in the S.V.D.
-
-22-
It u; also possible to evaluate the error made during such a fit:[IOl
absolute error: IA-BI = lEI =
relative error:
,/~ ~
('
J. j=k+1
/' J.
j=1
Jf 2: ? fT~ "" = J 1,j j=k+1 J 2
IT. J
2 IT.
J
This theorem will be the basic tool for the system order determination and
approximation of the Hankel matrix.
3.3 Some properties of the S.V.D. (10(.
Property I: If the s.v.n. of A is ~lven by A A + of A wi 11 be (2 ( , (I () ( :
and the singular values of A+ are:
u n vT, th~ pseudo lnverse
-I -I (J IT
fJ ' P-I'
Property 2: From the S.V.D. and orthonormality of the columns of V it
follows that
T tr(A A)
/' ;::
j=1
2 fT.
J
-
•
-23-
4. DERIVATION OF THE REALIZATION ALGORITHM USING THE S.V.D.
it wiH be demonstrated that the S.V.Il. delivers in a noise frel'! case an
exact realization algorithm which is equivalent to the Ho-Kalman algorithm
but intuitively simpler and sav1ng some computational efforts. For the
noisy case the S.V.D. will deliver for a chosen r the best approximation
in the least squares sense of the realization.
4. I. The noise free case
The whole procedure will base upon the S.V.D. of the Hankel matrix ~.
Having exact Markov parameters it is always possible to find .k,~ r the
system order. For such a case referring to the Theorems 2, 3, 4, 5 and
Property I we have:
where [D J according
n = n o
= n x n o 0
to theorem 2 the rank will be
(16)
( 17)
(loS)
Comparison of (17) and (18) may lead to the following equivalence (infinite
possibilities are available however)
(19)
V (20)
With the aid of (19) and (20) the HO-Kalman algorithm equations can be re-
written the following way:
A D -I T
U o-H k V (21 )
-I U
T IT D VT P VI EP (22) B D Epk = pk
C EqkU D VT V Eqk U D (23) q q
-
where
-
-:~)-
P f\ Q ["~~\~!] (Uqk) T Un D (Vn ) T [vn vPk- n] ; qk qk n pk pk pk fJ I qk-n
~-" J n I a EPk EPk ---;--- E" D t,n qk n n ·qk n o ,I k , q -n
This last completes the devivation of the realization algorithm which
c.an be used for further consideration.
Actually this shows especially for the n01se free case that P and Q
contain too many degrees of freedom, while U~k and v~k contain strictly
sufficient parameters necessary to construct the realization from the
Hankel matrix.
4.2. Estimation of the realization for the noisy case
In the noisy case there can be performed an easy test which singular
values are substantial and which can be neglected comparing their rate
of decrease.
4.2.1. Estimation of the system order n "
It is assumed that there are estimated 2k Markov parameters. From those
Markov parameters it is possible to construct the following Hankel and
shifted Hankel matrices:
Hk ; aHk
Performing the S.V.D. there also 1S found a vector of singular values
°13-°2;' 30 . (where s = k * min. (p, q). Comparison of singular values s gives a solution to the order test. Neglecting
values we determine no as the dimension of the
s-n singular values. o
smallest s-n singular o
realization. Consequently
we omit the smallest
A criterion deciding which singular values are sufficiently small is very
problem-dependent. But in all investigated cases (model to model) there
always existed very sudden changes in singular values for an increasing
position index. This situation is illustrated in fig. 3.
-
o
0' J
•
\
2
\
\
\
'--
3
-20-
~/A
-
-:'7
Because the Markov parameters have been found as an consistent L.S.
estimate (also efficient) of ideal ones, the S.V.D. appea~
one more filtration of the noisy data in the L.S. sense. The
approximation of the realization takes the following form:
A n- I 'l' (61'k ) V V n n n
B -I
VT
V VT EP VT
EP D D n n n n n pk n pk
C Eqk V D VT
V Eqk
V D q n n n n q n n
to be
(Z7)
(Z8)
(Z9)
still remains one more problem to solve, the estimate of the shifted
Hankel matrix aHk
. To solve this it is necessary to take a deeper look ~ ~
into the structure of Hk and aHk
•
4.Z.3. Estimation of the shifted Hankel matrix
~
Let us remember once again the structure of the Hk and crHk matrices:
M MI MZ ~-I a
MI MZ M3 ~ H =
k
~-I ~ ........ MZk- Z
MI MZ Y7Y M2 M3 / /Y /~+I
"II = M3 M4 / ~+I/ k / / / / / / /MZk- Z
/ / ,/ - - - - . - / - .-~t ~+I
/ MZk-ziMZk-1
From equations (30) and (31) it 1S seen that only one element in aHk
differs from elements of Hk
, and this is the last Markov parameter
estimated for the dynamical system MZk_ I '
(30)
(31 )
-
-28-
"" Huwever, after the least squares fit on the 1\ we have:
Ilk
1JJ. where 11, r
II 12 110 111
ZI 22 111 I1Z
11 32 I1Z 113
kl kZ 11 k-I 11k
13 112
Z3 po
J
33 1'4
k3 I1k+ I
Ik 11k - I
Zk 11k
,3k l'k+ I
kk 112k -k
J; i, J I • Z •••• k
{ 0, I ..• Zk-Z,
(12)
whic',l means that the block synunetry property in the 1\ matrix is lost. It s,~ems like Hk matrix was the Hankel matrix of a vari linear system.
Taking this under consideration there can be proposed two structures
of the aHk matrix:
1,
aHlk =
IZ 13 PI )lZ
22 Z3 I1Z 113
3Z 33 113 114
kZ k3 \lk \lk+1
Ik Zk Pk-I 11k
Zk/3k 11k I1k+1
3k/· I1k+1
/~k (33) I1Zk-Z
kk / - ----
I1Zk-Z : ,
The last column of the shifted matrix is constructed copy~ng elements
of the last column of Ilk ,which is due~to the assumption about the vari-
linear nature of the system described Hko Making such a choice we
possibly cormnit the smallest nonaccuracy in aHk estimation.
-
-29-
kk Al so it is proposed to take [or 1l
2k-
1 the c:orrcBponding real vaJ lie of
the estimate M2k
- l • As it has been experimentally checked, the
realization (27), (28) and (29) is not very sensitive to changes of
jl ~~_I' However, M2k_ 1 is taken from a different matrix space, which has the realization of the higher order then estimated r, as it
incorporates the noise.
2. 21 22 23 2k ]1 I ]12 113 Ilk
31 32 33 3k 112 113 114 Il k+ I
a H2k
kl k2 k3 kk Ilk-I Ilk Ilk+1
kZ/ k3/ • Ilk Ilk+1 Il
• IlZk-Z /' ---
kk : IlZk-Z: t
MZk- 1
The last row of the shifted matrix ~s constructed cory~ng elements of
the last row of Hk , which again is due to the assumption about a vari-
linear nature of system described Hk
. And again it is proposed to take kk
for PZk-1 the corresponding real value of the estimate MZk_l~
There are many other possibilities of the solution for the aRk matrix
estimation. However, having in mind also a simplicity of the algorithm,
which in case of multivariable and multidimensional systems can be even
more important than a slight sophistication of the formalism , one or
two above mentioned methods is proposed as a solution.
In numerical experiments there was chosen the realization algorithm
incorporating the allik shifted Hankel matrix.
As the complete algorithm for the noisy case is explained now and its
profits will be illustrated by means of a number of examples, some
remarks should be made of the drawbacks as well and more especially
about the heuristical property of the algorithm or the impossibility till
noW to prove the algorithm in some mathematical sense. Therefore some
remarks will be made in the appendix in order not to disturb the progress
of the explanation of the algorithm.
-
-30-
5. RESULTS OF THE SIMULATION-EXAMPLES
Example 2.
ConGider the two input, two output system given the following block
diagram - fig. 4.
1I1(z)
U (z) K (z) o
"I (z) 1: 2 (z) ,.._~_-L_~
+
J 1 (z)
J L (z)
Fig. 4. the transfer function model of the identified system.
The
the
K (Z) o
[
(Z - 0.8)
0.0
0.2 ]
(z _-_0_._8.:..) _(_Z_- O. 6 )
(z - 0.6)
*) fact that Ko(Z) = K~(Z) means that this system is equivalent to
system with an additive white input noise, if we omit initial
conditions.
The realization of the K (Z) 0
transfer function is given by:
A [o.s 0.2J [1.0 o.o];c= [1.0 0.0 ] B 0.0 0.6 0.0 1.0 0.0 1.0
and Markov parameters are:
[1.0 o·t = [0.8 0.2] . ~.64 0.28J M = MI M = 0 0.0 0.0 ' 0.0 0.6 ' 2 0.0 0.36 t· 512 0.296] ; M = [0.4096 0.28 J M = 4 . . . . 3 0.0 0.216 0.0 0.129
*) Tlle poles may be distinct or cornmon. As we neglect the initial
c·:mditions~ the poles can always be looked upon as cotmllon.
-
-'11-
Eigenvalues of the state m.:1trix /\ are:
Case a: ,n order to show how the modified lio-Kalman algorithm works
for the sake of the ideal systems modelling, the exact Markov
parameters were taken as an input to the algorithm.
The singular value decomposition of the H4
singular values:
°1 2.5701194
°2 1.4734138
°3 8.6947604 10- 12
~ 0
°4 1.5247013 10- 12 - 0
°5 I. 5134382 10-
12 - 0
H4 delivered following
It is quite clear that the system dimension n = 2. o
The estimate H4 used for the realization is exactly (i.e. within
accuracy of the computer) equal H4 , such that the system is exactly
realizable. Also oH4
can be evaluated exactly.
~0.834575 -0.068194) A =
-0.011896 0.565417
~o. 568066 -0.259609] B
-0;288067 0.772964
C = r 1 .504 1 70 -0.560574
-0.505194]
1.105450
This realization generates exactly ideal Markov parameters i.e. - -; - .
M. = CAB = C A'B. , Eirr,en valueI)' of the A matrix are:
0.79999 ...
0.59999 ..•
0.8
0.6
which shows practically exact ness of the proposed algorithm.
-
-32-
CaBe b: An additive coloured noise sequence is generated on the output
of the systC'm. The i npllt is gC'IlC'r:1 tC'd ;ts tIle wll i tC' nOJ s('
having a rectangular density function between (-I, 1)
while the noise filter input 1S generated
as the white gaussian no~se with the standard deviation 0.1.
The sequence of 10 Markov parameters was estimated in 6 runs
based on 100 samples in every run, and averaged over those 6
runs. The Markov parameters estimated this way with the G.M.R.E.
method are:
M=~I.OIO 0.019J M = [0.746 0.239"] ; [0.637 M = o -0.005 0.995 I 0.024 0.558 2 0.009
[0.481 0.294] M = [0.439 0.315 ] [0.362 M = M = 3 0.030 0.201 4 0.001 0.084 5 0.030
~O. 315 O. 188 ] M = [0.237 0.168 J M = [0. 102 M = 6 0.036 0.038 7 0.032 0.019 8 0.072
r· 027 0.083] M = 9 0.083 0.018 The S.V.D. of the H4 delivered following singular values.
°1
°2
°3
°4
2.0
1.0
2.591692
1.473426
0.231209
0.199619
2 o J
°5 0.146380
°b 0.088079
°7 0.039385
°8 0.021803
4 5 (, 7
Fi~. 5. singular values if the H4 matrix.
8
0.260 J 0.385
0.243J
0.304
0.14
J 0.034
-
-33-
Again it i:-; ohvious th.1t the .... dimension no :::: 2 will be the best
approximation. The estimate H4 used for the realization is not any
more e~uAl H4 , but the deviation is really very small.
As shown before the overall relative error will be: R
\I ii-l! \I { U(Dk-Dn) vT
V (DltDn) UT}~ [ 2 trace i~3 o· 1 .014 ~
Ill! II {u Dk VT V Dk uT 1 8 trace 2 [ i=1 o· 1
So the relative error 1n each element of H will be in the region of
~~ .12. This overall relative error of 12% is accomplished by an error of -4%
for the big numbers -I and by an error up to -100% for the small numbers
-.01. As the estimates of the Markov parameters have been given before,
the differences between H, Ii and H can be checked given the following found H:
0.975 0.024 I 0.766 0.222 I 0.638 0.272 0.520 0.268 -0. 00 2 0.990 I 0.025 0.600 1 0.021 0.408 0.039 0.276 I + . .. - I 0.766 0.244 0.607 0.300 0.505 0.298 0.417 0.282 0.014 0.599 0.027 0.366 1 0.023 0.251 0.032 0.17l
-I . , . l! ~ 4 0.637 0.268 0.507 0.298 I 0.423 0.281 0.350 0.257
0.003 0.409 0.01/, 0.248 0.011 0.170 0.018 O. liS I I . 0.517 0.293 1 0.413 0.288 0.344 0.260 I 0.287 0.230
-0.000 0.284 I 0.007 0.172 0.006 0.117 ,
0.011 0.080 ,
The matrix H4 has "almost" a block symmetric structure and assumptions
about a slight varilinearity of the system modelled H4 for the sake of
the aH4 reconstruction seems to be quite reasonable.
The second order approximation of the realization is found using the
simplest method for estimation of the aH4 i.e. filling M7 for the
lacking element of the oH4'
The approximation of
A [0.847094
-0.106906
B [
-0.539030
0.345924
the realization
0.053035·1 ;
0.576929
-0.299207 J -0.742041
takes form:
_ [-1.45438 C -
-0.677517
0.55351J
-1.06114
-
I
I
-34-
ThlS realization generates .1. very r,ood sequence of Markov pnr,amcters
es'~imates, being very close to original ones:
(fDr five first Markov parameters)
Ideal Markov Markov parameters Markov parameters
parameter generated via HO- generated via S.V.D.
Kalman realization realization - -
{M. } 1
{M. } 1
{M. } 1
1.0 0.0 1.0 0.00151 -- 0.97543 0.24434 M M M
0 0.0 0
-0.0041 1.0 0
-0.001871 0.99013 1.0
0.8 0.2 - 0.77918 0.20118 - 0.7797 0.20660 MI
0.0 0.6 MI
-0.005519 0.59000 MI
0.'124006 0.618721
0.64 0.28 0.6066 - 0.27492 - 0.62817 0.28213 H2 0.0 0.36 M2 -0.005671 0.34747 M2 0.03511 0.3915
0.512 0.296 0.47206 - 0.28393 .. 0.50902 0.29902 M3 0.0 0.216 M3 -0.0052270 0.20416 M3 0.003813 0.25172
0.4096 0.28 0.36716 0.26230 - 0.41430 0.28718 -M4 0.0 O. 1296 M4 -0.0045473 0.11957 M4 0.036940 0.164871
Th,~ minimal polynomial coefficients for Ho-Kalman and S. V. D. real izations
co:nparing wi th ideal ones are:
-
I
-35-
I I Minimal polynomial Minimal polynomial Minimal polynomial , 1 I coefficient coefficient coefficient I ideal system Ho-Kalman S.V.D. I realization realization I
- -a l -0.48 at -0.46 at -0.49 - -a 2 ' .4 a2 1.37 a2 1.42
Comparison of elgen values for state matrices of the Ho-Kalman reali.za-
tion and S.V.D. realization shows the following:
~ , 0.777 } HO-Kalman realization
A2 0.593
A, = 0.7906 } S.V.D. realization A2 0.6035
While ideal elgen valuES are A, = 0.8; A2 = 0.6.
It also occurs to be a very interesting experlence to observe the
norm of the following matrices:
II (Mk - ~ ) II 110 and
II (~ - ~ ) II S. V. D.
where T ~= (M 0 M, M2 ~)
T -~= (M M, MZ ~) 0 - T ~= (M 0 M, M2 ~)
-
-)6-
For k=2, which is the sufficient index to construct the He-Kalman
approximation of the realization there is:
II (~ ~) _ 0.0015
thus for k=2
II ~! - t\ II < II~ - ~II II S.V.D. I: 0 [or k=4
II ~ - ~II H 0.0216 0
II ~ - ~II S.V.D. = 0.02149 so
II ~ - ~IIH 0 II~ - ~II S. V.D. 0
but for k >4 for example k=IO
II ~\ - ~111l O. 1976 0
II ~\ - ~II = 0.03544 S.V.D. II ~ ~II > II~ - ~II S.V.D. II
0
Which induces the conclusion that while Ho-Kalman approximation of the
realization gives slightly better modelling of the syst€r.t in the
transient state, the S.V.D. approximation gives almost equally good
approximation in the transient and steady state, which means that it
gives a better overall fit.
This phenomenon is caused by the fact that S. V.D. incorporates more
available information containe d in noisy data than the H crKalman
algorithm.
Case c. The additive coloured noise 1S generated at the output of the
system. The input is generated as the white nOl.se having a
r~ctangular density function between (-I, I)
while the noise filter input is generated as the
white gal1Bsion noise with the standard deviation 0.5.
-
-37-
The Harkov parameters estimated under such conditiong with th£'
C.M.R.E. method arc:
(10 Markov parameters - 100 samples)
M = ~ I. II 0.2 ] M=(0.439 0.465J M = [ 0.t,39 a -0,112 I. 09_ I 0.0601 0.490 ~ -0.0247
M = l 0.101 0.22~1 M {0.294 0.32~; M = U· 377 3 0.110 0.189 4 0.0318 -0.102 5 0.073 M = [ 0.622 -0.022~] M =[0.677 0.124]. M = [0.342 6 0.0912 -0.0362 7 0.0423 -0.0287 ' 8 0.235
M = [ 0 .298 0.268 J 9 0.277 0.0038 The S.V.D. of the H5 delivered following singular values:
(JI
"2
= 2.0950862
= 1.3087709
~ 6. 3 1 L
2
°3
"4
2
= 6.8433805.10- 1 °5
= 5.5023061.10- 1 °6
°7
°8
3 4 5 6 7
Fig. 6. singular values of the 114 matrix.
= 5.0606823.10- 1
2.5954151.10- 1
1.326579 .10- 1
3.036047 .10-2
i 8
0.144J; 0.571
0.044j; -0.191
0.32~} 0.070
-
The result of the dimension test 1S less prOnOllnCJ.ng 1.n this case,
bllt still it LS possible to decide for n ~2. o
The estimate 114 used for the realization certainly suffers deviation
fn)m the block symmetry property, but also this deviation is rather
small considering the noise level.
Again the second order approximation of the realization is found using
the simplest method of the a1l4 estimation.
The approximation of the realization takes form:
A~ [ O. 741028 -0.0183221J - [-1. 12064 -1.01055 J c--0. I 10 I 77 0.33448 0.776194 -0.865162
- [-0.432248 -0.56872
J B~ 0.662446 -0.609601 Considering the noise level 50%, this realization is also a very good
api')roximation of the original one,havirig eigen values of the A:
, ~ 0.74593 ::1 A 2 =: 0.32957
Example 3
(0.8)
(0.6)
The following system, having a structure presented on the fig. 4. was
simulated under following conditions.
wh!:!:re
and
where
r·' 0 0 A ~ 0 0.4 'J B ~ 0 0 o. K (7)
0 C (17 - /1)-1 B
/I_~ rO• 1 0 ] LO 0.7
B ~ E,
K (I.) !;
C (II - /I ) - I B r, r; r,
l: -:] [~ 0 -~] c -I
-
-39-
Markov parameters COl" this system are:
H o
=CB=[-l,OOJ
1.0 1.0
[
-0.64 0.6-]
0.4 0.4
and 50 on.
CAB r-O. 8 L 0.4
0.61 0.4
[
-0.4096
0.0154
0.408J
0.0154
The intensi ty of the simul;lted noise "as 10% of the output signal
amplitude,
"i~envalues of the A matrix ),1=0.8, ),2=0.4, ),3=0.2.
Estimates of Markov parameters derived via C.M.R.E. method are:
to.994 -0.003] [-0.788 0.586]
H = MI 0
1.0 I .0 0.403 0.397
H2 [-0.611
0.162
0.s7
J 0.166 H) [ -0.469
0.069
0.47IJ
0.068
M4 [-0.347
0.028
0.376]
0.028 Ms
[ -0.246
0.015 O.27~J ; ....... 0.010
Singular values of the Hs matrix for such a set of Markov parameters
are:
°1 2.7706702
(J2 I. 7184590
l13 3.582625.10-1
"4 3.7090322.10-2
"5 8.1052389.10-3
-)
°6 3.819628.10 -3 "7 2.842466.10
°8 1.0787837. 10-3
-
-40-
0. L
1
.,
2 3 4 5 7 8 i
Fig. 7. singular values of the 114 matrix.
Tn this case the more pronouncing will be testing of the ra.tio:
o. 1
x
10
3 \ / \ \
r, \
\ 4 2
"-~ .•.. //
2 3 4 5 6 7 8 i
Fig. 8. ratios of sinr,ular values of the H4
which apparpntly stands for no 3.
-
-1,1-
After approximation of the 114 with the 114 calculated from the S.V.Il.
and applying the realization alf;orithm, the approx.imation of the
realization will be:
[ ,. '",''' A = 0.0852291
-0.547107
t ,.,m" B = -0.458556
0.510278
c = [-I . 12M, 7 0.459472
0.175944
0.429488
-0.421211
-,. ""'''] -0.775701. -0.566931
0.217971
-1.47049
0.""""] 0.0104386 0.205312
0.269546 J 0.0988497
ICigen values of the A matrix are:
0.7325
0.4075
0.2875
which again is a very good estimate of original ones.
Example 4
This case deals with the no~se free, two input/two output system
being of the first order and having the dimension I.
This example is "'eant to show that in such case also the S.V.D. aided
realization nlgorithm is capable to deliver correct results.
U 1 (z)
~L __ K_(_Z) ____ ~ ____ Y~I_(7_') ______ : -1 ... Y2(z) -Fig. Y. the block diagram of the considered system.
-
-42-
-1.0 1.0 (z 0.5) (z - 0.5)
K( ~)
2.0 -2.0
(z - D.5) (2 - 0.5)
The realization of this system can he:
A = 0.5 B=(I,-I)
The Harkov parameters are:
M o [
-I 11 2 -2
~1. I = 0.5 M. 1+ 1
c
[
-0.5 0.5J
I -I
The singular values of the H4 are:
"I I,. 1999
-II "2
1.4551915.10
"3 0
a4 a
M2 = [-0.25
0.5
and all a. 1
o for i = 3, 4, 5, 6, 7, 8, 9, 10.
Thi~: glves an exact answer for n = 1. o _
The realization computed from the 114 there is:
A 0.5
0.25·J -0.5
B ( -0.613572 0.613572 ) ( - 1.0 I .0 ) * 0.6 I 3572
Then
C
via
A
Il
C
[ 1.6298 J 1.6298 [-~J -3,2596 tIle similarity relation for T = 0.613572 we have
'1'-1 ~ T
'1'-1 R C T
1.6298+0.5.>'-0.613572 = 0.5
1.6298 ..-0.16572 ;, (-1.0 1.0) (-1.0 1.0)
I.6298H.I('357U[_~ J= [_~] which leads to the origi.nal reali.zalion.
-
(,. CONCLUS TONS
Along this report there has been made an attempt to unify and redefine
some notions important for the multivariable systems theory. Also
another version of the realization algorithm has been proposed. This
algorithm, called the Singular Value Decomposition Realization, shows
some important and jnteresting properties:
I. For the noise free case the dimension test is an intrinsic
part of the S.V.Il. realization.
2. For the no~sy case the S.V.Il. clgorithm delivers an excellent
and convincing dimension test, which can be implemented
automatically.
3. For the noise free case the S.V.D. realization ~s equivalent
to the Ho-Kalman realization (see 4.1.) and in the noisy case
the equivalency exists as well, if the minimal amount of
Markov parameters is used to constitute a Hankel matrix H. 4. For the noisy case the S.V.Il. algorithm lead" to an
approximate realization, which is hased upon all estimated
Markov parameters improving an overall fit of the model to
a g~ven system.
The whole procedure illustrating the identification of the multivariable
system in terms of the Markov parameters may be represented in the
following graph:
-
g
er 10th
! re.a I
lizations ~
i bas L.--
:Lng on Hk
-44-
~
- -- .. ~-----.
System _r ~ C.M. R.E.
method
{N.j .v
Ilk Order
Test
L ."-
S.V.D. -'\
Order test
4. ~ , Estimate llk
S.V.D.R. I-- uses (realizations)
~nly 1I~
~ r
[~i1
+ + .y Comparison
lIses only -J I!~ r Ho-Kalman
Realization
{Mil
, 1
, Conclusions!
I ,
- I
-
· I, ').
The conclltsions may he, that. the lIo-Kalman alr,orithm gives it better
fi t to till' first M~rkov p;lr;lllletcrs, while the s.v.n. alr,orithm provides a hetter overall fit especially for long Markov sequences.
This can be explained in the following 'Way:
In order to obtain a good, unbiased estimation of Markov sequences
from input/output signals, it is necessary to estimate all Markov
parameters above the noise level. If some time-constants of the
system are large compared to the sample time, quite a lot of Markov
parameters have to be taken into account.
The Ho-Kalman algorithm can only use a small number of Markov
parameters, determined by the order of the system, to constitute a
Hankel matrix and a consequent realization. In that way all information
contained in the remaining, estimated Markov parameters, which are not
used, is lost. Some errors in the first Markov parameters may lead to
great deviations, as they are not compensated by the information in
the latter estimated Markov parnmeters.
The algorithm, discussed in this report, uses all estimated Harkov
parameters, so it shows not the drawbacks of Ho-Kalman algorithm.
Nevertheless it is not ideal, as not all Markov parameters are weighted
equally in the least squares sense (see appendix). The Markov parameters
in the middle of the used sequence have much more influence on the
realization, than the first and latter Markov parameters. Nevertheless
it shows to be an excellent and comparatively simple tool to determine
the dimension of a system and producing a good estimate of the realiza-
tion of that system.
-
APPENDIX
Short remarks about the limitations of the Singular Value
Del~omposi tiOll Realization
Ci ven a lIankel matrix 113 as an example of the more general case:
• pk - ...... P .... 1 M MI 112 r M3 i a qk MI M2 M3 •
1 G M4 qk k ~ r
1 H2 ~13 Ill, Me 1 J t f t t ~I ~2 ~3 ~ '~
II
Lml,:icitly we were trying to obtain a solution for this equation In
the sense, that the 'variability' of R is minimal i.e.r dep,rees of
fre€~dom, where .0. ought to he:
T Il
(\ , I' I '2
o -p------ --pk -----
To t·.oat purpose we performed a Singular Value Decomposition on Hand
here we made (at least) four 'mistakes':
I. There -is also \lOI.se on~, so much more a A.S.V.D. should be performed
on H and ~ tor,ether like in the example of Golub and Reinsch. ::t)
M: the noj se on each M. is expected to be of equal level, we should ~
also equally wei!~ht ~.
(1 n the example of Colub and Reinsch k 1).
*) Go]uh and Hcinsch, "S.V.D. and L.S. solutions", Hand hook Series
Linear Algehra, Numer. Math. 14,1970, example 2J •. on page 408.
-
11 Even if W(' incorporate ~ and perform an S.V.D., we will
obtain a pragmatic solution, where not all (lM. have equally 1
III
been weighted. In the above example (lM2
has been weighted 3
times, compared with 6M1 and 6M3 2 times and 6Mo and (lM
4 only
once.
No necessary
H by H = V n n
restrictions were T
D V the general n n
il.= V 1)-1 VT V n n n ~ + s-n Y
s = m~n (pk. qk)
and Y is a (s-n) x p matrix.
made upon~. As we approximate
solution for ~ becomes:
In theory we are only able to use (s-n) 1< p degrees of freedom
to adjust il. to the restrictions i.e.:
a. Each pxq block contains equal
b. All off-diagonal elements are
s· 's on the main diagonal. 1
equal to zero.
c. The first s-r B. 's can be defined as zero, as the minimal 1
-
IV
-48-
T = U D V is not 'block-symmetric': n n n
The Hankel matrix H n
we cannot distinguish uniquely the respective Markov parameters.
This implies, that we cannot apply a shift operator, which is
used in the proof of the lIo-Kalman algorithm. Although not all
conditions for the IIn-K.oIman algorithm are fulfilled we use the
resulting formula to define the realization.
Ln practical applications it turns out to be profitable, but
there is sti 11 a lack of proof.
-
-49-
References
(I) Andree, R.V. C:OHI'UTATTON OF TilE 1 NVERSE OF A HATRTX.Ameri C;1n Hathcmatical Honthly, Vol. ')8(19",1), p. 87-92.
(2) Albert, A. RECRESS roN AND TilE HOOlm-l'lmROSE I'SEIIDO- I NVICHSE • New York: Ac'ldcmic: Press, 1C)72. ~tathem
-
-50-
(13) T;I11II0\l, .1.1.. "1'1' RIIX I flATE \l I: A II S S -fiA RKIIV I': ST I fIATIII{S AN II I{I': I ,A'I'I': II :: I :111': m::: • Eindhoven IIllivcrsi ty or 'I'(~clm()logy, 1971. -""I'~rttnent of Electric
-
E1NDIIOVEN UNIVU{SITY OF TH'IINOLOGY THE NETlIERLANIlS DEPARTMENT OF ELECTI{lCAL EN(;INFEI{IN(;
Reports:
I) Dijk, j" M. Jcuken and EJ. Ma'"Hlers AN ANTENNA FOR A SATH.LlTI' COMMUNICATION (;ROUNIl STATIUN (PROVISIONAL ELECTRICAL m'SI(;N). TII-Re'port (,X-E-Ol. 19C1X. ISBN '10-Cl I 44-00 1-7
2) Veefkiml, A., 1.11. Illom and L.H.Th. Rietjens TIII'OKETICAL AND I'XPERIMENTAL INVl'S"nl;ATION OF A NUN-EQUILIBRIUM PLASMA IN A MilD CIIANNEL.. Suhmilled 10 the SYIllPosium on Magnctohydrodynamic Ekclrical Power (;cneralion, Warsaw, Poland, 24-30 luly, 1'I6I~. TII-KqHlrt hX-E-02. 19hX. ISBN 90-61 44-0o:!-5
31 Boom, A,l.W. van den ,,,,
-
l'INIIiIOVI'N IINIVHlSITY 01' TH 'IINOIO(;Y 1m, NLTIIHllANIlS DEPARTME':NT OF ELH"I'RIl'AI. I'N(;INI'TIHN(;
ReporlS:
14)
15 )
16)
17)
18 )
19 )
21 )
23)
24)
25)
21»
27)
Lorelldll, M, AUTOMATic METEOR REFLECTIONS RECORDIN(; EQUIPMENT, TII-Rel'ort 70-10-14, 1970, ISBN 90-6144-014-9
SlIlcts, A,S, '1'111' INSTRUMENTAL VARIABLE METIIOD AND RELATED IDENTII'ICATION SCIIEMES, 'I'll-Report 70-1'-15,1970, ISBN 90-(,144-015-7
White, Jr" R,C. A SURVEY OF RANDOM METHODS FOR PARAMETER OPTIMIZATION, 'I'll-Report 70-1'-16,1971. ISBN 90-6144-016-5
Talmoll, J, I.. APPROXIMATED GAUSS-MARKOV ESTIMATORS AND RELATED SCHEMES, TH- Report 71-E-I7, 1971. ISBN 90-6144-017-3
v Kalasek, V, MEASUREMENT OF TIME CONSTANTS ON CASCADE D,C, ARC IN NITROGEN, HI-Report 71-1'-18,1971, ISBN 90-6144-018-1
lIossekt, L.M,L.F, OZONHILDUNG MrrTELS I;LEKTRISCIII;R ENTLADUNGEN, TII-RL'port 71-1'-19,1971. ISBN 90-1>144-019-X
Arts, M,G,J, ON '1'111; I NSTANTANI;OllS M I;ASU REM ENT OF BLOODFLOW BY UL TRASON IC MEANS, TII-Rc'pllrt 71-1'-20,1971, ISBN 90-6144-020-3
Roer, 11'h,G, van de NON-ISO TIIERMAL ANALYSIS OF CARRIER WAVES IN A SEMICONDUCTOR, 'III-Report 71-1'-21. 1971. ISBN 90-1>144-021-1
Jell!.cn, P,J" C. Hllhcr and CE, Mlliders SENSIN(; INERTIAL ROTATION WITII TUNING FORKS, 'I'll-Report 71-1'-22,1971, ISBN 90-6 I 44-022-X
Dijk, J., J,M. Berends and E.J. M,mnders APERTURE BLOCKAGE IN DUAL REFLECTOR ANTENNA SYSTEMS - A REVIEW, TH-Report 71-1'-23,1971, ISBN 90-6144-023-8
KrcgtiOlg, J. al1144-024-1>
Damen, A,A,II. and H,A,L. I'iceni Till' MliLTIPLE DIPOLE MODEL OF TilL; VI':NTRICULAR DEPOLARISATION, 'I'll-Report 71-1'-25, 1971, ISBN 90-1> 144-025-4
Bremmer, II, A MATHEMATICAL TIIU)I{Y CONNECTIN(; SCATTERIN(; AND DIFFRACTION PIIENOMENA, INCLUDINC BRAGG-TYPE INTERFERENCES, TII-Rcport 71-1'-26,1971. ISBN 90-6144-026-2
Bokho"en, W,M.G. van METIIODS AND ASPECTS OF ACTIVE R(,-FILTI'RS SYNTHESIS. 'I'll-Report 71-1;-27, 1970. ISBN 90-6144-027-0
Boes(>holcll. F. TWO I'LlIl()S MODEL RI;I'XAMINI'I) FOR A COLLISiON LESS PLASMA IN THE STATIONARY STATF. TII-Rl'J,ort 72-E-28, 1972, ISBN 90-h 144-028-9
-
ElNDIIOVI'N UNIVI-:!{SITY OF T1TIINOLO(;Y THE NETHeRLANDS DEPARTMENT OF ELHTI{ICAL ENWNEERINC
Reports:
~l) RI-PORT ON 1'111,: CI.OSI:D CYCLI: MilD SPECIALIST MH:TlN(;. Working group of the joint l'!I;t,A/IAEA Inll'rnalional MilD Liaison Group. EindhovL'Il, The Netherlands, September 20-22, 1971. Euited by L.H.Th. Rietjells. Til-Report 72-1'-2'1.1'172. ISBN 90-6144-029-7
30) Kessel, C.C.M. vall and J.W.M.A. Houben LOSS MECIIANISMS IN AN MilD GENERATOR. 'I'll-Report 72-1'-30. I (n~. ISBN 90-6144-030-0
3 I) Veefkind, A. CONDUCTION GRIDS TO STABI L1ZE MilD GENERATOR PLASMAS AGAINST IONIZATION INSTABILl'rll'S. 'I'll Report 72-E-31. 1972. ISBN '10-6144-031-9
32) Daalder, J.E., and C.W.M. Vos DISTRIBUTION FUNCTIONS OF TilE SPOT DIAMETER FOR SINGLE- AND MULTI-CATIIO])[: DISCIIAR(;]':S IN VACUUM. '1'11- Reporl 73-1'-32. I (J73. ISBN 90-6144-032-7
33) Daalder, J.E. JOULI: III,ATIN(; ANI) DIAMETI·:R OF TIll: ('ATIIODE SPOT IN A VACUUM ARC'. TII-Reporl 73-1:-33. I (!73. ISBN 90-6 I 44-03J-5
341 Huber, C. BEHAVIOUR OF Till: SPINNIN(; GYRO ROTOR. TH-Report 73-1'-34.1973. ISBN 90-6144-034-3
35) Bastiall, C'. et al. TilE VACUUM ARC AS A FACILITY FOR I{I':U,VANT I:XPI':RIMI'NTS IN FUSION RESI'ARCII. Annual Reporl 1972. EURATOM-T.lI.E. Croup 'Rotating Plasma'. I'll-Report 73+:-35.1973. ISBN 90-6144-035-1
31> I Blom, J. A. ANALYSIS OF PIIYSIOLO(;ICAL SYSTEMS BY PARAMETI'.R I':STIMATION TH'IINH)lII'.S. '1'11- Report 73-1'-3(>. 1973. ISBN '10-6 I 44-036-X
37 I Ca ncelleu
3111 Andriessen, F.J., W. Boerman and I.F.E.M. Holtz CALCULATION OF RADIATION LOSSES IN ('YUNDER SYMMETRIC HIGH PRESSURE DISCHARGES BY MEANS OF A DIGITAL COMPUTER. Til-Report 73-E-31l. 1973. ISBN 90-6 I 44-031l-6
39) Dijk, J., C.T.W. vall Diepellileck, E.J. Maandcrs amclI, A.A.H. A (,OMI'ARATIVI' ANALYSIS OF SEVI'RAL MODELS OF TilE VENTRICULAR DEPOLARIZATION; INTRODUCTION Or: A STRING-MODFL. 'I'll-Report 73-1'-41. 1973. ISBN 90-6144-041-6
-
ElNDIIOVEN UNIVERSITY OF T1'l'IINOLOGY TilE NETIlERLANJ)S J)EpAlnMENT OF ELElTRI('AL I':N(;INEEI~IN(;
Reports:
421 Oijk, 'G, H.M. van THEORY or (;YRO WITII ROTATIN(; (;IMBAL AND I''LLXURAL PIVOTS. '1'11- R"port 73-1'.-42. 1973. ISBN 9()-h 144-042-4
43 I Breimer, A.J, ON TilE I DENTI FICATION OF CONTINOUS LI NLA R PROCESSES. 1'11- Report 74-10-43. 1974. ISBN 90-6144-043-2
441 Lier, M,e. van 'Illd R.H.J.M, Otten CAD OF MASKS AND WIRIN(;. 'I'II-Repurt 74-E-44. 1974. ISBN 90-6144-044-0
4S1 Bastian. C et al. FXpl'RIMI'NTS WITII A LAI{(;E SIZED 1I0LLOW CATIIODE DISCHARDE FED WITII AR(;ON. Annual Report 197.1. EURATOM-T.II.E. Group 'Rotating Plasma'. Til-Report 74-1'-45. 1974. ISlIN 90-6144-045-9
4(, I Roer. TII.G. van de ANALYTICAL SMALL-SI(;NAL TIIEORY OF BARITT DIODES. TII-Rt·pmt74-E-46.1974.ISBN90-6144-046-7
471 Leliveld. W.H. TilL DESI(;N OF A MOCK CIRCULATION SYSTEM. TII-R"port 74-E-47. 1'174. ISBN 90-hI 44-047-S
4X I Damen. A.A.H, SOMI' NOTES ON '1'111' INVERSE PROBLEM IN ELECTRO CARDIOGRAPIIY. TII- RC'port 74-E-4X. 1974. ISBN 90-h I 44-04X-3
491 Meehcrg. L van de A VITERBI DECODER. Tlt-Rvport 74-1'-49. 1'174. ISBN 90-(,144-049-1
501 Poel, A,p.M. van der A COMPUTI'R SEAI{(,II FOR (;OOD CONVOLUTIONAL CODES. Til-Report 74-E-50. 1974. ISBN 90-6144-050-5
5 I I Sam"i.c, G. THE BIT ERROR PROBABILITY AS A FUNCTION PATH REGISTER LENGTH IN TilE VITEIWI DECODER. TII-Repml 74-E-51. 1974. ISBN 90-6144-051-3
Sci Scllalkwijk. J.P.M. CODIN(; FOR A COMPUTI'R NETWORK. TII- Repurt 74-1'-52. 1'174. ISBN 90-6144-052-1
53 I Stappc-r, M. MEASUREMENT OF TilE INTENSITY OF PROGRESSIVE ULTRASONIC WAVES BY MEAI\S OF RAMAN-NATII DIFRACTION. 'I'll-Report 74-10-53. 1'174. ISBN 90-6 I 44-053-X
54) Scllall;wijk, J.P,M. and A,J. Vinck SYN DROM F DECODI N(; OF CONVOLUTIONA L CODES. 'I'll-Report 74-E-54. 1974. ISBN ')0-6 I 44-054-X
55) Y"kimov. A, FLUCTUATIONS IN IMPATT-DIODE OSCILLATORS WITH LOW Q-FACTORS. 'I'll-Report 74-1'.-55. 1'174. ISBN 90-6 I 44-055-h
-
HNIlIIOVt:N lINIVI':I~SITY (W TlTIlNOIO(;Y THE NETIII'I{IANIlS DEPARTMI'NI' OF I'LHTRI('AL t:N(;INH':RIN(;
Reporls:
56) Plaals, J. van del' ANALYSIS OF TIIRU': CONDUCTOR COAXIAL SYSTEMS. COlllplller·Hltleti tlclcnninalion of the fl"l'qllCIlCY ciJaracicrislics and the impulse and step response of a two-pori t:ullsisting of a syslL'1ll of Ihrl'l' l'o;!\ialcOlldudofS terminating ill IUlllped impedances. TlI·RL'J'o!'1 75-1'-51>. l'n5. ISBN
-
UNIlI/OVII'N IINIVI'IISPERSION RATIOS IN NONLINEAR SYSTEM IDioNTI FICATI
-
EINDHOVEN UNIVEI{SITY OF TH'IINOLO(;Y THE NETlIERLANDS DEPARTMENT OF ELECTRICAL I':N(;INEERIN(;
Reporls:
XI, Kalll, J.J, v:llliler alld A.A.II. Dallll'lI (IIISI,:RVABII'TY OF I':U'CTRI('AL III'ART ACTIVITY STUDIES WI'I'lIllll': SINGULAR VALli I' IJFCOMPOSITION 'III-Report 7!l-E-XI. I'nx. ISBN '10-hI44-0XI·5
82) Jallsell, J. alld J.F. Barrell ON Till' TIIEORY OF MAXIMUM LlKFLIIIOOD ESTIMATION OF STRUCTURAL RI-:! .. ATIONS. Pari 2: Mulli-dilllellsiollal case. TII-Reporl 7S-E-X2. 1'I7X. ISBN 90-(, I 44-0X2-3
X3) Ellell, W. vall allti E. de J illig OPTIMUM TAPPED DELAY LINES FOR TilE E()lIALIZATION OF MULTIPLE CHANNEL SYSTEMS. 'I'll-Report 78-l'-!U. I 'nx. ISBN 90-(,1 44-0X3-1
li31 Villck, A.J. MAXIMUM LlKELlIIOOD SYNDROME DECODING OF LINEAR BLOCK CODES. TII-Rq)orl 7X-I'-X4. 1'I7K ISBN 90-(d44-0X4-X
X5) Spnoil, W.P. A DI(;!TAL LOW FRH)UENCY SPECTRUM ANALYZER. lISIN(; A PROGRAMMABLE POCKI'T CAL( ·lILATOR. TlI-Reporl 7X-E-XS. I97X. ISBN 90-hI44-0!i5-X
8(») Beneken, J.l'.W .• '1 al TREND PREDICTION AS A BASIS FOR OPTIMAL TIIERAPY. TII-Rl'port 7X-I'-Xh. I97X. ISBN 90-1> I 44-0XI>-()
X7) Ceus, LA.M. alld J. Dijk ('A LCli LATION OF API'.RTli RE AND FA R-FI ELD DISTRI BUT ION "'ROM M EASU REM ENTS IN Till'. IRI:SNI'.L ZONI' OF LAI{(;E REFLHTOR ANTENNAS. 'I'll-Report 7X-L-X7. 1'I7X. ISBN 90-1> I 44-0X7-4
!lX) lIajtiasinski, A. K. TilE (;ALJSS-MARKOV API'ROXIMATED SCIIEME FOR IDENTIFICATION OF MliLTIVARIABELE DYNAMICAL SYSTEMS VIA TilE REALIZATION THEORY. An Explicit Approach. TII-Reporl 78-1'-88.1'178, ISBN 90-6144-088-2
89) NicdcriillSki, A. THE GLOBAL ERROR APPROACII TO THE CONVERGENDE OF CLOSED-LOOP IDENTIFICATION. SELF-TUNINC REWJLATORS AND SELF-TUNING PREDICTORS. 'I'll-Report 78-E-X9. 1'178, ISBN 90-1> I 44-0X9-0
90) Vi lick, A.J. alld A.J.p, dc Pacpc REDUCING Till' NUMBFR OF COMPliTATIONS IN STACK DECODING OF CONVOLUTIONAL CODES BY 1':Xpl,OITING SYMME I'RIES OF THE ENCODER. TII-Reporl 78-1'.-90. 1'178. ISBN lJO-h 144-090-4
9 I) Gculjcs. A.J. and \).J. Klcyn A PARAMETRIC STU\)Y OF 1000 MWe COMBINED CLOSED CYCLE MHD/STEAM ELECTRICAL POWER GENERATING PLANTS. Til-Report 78-E-91. 1'178. ISBN 90-hI44-091-2
92) Massec, 1'. Till' DISPERSION RELATION OF ELECTROTHERMAL WAVES IN A NONEQUILIllRIUM MilD PLASMA. Til-Report 7X-E-92. 1'I7X. ISBN 'IO-h 144-092-0
-
UNIlIIOVEN UNIVEHSfTY OF TlTIINOLO(;Y TIlE NEIIIERLANUS DEPARTMENT OF ELHTRICAL ENGINEERING
~JJ) Delln, C.A. van
1J.l POLE SCl\'l"l'EHING OF ":U:Cl'HOMAGNE'l'IC WAVES PHOI'A(;ATION THHOUGH A RAIN
MEDll:M. '1'11- Heport }