kalman like filter 2011

8/6/2019 Kalman Like Filter 2011

1/24Copyright (c) 2011 IEEE. Personal use is permitted. For any other purposes, Permission must be obtained from the IEEE by emailing [email protected].

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. XX, NO. XX, XXXX XXXX 1

An Iterative Kalman-Like Algorithm Ignoring

Noise and Initial Conditions

Yuriy S. Shmaliy, Fellow, IEEE

Abstract

We address a p-shift finite impulse response (FIR) unbiased estimator (UE) for linear discrete time-

varying filtering (p = 0), p-step prediction (p > 0), and p-lag smoothing (p < 0) in state space with

no requirements for initial conditions and zero mean noise. A solution is found in a batch form and

represented in a computationally efficient iterative Kalman-like one. It is shown that the Kalman-like

FIR UE is able to outperform the Kalman filter if the noise covariances and initial conditions are not

known exactly, noise is not white, and both the system and measurement noise components need to be

filtered out. Otherwise, the errors are similar. Extensive numerical studies of the FIR UE are provided

in Gaussian and non-Gaussian environments with outliers and temporary uncertainties.

I. INTRODUCTION

The terms Kalman-like and Kalman-type are often used whenever the standard Kalman filter (KF)

[1] is modified to estimate state of a nonlinear model, under unknown initial conditions, and in the

presence of nonwhite or multiplicative noise sources. In such unsuited applications for KF, the Kalman-

like one is designed to save the recursive structure, while connecting the algorithm components with

the model in different ways. Because there can be found an infinity of Kalman-like solutions depending

on applications, we meet a number of propositions suggesting some new qualities while saving (or not

deteriorating substantially) the advantages of KF: fast computation and accuracy. Below, we observe most

recognized Kalman-like solutions.

Soon after the linear Kalman algorithm was published, Cox in [2] and others have derived the extended

KF (EKF) for nonlinear models by a linearization of the state-space equations. Referring to the fact that

Manuscript received...; revised...

Copyright (c) 2011 IEEE. Personal use of this material is permitted. However, permission to use this material for any other

purposes must be obtained from the IEEE by sending a request to [email protected].

The author is with the Department of Electronics, Guanajuato University, Mexico (e-mail: [email protected]).

March 9, 2011 DRAFT





EKF can give particularly poor performance when the model is highly nonlinear [3], Julier and Uhlmann

employed in [4] the unscented transform and proposed the unscented KF (UKF). Both EKF and UKF

have then been used extensively and the former was developed in [5] to the invariant EKF for nonlinear

systems possessing symmetries (or invariances). For high-dimensional systems, the ensemble KF was

proposed by Evensen in [6] and, for systems with sparse matrices, the fast KF applied by Lange in

[7]. Applications has also found the robust Kalman-type filter designed by Masreliez [8] [9] for linear

state-space relations with non-Gaussian noise referred to as heavy tailed noise or Gaussian one mixed

with outliers. An essential contribution to robust and highly robust estimation was also made by Martin,

Yohai, Rousseau, Maronna [10], and others.

Several particular modifications have also been addressed. In [11], Nahi derived an algorithm for

uncertain observations by including the multiplicative noise component to the measurement matrix. For

hidden Markov trees, the efficient restoration Kalman-like algorithm was discussed in [12] by Basseville

et al. Implying nonlinear modeling for hidden Markov chains, the KF was modified in [13] by Baccarelli

and Cusani to have the gain dependent on the observations. Most recently, Ait-El-Fquih and Desbouvries

applied in [14] the Kalman-like approach to triple Markov chains. For sparse signals with unknown and

time-varying sparsity patterns, Kalman filtering was developed by Vaswani in [15] and Carmi, Guifil and

Kanevsky in [16]. The nonlinear recursive Kalman-type estimator was addressed in [17] by Baccarelli,

Cusani, and Galli for mobile radio communications. We meet a new Kalman-like tracking algorithm

[18] applied to the autoregressive channel process estimation with fading by Stefanatos and Katsaggelos.

Different kinds of the Kalman-like algorithms can also be found in the area of control. Becis-Aubrya,

Boutayebb, and Darouach discussed in [19] the two-step estimation algorithm with a switching gain

matrix. In [20], Carli et al. employed the concept of the centralized KF for state estimation in the

complex sensor networks. Rawicz et al. reported in [21] explicit Kalman-Bucy-like solutions for two-

state H and H2 filters. The Kalman-Bucy-like state observers have gained currency as well [22], [23],

and the list of developments can be extended.

In spite of great efforts in extending the applications and improving the performance of KF, its structurestill remains recursive thus has the infinite impulse response (IIR). Investigating in [3] both the IIR and

finite impulse response (FIR) filters, Jazwinski concluded that the limited memory filter (having FIR)

appears to be more robust against the unbounded perturbation in the system. Referring to [3], optimal FIR

filtering has then been developed by W. H. Kwon et al. to the theory of receding horizon control [24].

There were also proposed several Kalman-like FIR estimators. A receding horizon Kalman FIR filter was

designed by Kwon, Kim, and Park in [25]. In [26], Han, Kwon, and Kim suggested a relevant algorithm

March 9, 2011 DRAFT





for deterministic time-invariant control systems and, in [27], Shmaliy derived an iterative algorithm for

the p-shift time-invariant FIR unbiased estimator (UE). A distinctive feature of such algorithms is that

white Gaussian noise in the convolution-based estimate obtained over N past measured points is reduced

as a reciprocal ofN [28] disregarding the model [27]. Moreover, the unbiased and optimal FIR estimates

typically become strongly consistent if N occurs to be large [29] or the mean square initial state function

dominates the noise covariance functions in the order of magnitudes [27]. It is also known that the optimal

horizon Nopt makes the FIR estimate (optimal or unbiased) similar or even better than the Kalman one

[24], [27][32].

Owing to the exciting engineering features of the Kalman-like FIR algorithms uniting advantages of

KF and inherent properties of FIR structures such as the bounded input/bounded output (BIBO) stability

as well as better robustness against temporary model uncertainties, non Gaussian noise, and round-off

errors, it may be expected their wide application in engineering practice. It can also be expected that

FIR UE ignoring noise and initial conditions will serve efficiently instead of optimal filters in many

applications.

In this paper, we extend in part the results of [27] to the more general time-varying case and address a

p-shift discrete linear Kalman-like FIR UE intended for filtering (p = 0), p-step prediction (p > 0), and

p-lag smoothing (p < 0) in state space with no requirements for initial conditions and zero mean noise.

The rest of the paper is organized as follows. In Section II, we describe the model and formulate the

problem. The time-variant p-shift batch unbiased FIR estimator is derived in Section III and developed

in an iterative Kalman-like form in Section IV. The estimate errors are discussed in Section V along

with the noise power gain and error bounds. Section VI gives examples of filtering and prediction of the

polynomial and harmonic state-space models in Gaussian and non-Gaussian environments with outliers

and uncertainties. Finally, concluding remarks are drawn in Section VII.

I I . SIGNAL MODEL AND PROBLEM FORMULATION

Consider a class of discrete time-varying (TV) linear state-space models represented with the state and

observation equations, respectively,

xn = Anxn1 + Bnwn , (1)

yn = Cnxn + Dnvn , (2)

where xn K and yn

M are the state and observation vectors, respectively. Here, An KK,

Bn KP, Cn

MK, and Dn MM. The state noise vector wn

P and measurement noise

March 9, 2011 DRAFT





An

Gain

Actualsignal

Time

AnN+2

An N+2 Gain

AveraginghorizonofN points

An

Es

tima

te

Recursivestrategyofthe KF

IterativestrategyoftheKalman-likeFIRUF

n N+- 1 n

opt

Fig. 1. Strategies of the recursive KF and iterative Kalman-like FIR UF algorithms.

vector vn M have zero mean components, E{wn} = 0 and E{vn} = 0. It is implied that these

vectors are mutually uncorrelated, E{wivTj } = 0, and have arbitrary distributions and known covariances

Qw(i, j) = E{wiwTj } , (3)

Qv(i, j) = E{vivTj } , (4)

for all i and j, to mean that wn and vn can not obligatorily be Gaussian and delta-correlated.

Following the strategies of the recursive KF [1] and iterative Kalman-like FIR unbiased filter (UF)

[27], the TV estimates of xn can be obtained as shown in Fig. 1. Here, KF starts at some initial point

n N + 1, where N 2, with known initial conditions and recursively produces estimates at each

subsequent point up to n. The estimate is formed in two steps. First, the system matrix AnN+2 makes

a projection from n N+ 1 to n N+ 2 and then the Kalman gain adjusts the result to be the estimate

at n N + 2. The procedure repeats recursively, provided the covariances of white noise sequences.

The Kalman-like FIR UF does not need the noise covariances and initial errors [30], [33]. Instead,

it relies on an optimal averaging horizon Nopt [34] in order to minimize the mean square error (MSE)

[27], [31]. It starts with any value at n N+ 1 and iteratively produces estimates at subsequent points intwo steps, similarly to KF, with a true output at n. The algorithm can operate in any noise environment

that makes it highly attractive for engineering applications. A payment for these advantages, an about N

times larger computation time, will not be necessary in parallel computing.

Let us now formulate the problem. Given the discrete TV state-space linear model (1) and (2), we

would like to design a general p-shift Kalman-like UE for filtering (p = 0), p-step prediction (p > 0),

and p-lag smoothing (p < 0) without the requirements for the initial conditions and zero mean noise. We

March 9, 2011 DRAFT





also wish to investigate the trade-off of this estimator with the Kalman one in Gaussian and non-Gaussian

environments in the presence of outliers and temporary uncertainties.

III. TIM E-VARYING BATCH UNBIASED FIR ESTIMATOR

In order to find the FIR UE, (1) and (2) can be extended on a horizon of N past points from m =

n N + 1 to n following [35], [36] and similarly to [29] as, respectively,

Xn,m = An,mxm + Bn,mWn,m , (5)

Yn,m = Cn,mxm + Gn,mWn,m + Dn,mVn,m , (6)

where Xn,m KN, Yn,m

MN, Wn,m PN, and Vn,m

MN are specified by, respectively,

Xn,m = xTn x

Tn1 . . . x

Tm

T

, (7)

Yn,m =

yTn yTn1 . . . y

Tm

T, (8)

Wn,m =

wTn wTn1 . . . w

Tm

T, (9)

Vn,m =

vTn vTn1 . . . v

Tm

T, (10)

and An,m KNK, Cn,m

MNK, Gn,m MNPN, and Dn,m

MNMN are given with,

respectively,

An,m = Am+1T

n,0 Am+1T

n,1 . . . ATm+1 I

T

, (11)

Cn,m =

CnAm+1n,0

Cn1Am+1n,1

...

Cm+1Am+1

Cm

, (12)

Gn,m = Cn,mBn,m , (13)

Dn,m = diagDn Dn1 . . . Dm N

, (14)

where we assigned

Argr,h =

gi=h

Ari ,

Cn,m = diag

Cn Cn1 . . . Cm N

,

March 9, 2011 DRAFT





Bn,m =

Bn AnBn1 . . . Am+2n,0 Bm+1 A

m+1n,0 Bm

0 Bn1 . . . Am+2n,1 Bm+2 A

m+1n,1 Bm+1

......

. . ....

...

0 0 . . . Bm+1 Am+1Bm0 0 . . . 0 Bm

, (15)

and performed Bn,m KNPN as (15) (see next page).

The model, (5) and (6), suggests that the state equation at the initial point m is xm = xm + Bnwm

that for Bn specialized with (15) can uniquely be satisfied with wm zero-valued. The initial state xm

must thus be known a priori or estimated optimally a posteriori as shown in [27].

By the discrete convolution, the estimate1 xn+p|n of xn can now be obtained if we assign a K M N

gain matrix Hn,m(p) and claim that

xn+p|n = Hn,m(p)Yn,m (16a)

= Hn,m(p)(Cn,mxm + Gn,mWn,m

+Dn,mVn,m) . (16b)

The estimate (16a) will be unbiased if and only if the following unbiasedness condition is satisfied

E{xn+p|n} = E{xn+p} , (17)

where E means averaging of the succeeding relation.

Averaging in (16b), by (17), means removing the zero mean noise matrices that gives us E{xn+p|n} =

Hn,m(p)Cn,mxm, where Hn,m(p) is the FIR UE gain. In turn, E{xn} can be substituted with the first

vector row of (5) by removing noise as E{xn} = Am+1n,0 xm. Since n can be arbitrary, one can substitute

it with n + p and write

E{xn+p} = Am+1

n+p,0xm . (18)

Equating E{xn+p|n} to (18) leads to the unbiasedness constraint for TV models

Am+1n+p,0 = Hn,m(p)Cn,m . (19)

1xn+p|n is an estimate at n + p via measurement from the past to n; xn+p|n mean optimal and xn+p|n unbiased.

March 9, 2011 DRAFT


7/24





TABLE I

FULL -H ORIZON TV KALMAN-L IK E FIR UE ALGORITHM

Stage

Given: K, p, n K

Set: n(p) by (I.9)

P = (CTK1,0CK1,0)1

FK1 = A1K1,0PA

1T

K1

xK+p1|K1 = A1K+p,1PC

TK1,0YK1,0

Update: Fn = [CTnCn + (AnFn1A

Tn )1]1

xn+p|n = An+pxn+p1|n1 +An+p1n (p)FnC

Tn

[yn Cnn(p)xn+p1|n1]

where s = m + K 1, m = n N + 1, and an iterative variable l ranges from m + K to n, because

CTl,mCl,m is singular with l < m + K. The true estimate corresponds to l = n.

Proof: Proof is postponed to Appendix I.

As can be seen, (22) is the Kalman estimate, in which Al+p1l (p)FlC

Tl plays the role of the Kalman

gain that, however, does not depend on noise and initial conditions. The algorithm has two batch forms,

(24) and (25), which can be computed fast for typically small K.

A. Full-Horizon Time-Varying Kalman-Like Estimator

In special cases when noise is nonstationary or both the system and measurement noise components

need to be filtered out, all the data available should be processed. By letting N = n+1 and l = n K in

(22)(26), the relevant full-horizon algorithm becomes as shown in Table I and, for TI models, simplifies

to that proposed in [27, Eqs. (76)(81)]. As can be seen, the algorithm (Table I) requires only K and p,

thus has extremely strong engineering features.

V. ESTIMATE ERRORS

A. Mean Square Error

The MSE in the FIR UE can be ascertained at n + p by

J(p) = E{(xn+p xn+p|n)(xn+p xn+p|n)T} . (27)

March 9, 2011 DRAFT





If we substitute xn+p|n with (21b), xn+p with the p-shifted first vector row in (5),

xn+p = Am+1n+p,0xm + Bn+p,mWn+p,m ,

whereBn+p,m is the p-shifted first vector row in (15), account for the orthogonality condition [35], and

provide the averaging, then J(p) can be transformed to

J(p) = Hn,m(p)(Zw + Zv)HTn,m(p) + Zw(p)

2Zw(p)HTn,m(p) , (28)

in which

Zw = Gn,mE{Wn,mWTn,m}G

Tn,m ,

Zv = Dn,mE{Vn,mVTn,m}D

Tn,m ,

Zw(p) = Bn+p,mE{Wn+p,mWTn+p,m}B

Tn+p,m ,

Zw(p) = Bn+p,mE{Wn+p,mWTn,m}G

Tn,m ,

where the noise components in Wn+p,m associated with p-step prediction must be set to zero. As can be

seen, J(p) inherently becomes zero-valued when the noise matrices are zeroth. Provided (28), the optimal

horizon Nopt can further be found by minimizing the trace of J(p) by Hn,m(p), as shown in [34] for

polynomial models. An alternative way to minimize J(p) by Nopt was discussed in [27], although an

experimental determination ofNopt for a reference model still remains preferable for many authors. When

Nopt and sampling time need to be specified individually for each of the states, the thinning algorithms

can be used [37].

B. Noise Power Gain and Error Bound

Setting aside the rigorous MSE matrix (28), the estimate error bound (EB) can be found if we employ

the ratio of the output-to-input noise variances introduced for FIR filters by Blum in [38] and now known

as the noise power gain (NPG). As shown in [39], the NPG Kk Kk(n,N ,p) can be computed for the

March 9, 2011 DRAFT





kth state measured with noise having the variance 2k as

Kk = HkHTk , (29a)

=

Kk(11) . . . K k(1k) . . . K k(1K)...

. . ....

. . ....

Kk(k1) . . . K k(kk) . . . K k(kK)...

. . ....

. . ....

Kk(K1) . . . K k(Kk) . . . K k(KK)

.

(29b)

where the thinned K N gain Hk

Hn,m(p)k

is composed by Kth columns of Hn,m(p) starting

with the kth one. The main component Kk(kk) occupies a central place in (29b), representing the k-to-

k channel. Other important ones, Kk(vv), v = k, placed on the main diagonal characterize the k-to-v

channels. The remaining ones Kk(vg), g = k = v, g [1, K], play rather an auxiliary role, representing

interactions in the estimator channels and completing the noise reduction picture.

Provided Kk, the EB can be specialized in the three-sigma sense with

k(vg)(n,N ,p) = 3kK1/2k(vg)

(n,N ,p) (30)

to characterize denoising in the vtog channel via measurement of the kth state. In what follows, we

shall show that (30) can serve as an efficient measure of errors in FIR UE.

VI . EXAMPLES OF APPLICATIONS

Below, we examine estimates obtained with the Kalman-like FIR UE and Kalman algorithms applied

to basic linear TV models in different noise environments. All the way, we lay stress on the trade-off

between these estimates in order to sketch a guide for users. Measured the first state, EB for the first and

second states is computed below via (30) as 1(11)(n,N ,p) and 1(22)(n,N ,p), respectively. Numerical

comparisons of KF and FIR UE can also be found in [27][31], [33], [36], [40], [41].

A. Two-State Polynomial Model

In this section, we provide filtering (p = 0) and prediction (p > 0) of the two-state polynomial model,

(1) and (2), specified with Bn = I, Dn = I, Cn = [ 1 0 ], and

An =

1 (1 + dn)

0 1

, (31)

March 9, 2011 DRAFT





where is a constant in units of time, dn temporary takes different values. Such a situation occurs in

oscillators undergoing temporary frequency jumps or in moving vehiculars with velocity jumps. In

TI filtering, dn is uncertain and, for the TV case, we set it exactly. Noise is mostly allowed to be white

Gaussian and we notice that the relevant investigations for the uniformly distributed and highly intensive

sawtooth noise were provided in [28], [33], [42].

1) Filtering of a Distinct Model: In the first experiment, we let dn = 20 if 120 n 160 and

dn = 0 otherwise. The noise variances in the first and second states are allowed to be 21 = 10

4 and

22 = 4 102/s2, respectively. Measurement was organized with noise having 2v = 50

2 and the process

simulated at 340 subsequent points. For the Kalman-like FIR UF, optimum horizons were found to be

N = 54 and N = 12 beyond and within the region of variations, respectively, provided the minimum

MSE in the estimate.

Figure 2 sketches typical estimates obtained with the TV and TI filters. As can be seen in Fig. 2a,

the TI KF and Kalman-like FIR UF produce large errors caused by dn = 0. In contrast, the difference

between the TV ones is poorly distinguishable and we go to Fig. 2b. Here, the TV Kalman-like filter

demonstrates an important peculiarity that still has not been discussed in the literature. Because dn varies

the model, Nopt is also varied. In fact, set N = 54, errors in the TV Kalman-like filter have appeared

to be larger in the region of variations. However, both filters produced similar errors with N = 12 in

this region. Another difference is that the transient is inherently finite in the Kalman-like filter (limited

with N) and infinite in KF. Fig. 2c sketches filtering errors in the second state. The TV Kalman-like FIR

UF is low sensitive to N in this case and we watch for similar behaviors of two filters, except for the

inherently longer transient in TI KF.

It can also be observed that all TV estimates trace well within a gap between EB and EB formed

by 1(11)(n, 54, 0) and 1(22)(n, 54, 0).

2) Filtering with Errors in Noise Covariances: To comprehend the effect of errors in noise covariances,

we repeat the experiment for x10 = 1, x20 = 0.01/s, 21 = 10

4, 22 = 4 106/s2, and v = 0.15.

Here, we set dn = 5 from 160 to 200 and dn = 0 otherwise. Because the system noise is often hard todetermine exactly, we reduce the standard deviations in the first and second states by the factors of 2

and 4, respectively, to have 21 = 0.25 104 and 22 = 0.25 10

6/s.

Figure 3 gives us a typical reproducible example of errors in the KF and Kalman-like FIR UF for

this case. Immediately, one notes that the Kalman-like filter outperforms the Kalman one in both the TI

case (Fig. 3a) and TV one (Fig. 3b), as being independent on noise performance. Here, EBs are sketched

by 1(11)(n, 54, 0) and 1(22)(n, 54, 0) when 0 n 160 and n 200 and by 1(11)(n, 14, 0) and

March 9, 2011 DRAFT





1(22)(n, 14, 0) for 160 < n < 200.

3) Prediction of a Distinct Model: It follows from (22) that the Kalman-like FIR unbiased prediction

can be organized at any fixed n, by increasing a step p > 0. In turn, the Kalman prediction can be

obtained if to project the estimate from n to n + p as follows:

xn+p|n = An+1n+p,0xn|n . (32)

The process was simulated with x10 = 1, x20 = 0.01/s, 21 = 10

6, 22 = 104/s2. Measurement

was provided with the uniformly distributed non-Gaussian noise having 2v = 0.04 and associated with

sawtooth induced by the Global Positioning System timing receiver [36]. Here, we set dn = 20 from 150

to 152 and dn = 0 otherwise.

Figure 4a shows the predicted behaviors for known variances and initial conditions. Instead, Fig. 4b

sketches the trends affected by not fully known ones, in which case we increase the standard deviations in

the first and second states by the factors of 2 and 4, respectively, and allow for x10 = 2 and x20 = 0.03/s.

We see that the Kalman-like predictor outperforms the Kalman one, although the estimate difference is

most brightly pronounced in Fig. 4b.

4) Filtering of a Gaussian model with outliers: We finally examine the TI Gaussian model by

generating a process with x10 = 1, x20 = 0.01/s, 21 = 10

4, 22 = 4 106/s2, and 2v = 0.0225.

Assuming that the system noise is not known exactly, we increase 1 and 2 by the factors of 2 and 3,

respectively, add to the measurement seven outliers as shown in Fig. 5a, and provide filtering. It then

comes out that the Kalman-like filter exhibits better robustness against outliers (Fig. 5b and Fig. 5c).

B. Harmonic Model

We further examine a two-state harmonic model, (1) and (2), with the following vectors and matrices:

wn = [ wn ], vn = [ vn ], Bn = [ 1 1 ]T, Dn = [ 1 ], Cn = [ 1 0 ], and

An =

cos64

+ n sin

64

sin 64 cos 64 + n

, (33)

where n is TV. Such a model is often exploited to estimate different kinds of oscillatory and vibrating

systems in steady state. The process was generated with x10 = 100, x20 = 102, 2w = 1, and

2v = 100.

We allowed n = 0.04 from 350 to 400 and n = 0 otherwise. An optimal horizon was found here to be

Nopt = 111.

March 9, 2011 DRAFT





1) Filtering with Uncertainties: Figure 6 sketches filtering estimates and errors. In the initial time

interval up to 350 (Fig. 6a), both filters produce negligible errors. Then n = 0 causes excursions and

transients, although the TI Kalman-like FIR UF reaches a new state much earlier (during Nopt = 111

points, starting with 400) than the TI KF (Fig. 6b). In TV filtering, both filters exhibit similar errors (Fig.

6c) with a bit better performance in the Kalman-like case.

2) Full-Horizon Filtering of a Distinct Model: We finish our experiments employing the full-horizon

filtering algorithm (Table I, case p = 0). As it was shown in [27], [40], such an algorithm allows for

filtering out both the measurement and system noise components. It also suggests the best unbiased fit

if one fixes n and changes p [43].

Figure 7 gives us a typical example for n = 0 in (33) over all n. One infers that the full-horizon

Kalman-like FIR UF tracks the noise-free model, whereas the KF inherently follows its actual behavior

affected by system noise. Three typical regions can be recognized in Fig. 7b. With small N < 30

insufficient for effective noise reduction, KF produces lower errors. In the intermediate range, 30 < N 50, the FIR filter

outperforms the Kalman one. This effect, well reproducible disregarding the model, can be employed to

determine Nopt for the fixed-horizon FIR filters.

VII. CONCLUSION

In this paper, we proposed and investigated a Kalman-like FIR UE intended for filtering (p = 0),

p-step prediction (p > 0), and p-lag smoothing (p < 0) of discrete TV state-space models, ignoring noise

and initial conditions. The most common conclusions that can be made are the following. In the ideal

case of a model, initial conditions, and white noise covariances, all known exactly at each time point,

the optimal Kalman filter outperforms the unbiased FIR one typically on several percents in terms of

the MSE. The latter is able to outperform the former otherwise and if both the system and measurement

noise components need to be filtered out. Examples considered have also demonstrated better robustness

of FIR UE against KF, although we made no efforts to improve this performance. A natural payment for

these advantages is an about N times larger consumption of the computation time in the iterative FIR

procedure. This shortcoming can easily be circumvented by parallel computing that makes the algorithm

proposed highly attractive for engineering applications.

The most critical problem still faced in FIR estimators is a lack of a universal analytical relation for

Nopt minimizing MSE. Therefore, Nopt is commonly set by the authors experimentally for some reference

March 9, 2011 DRAFT





models. Since the latter is not always available, wide applications of FIR estimators can be expected if

to ascertain Nopt rigorously and in engineering approximations that is currently under investigation.

APPENDIX IITERATIVE KALMAN-L IK E FORM OF FIR UE

Consider (21b), substitute n with an iterative variable l ranging from m + K to n, and rewrite (21b)

as

xl+p|l = Am+1l+p,0(C

Tl,mCl,m)

1CTl,mYl,m . (I.1)

Following Appendix I in [27], assign

P1l = CTl,mCl,m , (I.2)

and expand P1l as

P1l = P1l1 + A

m+1T

l,0 CTl ClA

m+1l,0 . (I.3)

By employing the matrix inversion lemma [44],

(B + D)1 = B1 B1(I + DB1)1DB1 , (I.4)

the inverse of (I.3) can now be represented as

Pl

= Pl1

Pl1

I + Am+1T

l,0 CTl ClA

m+1l,0 Pl1

1

Am+1T

l,0 CTl ClA

m+1l,0 Pl1 . (I.5)

Next, expand CTl,mYl,m to

CTl,mYl,m = CTl1,mYl1,m +

ClA

m+1l,0

Tyl , (I.6)

rewrite (I.1) at l + p 1 as

xl+p1|l1 = Am+1l+p,1(CTl1,mCl1,m)1CTl1,mYl1,m ,

use the identities produced by (I.2) and (I.3), respectively,

P1l1 = CTl1,mCl1,m ,

P1l1 = P1l A

m+1T

l,0 CTl ClA

m+1l,0 ,

March 9, 2011 DRAFT





and express CTl1Yl1,m as

CTl1Yl1,m =

P1l Am+1T

l,0 CTl ClA

m+1l,0

Am+1

l+p,11

xl1+p|l1 . (I.7)

After the routine transformations, (I.1) becomes

xl+p|l = Al+pxl+p1|l1 + Am+1l+p,0PlA

m+1T

l,0 CTl

yl Cll(p)xl+p1|l1

, (I.8)

where l(p) is specified with

l(p) =

Al|p|l,0 , p 1 (smoothing)

Al , p = 0 (filtering)I , p = 1 (prediction)

p1i=1

A1li , p 2 (prediction)

. (I.9)

Finally, assign

Fl = Am+1l,0 PlA

m+1T

l,0 (I.10)

and transform (I.7) to (22), where Fl is given by (23) that appears by combining (I.4), (I.5), and (I.10).

REFERENCES

[1] R. E. Kalman, A new approach to linear filtering and prediction problems, Journal of Basic Engineering, vol. 82, no.

1, pp. 3545, Mar. 1960.

[2] H. Cox, On the estimation of state variables and parameters for noisy dynamic systems, IEEE Trans. Autom. Contr.,

vol. 9, no. 1, pp. 512, Jan. 1964.

[3] A. H. Jazwinski, Stochastic Processes and Filtering Theory, New York: Academic Press, 1970.

[4] S. J. Julier and J. K. Uhlmann, A new extension of the Kalman filter to nonlinear systems, in Proc. of SPIE, vol. 3068,

pp. 182-193, 1997.

[5] S. Bonnabel, Ph. Martin and E. Salaun, Invariant Extended Kalman Filter: theory and application to a velocity-aided

attitude estimation problem, in Proc. of 48th IEEE Conference on Decision and Control, pp. 12971304, 2009.

[6] G. Evensen, Sequential data assimilation with nonlinear quasi-geostrophic model using Monte Carlo methods to forecast

error statistics, Journal of Geophysical Research, vol. 99, no. C5, pp. 143162, May 1994.

[7] A. A. Lange, Simultaneous statistical calibration of the GPS signal delay measurements with related meteorological

data, Physics and Chemistry of the Earth, Part A: Solid Earth and Geodesy , vol. 26, no. 6-8, pp. 471473, 2001.

[8] C. J. Masreliez, Approximate non-Gaussian filtering with linear state and observation relations, IEEE Trans. Autom.

Contr., vol. AC-20, no. 1, pp. 107110, Feb 1975.

March 9, 2011 DRAFT





[9] R. D. Martin and C.J. Masreliez, Robust estimation via stochastic approximation, IEEE Trans. Inform. Theory, vol.

IT-21, no. 3, pp. 263271, May 1975.

[10] R. A. Maronna and V. J. Yohai, Robust Statistics Theory and Methods, New York: J. Wiley, 2006.

[11] N. E. Nahi, Optimal recursive estimation with uncertain observation, IEEE Trans. Inform. Theory, vol. IT-15, no. 4,

pp. 457462. Jul. 1969.

[12] M. Basseville, A. Benveniste, K. C. Chou, S. A. Golden, R. Nikoukhah, and A. S. Willsky, Modeling and estimation of

multiresolution stochastic processes, IEEE Trans. Inf. Theory, vol. 38, no. 2, pp. 766784, Mar. 1992.

[13] E. Baccarelli and R. Cusani, Recursive Kalman-type optimal estimation and detection of hidden Markov chains, Signal

Process., vol. 51, no. 1, pp. 5564. May 1996.

[14] B. Ait-El-Fquih and F. Desbouvries, Kalman Filtering in Triplet Markov Chains, IEEE Trans. Signal Process., vol. 54,

no. 8, pp. 29572963, Aug. 2006.

[15] N. Vaswani, Kalman filtered compressed sensing, in Proc. of 15th IEEE Int. Conf. on Image Process., pp. 893896,

2008.

[16] A. Carmi, P. Gurfil, and D. Kanevsky, Methods for sparse signal recovery using Kalman filtering with embedded pseudo-measurement norms and quasi-norms, IEEE Trans. on Signal Process., vol. 58, no. 4, pp. 24052409, Apr. 2010.

[17] E. Baccarelli, R. Cusani, and S. Galli, A novel adaptive receiver with enhanced channel tracking capability for TDMA-

based mobile radio communications, IEEE J. Select. Areas in Commun., vol. 16, no. 9, pp. 16301639, Dec. 1998.

[18] S. Stefanatos and A. K. Katsaggelos, Joint data detection and channel tracking for OFDM systems with phase noise,

IEEE Trans. Signal Process., vol. 56, no. 9, pp. 42304243, Sep. 2008.

[19] Y. Becis-Aubry, M. Boutayeb, and M. Darouach, State estimation in the presence of bounded disturbances, Automatica,

vol. 44, no. 7, pp. 18671873, Jul. 2008.

[20] R. Carli, A. Chiuso, L. Schenato, and S. Zampieri, Distributed Kalman filtering based on consensus strategies, IEEE J.

on Selected Areas in Commun., vol. 26, no. 4, pp. 622633, May 2008.

[21] P. L. Rawicz, P. R. Kalata, K. M. Murphy, and T. A. Chmielewski, Explicit formulas for two state Kalman, H2 and

H target tracking filters, IEEE Trans. Aerospace and Electron Syst., vol. 39, no. 1, pp. 5369, Jan 2003.

[22] D. G. Luenberger, Observing the state of a linear system, IEEE Trans. On Military Electron., vol. 8, no. 2 pp. 7480,

Apr. 1964.

[23] J. P. Gauthier, H. Hammouri, and S. Othman, Observability for any u(t) of a class of bilinear systems, IEEE Trans.

Automat.Contr., vol. 37, no. 6, pp. 875880, Jun. 1992.

[24] W. H. Kwon and S. Han, Receding horizon control: model predictive control for state models , London: Springer, 2005.

[25] W. H. Kwon, P. S. Kim, and P. Park, A receding horizon Kalman FIR filter for discrete time-invariant systems, IEEE

Trans. Automatic Control vol. 44, no. 9, pp. 17871791, Sep. 1999.

[26] S. H. Han, W. H. Kwon, and P. S. Kim, Quasi-deadbeat minimax filters for deterministic state-space models, IEEETrans. Automat.Contr., vol. 47, no. 11, pp. 19041908, Nov. 2002.

[27] Y. S. Shmaliy, Linear optimal FIR estimation of discrete time-invariant state-space models, IEEE Trans. Signal Process.,

vol. 58, no. 6, pp. 30863096, Jun. 2010.

[28] Y. S. Shmaliy, An unbiased FIR filter for TIE model of a local clock in applications to GPS-based timekeeping, IEEE

Trans. on Ultrason., Ferroel. and Freq. Contr., vol. 53, no. 5, pp. 862870, May 2006.

[29] Y. S. Shmaliy, Optimal gains of FIR estimators for a class of discrete-time state-space models, IEEE Signal Process.

Letters, vol. 15, pp. 517520, 2008.

March 9, 2011 DRAFT





[30] P. S. Kim and M. E. Lee, A new FIR filter for state estimation and its applications, J. of Computer Science and Techn.,

vol. 22, no. 5, pp. 779784, Sep. 2007.

[31] P. S. Kim, An alternative FIR filter for state estimation in discrete-time systems, Digital Signal Process., vol. 20, no.

3, pp. 935943, May 2010.

[32] W. H. Kwon, P. S. Kim, and S. H. Han, A receding horizon unbiased FIR filter for discrete-time state space models,

Automatica, vol. 38, no. 3, pp. 545551, Mar. 2002.

[33] Y. S. Shmaliy, Unbiased FIR filtering of discrete-time polynomial state-space models, IEEE Trans. Signal Process., vol.

57, no. 4, pp. 12411249, Apr. 2009.

[34] Y. S. Shmaliy, J. Munoz-Diaz, and L. Arceo-Miquel, Optimal horizons for a one-parameter family of unbiased FIR

filters, Digital Signal Process., vol. 18, no. 5, pp. 739750, Sep. 2008.

[35] H. Stark and J. W. Woods, Probability, Random Processes, and Estimation Theory for Engineers , 2nd ed., Upper Saddle

River, NJ: Prentice Hall, 1994.

[36] Y. S. Shmaliy, GPS-based Optimal FIR Filtering of Clock Models, New York: Nova Science Publ., 2009.

[37] Y. S. Shmaliy, O. Ibarra-Manzano, L. Arceo-Miquel, and J. Munoz-Diaz, A thinning algorithm for GPS-based unbiasedFIR estimation of a clock TIE model, Measurement, vol. 41, no. 5, pp. 538550, Jun. 2008.

[38] M. Blum, On the mean square noise power of an optimum linear discrete filter operating on polynomial plus white noise

input, IRE Trans. Information Theory, vol. 3, no. 4, pp. 225231, Dec. 1957.

[39] Y. S. Shmaliy, O. Ibarra-Manzano, Noise power gain for discrete-time FIR estimators, IEEE Signal Process. Letters,

vol. 18, no. 4, pp. 207210, Apr. 2011.

[40] Y. S. Shmaliy, Full horizon optimal FIR estimator of clock state employing measurement of time errors, Int. J. Electron.

Commun. (AEU), vol. 65, no. 2, pp. 117-122, Feb. 2010.

[41] Y. S. Shmaliy and O. Ibarra-Manzano, Optimal FIR filtering of the clock time errors, Metrologia, vol. 45, no. 5, pp.

571576, Sep. 2008.

[42] Y. S. Shmaliy, A simple optimally unbiased MA filter for timekeeping, IEEE Trans. Ultrason. Ferroel. Freq. Control,

vol. 49, no. 6, pp. 789797, Jun. 2002.

[43] Y. S. Shmaliy, Linear unbiased prediction of clock errors, IEEE Trans. Ultrason. Ferroel. Freq. Control, vol. 56, no. 9,

pp. 20272029, Sep. 2009.

[44] G. H. Golub and G. F. van Loan, Matrix Computations, 3rd Ed., Baltimore: The John Hopkins Univ. Press, 1996.

March 9, 2011 DRAFT





Yuriy S. Shmaliy (M96, SM00, F11) received the B.S., M.S., and Ph.D. degrees in 1974, 1976 and

1982, respectively, from the Kharkiv Aviation Institute, Ukraine, all in Electrical Engineering. He serves

as Full Professor beginning in 1986. In 1992 he received the Dc.Sc. degree from the Kharkiv Railroad

Institute. Since 1985 to 1999, he had been with the Kharkiv Military University. In 1992, he founded

the Scientific Center Sichron where he had been a director by 2002. Since 1999, he has been with the

Guanajuato University of Mexico.

Dr. Shmaliy has 273 Journal and Conference papers and 80 patents. His books Continuous-Time Signals (2006) and Continuous-

Time Systems (2007) were published by Springer. His book GPS-Based Optimal FIR Filtering of Clock Models (2009) was

published by Nova Science Publ., New York. He also contributed with invited chapters to several books. He was rewarded a

title, Honorary Radio Engineer of the USSR, in 1991. He was listed in Marquis Whos Who in the World in 1998; Outstanding

People of the 20th Century, Cambridge, England in 1999; and Contemporary Whos Who, American Bibliographical Institute,

in 2002. He has Certificates of Recognition and Appreciation from the IEEE, WSEAS, IASTED, and various Universities. He

is currently an Associate Editor in Recent Patents on Space Technology and Editorial Board Member in several Journals. He

organized and chaired several Int. Conferences on Precision Oscillations in Electronics and Optics and is a member of Organizingand Program Committees of various Int. Symposia. He was multiply invited to give tutorial, seminar, and plenary lectures. His

current interests include optimal estimation, statistical signal processing, and stochastic system theory.

March 9, 2011 DRAFT





0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340

-1

0

Measurement

Time

(a)

Firststate

TIKalman

TIKalman-like

Measu

remen

tan

des

tima

tes

,inparto

f10

3

-2

-3

TVKalman-like

TVKalman

0 20 40 60 100 120 140 160 180 200 220 240 260 280 300 320 3406?

4?

2?

0

2

4

6

N=54

TIKalman

TIKalman-like

N=54N=12

Time

(b)

First state

Estimateerrors

,inparto

f10

2

80

EB

-EB

TVKalman-like

TVKalman

0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 34060?

40?

20?

0

20

TIKalmanTIKalman-like

Secondstate

Estimateerrors

Time

(c)

N=54

-EB

EB

TVKalman-like

TVKalman

Fig. 2. Kalman-like FIR UF and KF estimates of the TV two-state polynomial model (Section VI-A): (a) measurement and

estimates of the first state, (b) filtering error of the first state, and (c) filtering errors of the second state.

March 9, 2011 DRAFT





0 20 40 60 80 1 00 1 20 14 0 1 60 1 80 2 00 220 2 40 26 0 2 80 3 00 320 340 3 60 3 80 4 002.0?

1.8?

1.6?

1.4?

1.2?

1.0?

0.8?

0.6?

0.4?

0.2?

0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Kalman (withreducednoisevariances)

Kalman-like

Time

(a)

N=14N=29

TIfiltering Firststate

N=29

Filter

ing

errors

Kalman

-EB

EB

0 20 40 60 8 0 10 0 12 0 1 40 1 60 1 80 2 00 2 20 2 40 2 60 2 80 3 00 3 20 3 40 3 60 380 4001.2?

1.0?

0.8?

0.6?

0.4?

0.2?

0

0.2

0.4

0.6

0.8

1.0

1.2

Time

(b)

FirststateTVfiltering

Filter

ing

errors

N=29 N=29

N=14

EB

-EB

Kalman-like

Kalman

Kalman (withreducednoisevariances)

Fig. 3. Errors of Kalman and Kalman-like FIR unbiased filtering of the first state of the model (Section VI-A): (a) TI filtering

and (b) TV filtering.

March 9, 2011 DRAFT





0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 3 005?

0

5

10

15

20

25

30

Time

(a)

Pre

dictio

n

ofth

e

fir

st

st

at

e

Actual

Kalman

Kalman-like

Distinctmodel N=16

0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 3005?

0

5

10

15

20

25

30

Errorsinthenoisevariancesandinitialconditions

Actual

Kalman

Kalman-like

Pre

dictio

n

ofth

e

firs

ts

ta

te

Time

(b)

N=16

Fig. 4. Typical errors of the Kalman and Kalman-like FIR unbiased prediction of model behavior (Section VI-A): (a) distinct

model and (b) errors in the noise variances and initial conditions.

March 9, 2011 DRAFT





0 50 100 150 200 250 300 350 400 450 500

3?

2?

1?

0

1

2

3

4

5

6

7

8

9

Time

Measurementofthefirststate

(1)

(2)

(3)

(5)

(6)

(7)(4)

Actualstate

Measurement

(a)

(b)

Es

tima

teerrors

0 50 100 150 200 250 300 350 400 450 5001?

0.8?

0.6?

0.4?

0.2?

0

0.2

0.4

0.6

0.8

1

Time

FirstState

Kalman

Kalman-like

EB

EB

(c)

Es

tima

teerrors

0 50 100 150 200 250 300 350 400 450 5000.15?

0.1?

0.05?

0

0.05

0.1

0.15

Kalman

Kalman-likeEB

EB

Time

SecondState

Fig. 5. Typical filtering of a Gaussian model (Section VI-A) with outliers employing the TI KF and TI Kalman-like FIR UF:

(a) measurement, (b) errors in the first state, and (b) errors in the second state.

March 9, 2011 DRAFT





0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 8008?

7?

6?

5?

4?

3?

2?

1?

0

1

2

3

4

5

6

7

8

Kalman

Kalman-like

Measurement

Time

(a)

N=111

Filte

ringes

tima

tes

,in

partso

f10

2

0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 8006?

5?

4?

3?

2?

1?

0

1

2

3

4

5

6

Kalman

Kalman-like

Time

(b)

TIfiltering N=111

Filte

ringes

tima

teerrors

,in

partso

f10

2

0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 80010?

9?

8?

7?

6?

5?

4?

3?

2?

1?

0

1

2

3

4

5

6

7

8

9

10

Kalman

Kalman-likeFilte

ringes

tima

teerrors

TVfiltering

Time

(c)

N=111

Fig. 6. Filtering of a harmonic model (Section VI-B) with the Kalman and Kalman-like FIR UE algorithms: (a) measurement

and TI estimates, (b) errors in the TI estimates, and (b) errors in the TV estimates.

March 9, 2011 DRAFT


24/24



0 25 50 7 5 10 0 1 25 1 50 1 75 2 00 22 5 2 50 2 75 30 0 3 25 3 50 3 75 40 0 4 25 4 50 4 75 500200?

175?

150?

125?

100?

75?

50?

25?

0

25

50

75

100

125

150

175

200

Filte

ring

es

tima

tes

Noise-freemodel

Measurement

Time

(a)

Full-horizonfiltering

Kalman-like

Kalman

0 25 50 75 100 125 150 175 200 225 250 275 300 32 5 3 50 375 400 425 450 4 75 50060?

54?

48?

42?

36?

30?

24?

18?

12?

6?

0

6

12

18

24

30

36

42

48

54

60

Kalman

Kalman-like

Es

tima

teerrors

Time

(b)

Equal

errors

Kalmanoutperforms

Kalman-like outperforms

Fig. 7. Full-horizon filtering of the harmonic model (Section VI-B): (a) measurement and estimates and (b) estimate errors.

March 9, 2011 DRAFT

kalman like filter 2011

Documents