Professor A G

Constantinides© 1

AGC

DSP

AGC

DSP

Modern Spectral Estimation Modern Spectral Estimation is based on a

priori assumptions on the manner, the observed process has been generated

Validity of these assumptions is taken to hold over all possible realisations and to be of infinite temporal extent.

Thus limitations of FFT-based methods are circumvented

These assumptions may be entirely statistical or deterministic model-based or both.

Professor A G

Constantinides© 2

AGC

DSP

AGC

DSP

Modern Spectral Estimation Statistical methods make assumptions on the

probabilities pertaining to data generation. Wiener-Hopf, and Bayesian methods are

typical examples Model-based deterministic methods assume

a linear or a non-linear equation for the input/output process driven by a stochastic or a deterministic signal.

Linear Predictive Least SquaresTechniques are typical of this class

Professor A G

Constantinides© 3

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Main directions are: Least Squares Maximum Entropy

Professor A G

Constantinides© 4

AGC

DSP

AGC

DSP

Modern Spectral Estimation An optimisation problem: Measurements: Problem:

Find the best FIR model to filter to yield a given signal

We need a) order of FIR system b) decide on how to measure “best fit”

]}[{ nx

]}[{ nd]}[{ nx

Professor A G

Constantinides© 5

AGC

DSP

AGC

DSP

Modern Spectral Estimation

“Order Estimation” is an area by itself Goodness of fit is another large area Usually:

we have some idea beforehand on the order

we select an “error criterion” which reasonably reflects reality and is analytically tractable

Professor A G

Constantinides© 6

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Formulation: Assume FIR order be and unknown filter weights

Output of FIR filter is

Instantaneous error is

N]}[{ nh

xhTN

rrnxrhny

0][][][

xhTndnyndne ][][][][

Professor A G

Constantinides© 7

AGC

DSP

AGC

DSP

Modern Spectral Estimation The best solution would be when all such

errors are zero. However, this may not possible because of many reasons e.g. the order is not correct, the actual model is not FIR, or is not linear, the noise present in the data, etc

Hence need to be selected to minimise some measure of the error.

]}[{ nh

Professor A G

Constantinides© 8

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Error measure can take many forms We draw a distinction between

stochastic and deterministic measures For example (a) Stochastic (b) Deterministic

}][{min pneEJh

n

pneJ ][minh

Professor A G

Constantinides© 9

AGC

DSP

AGC

DSP

Modern Spectral Estimation

With Problem (a) is known as the Wiener

filtering problem Problem(b) is known as the Least

Squares problem These problems are also analytically

easily tractable

2p

Professor A G

Constantinides© 10

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Extensive work has been done in these problems in their various forms.

The absolute value squared error is

Or

**2 ][][][ xhxh HT ndndne

hhghhg THHHndne 22 ][][

Professor A G

Constantinides© 11

AGC

DSP

AGC

DSP

Modern Spectral Estimation Where for the stochastic case

While for the deterministic case we have the same expressions but Expectations are replaced by Summations.

)}()({ * jnxndE g

}{)}()({ *kjjnxknxE

Professor A G

Constantinides© 12

AGC

DSP

AGC

DSP

Modern Spectral Estimation

In both cases we have is the crosscorrelation between

the measurements (data) and the desired signal

is the autocorrelation matrix of the data

g

Professor A G

Constantinides© 13

AGC

DSP

AGC

DSP

Modern Spectral Estimation

The autocorrelation matrix for real signals is symmetric, positive definite

This is seen, for the stochastic case, from

Expanding

0}][][][][{ ** jnxknxjnxknxE

Professor A G

Constantinides© 14

AGC

DSP

AGC

DSP

Modern Spectral Estimation Differentiating with respect to

and setting the result to zero we obtain

Or

Differentiating again yields the autocorrelation matrix, which is positive definite and hence we have a minimum

2][ne h

hg T0

gh1

T

Professor A G

Constantinides© 15

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Differentiating with respect to and setting the result to zero we obtain

However,

2][ne h

hg T0 gh1

T

HTH

HTH

nd

ndne

xxhx

xxhx

][

][][

Professor A G

Constantinides© 16

AGC

DSP

AGC

DSP

Modern Spectral Estimation

On taking expectations we obtain

This is known as the orthogonality condition

“At the optimum the error vector is orthogonal to the data”

0}][{}][{ THTHH ndEneE hgxxhxx

Professor A G

Constantinides© 17

AGC

DSP

AGC

DSP

Modern Spectral Estimation For the stochastic case this solution is known

as the Wiener–Hopf solution. For the deterministic case the solution is

known as the Yule-Walker solution. The framework of modelling has been FIR or

Moving Average (MA). It can be extended to include more involved linear models such as Autoregressive (AR), and ARMA

Professor A G

Constantinides© 18

AGC

DSP

AGC

DSP

AR Spectral Estimation This is also known as the Maximum Entropy

Method and the Burg Method. Burg solved the problem of extrapolating a

given finite set of autocorrelations to an infinite set while keeping the autocorrelation matrix positive semidefinite.

In view of the infinite possibile solutions he postulated selecting that which produces the flattest PSD. Equivalently it maximises uncertainty (entropy) or randomness.

Professor A G

Constantinides© 19

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Thus the problem becomes the constrained optimisation problem

Subject to

deP j

mr

)(lnmax

][

][)( mrdeP xxmj

pm

pm0

Professor A G

Constantinides© 20

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Thus if the PSD of the observations is taken to be that of the output of an AR system driven by a white Gaussian process the problem reduces to finding the parameters of the following model

2221

2

...1)(

jNN

jj

jMEM

eaeaea

GeP

Professor A G

Constantinides© 21

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Where N is the number or poles. are obtainable in the

autocorrelation method from (N+1)X(N+1)

],...,,,[ 21 NaaaG

00

.0

.

1

]0[]1[...][

]1[.

.]1[]0[]1[

][...]1[]0[ 2

1

*

*

**G

a

a

rrNr

r

rrr

Nrrr

N

Professor A G

Constantinides© 22

AGC

DSP

AGC

DSP

Modern Spectral Estimation Where the autocorrelation sequence is

estimated as

The signal above is extended by padding with zeros whever the argument demands more samples.

1

0

* )()(1

][L

nlnxnx

Llr

Professor A G

Constantinides© 23

AGC

DSP

AGC

DSP

Modern Spectral Estimation

If we take only the entral part of the autocorrelation matrix containing no zero padding then we have the Covariance Method.

The signal vector in both cases may be windowed prior to the computations.

Professor A G

Constantinides© 24

AGC

DSP

AGC

DSP

Modern Spectral Estimation While the Burg method is a decided

improvement over the non-parametric methods, it has several disadvantages

1) Exhibits Spectral Line Splitting particularly at high SNR

2) For high order systems introduces spurious spectral peaks

3) In estimating sinusoids in noise it shows a bias dependent on the initial sinusoid phases

Professor A G

Constantinides© 25

AGC

DSP

AGC

DSP

Linear Prediction

Assume

From the measurements in conjunction with the assumed model we can write

])1[...]2[]1[(][ 121 Lnxanxanxanx L

Professor A G

Constantinides© 26

AGC

DSP

AGC

DSP

Linear Prediction

Na

a

a

a

Lx

LxLx

LxLxLx

NLxLxLxLx

xNxNxNx

xxx

xx

x

.

]1[0.00

]2[]1[0.0

]3[]2[]1[00

.....

]1[]4[]3[]2[

.....

]0[.]3[]2[]1[

.....

0]0[]1[]2[

00]0[]1[

000]0[

3

2

1

0

0

0

.

]1[

.

][

.

]3[

]2[

]1[

Lx

Nx

x

x

x

Professor A G

Constantinides© 27

AGC

DSP

AGC

DSP

Linear Prediction

The above can be seen as solving an underlying AR prediction problem

In a compact form

The solution to this can be cast as an optimisation problem

xXa

Professor A G

Constantinides© 28

AGC

DSP

AGC

DSP

Modern Spectral Estimation

Form the error function to be minimised as the difference between the two sides of the equation

Then seek solution as

The solution is (normal equations)

xXae

)()(minmin xXaxXaeeaa

HH

xXaXX HH )(

Professor A G

Constantinides© 29

AGC

DSP

AGC

DSP

Modern Spectral Estimation The autocorrelation matrix

is computed directly from the given signal.

Hence we obtain

Again in the covariance method, only a subset of the total possible rows used in the autocorrelation method, is taken .

XXH

xXXXa HH 1)(