8/12/2019 12 Adaptive Filtering

1/29

Real time DSP

Professors:Eng. Julian Bruno

Eng. Mariano Llamedo Soria

8/12/2019 12 Adaptive Filtering

2/29

Adaptive Filters

8/12/2019 12 Adaptive Filtering

3/29

Recommended bibliography

Saeed V. Vaseghi,Advanced Digital Signal Processing and

Noise Reduction, Second Edition. John Wiley & Sons Ltd. Ch 3: Probabilistic Models

Ch 6: Wiener Filters

Ch 7: Adaptive Filters

Ch 8: Linear Prediction Models

Stergios Stergiopoulos,Advanced Signal Processing

Handbook. CRC Press LLC, 2001

Ch 2: Adaptive Systems for Signal Process - Simon Haykin

B Farhang-Boroujeny.Adaptive Filters. Theory andApplications. John Wiley & Sons.

NOTE:Many images used in this presentation were extracted from the recommended bibliography.

8/12/2019 12 Adaptive Filtering

4/29

Stochastic Processes

The term stochastic process is broadly used to

describe a random process that generates

sequential signals such as speech or noise.

In signal processing terminology, a stochasticprocess is a probability model of a class of

random signals, e.g. Gaussian process, Markov

process, Poisson process,etc.

8/12/2019 12 Adaptive Filtering

5/29

8/12/2019 12 Adaptive Filtering

6/29

Strict-Sense Stationary Processes

A random processX(m) is stat ionaryin a

str ict sense if all its distributions and

statistical parameters such as the mean, the

variance, the power spectral composition andthe higher-order moments of the process, are

time-invariant.

E[x(m)] = x ; mean

E[x(m)x(m + k)] = rxx(k) ; variance

E[|X(f,m)|2] = E[|X(f)|2] = Pxx(f) ; power spectrum

8/12/2019 12 Adaptive Filtering

7/29

Wide-Sense Stationary Processes

A process is said to be w ide sense

stat ionaryif the mean and the

autocorrelation functions of the process are

time invariant:

E[x(m)] =x

E[x(m)x(m + k)]= rxx(k)

8/12/2019 12 Adaptive Filtering

8/29

Non-Stationary Processes

A random process is non-stat ionary if its distributions or

statistics vary with time.

Most stochastic processes such as video signals, audio

signals, financial data, meteorological data, biomedical

signals, etc., are nonstationary, because they are generatedby systems whose environments and parameters vary over

time.

8/12/2019 12 Adaptive Filtering

9/29

Adaptive Filters

An adaptive filter is in reality a nonl inear

device, in the sense that it does not obey the

principle of superposition.

Adaptive filters are commonly classified as:

Linear

An adaptive filter is said to be linear if the estimate of

quantity of interest is computed adaptively (at the outputof the filter) as a linear combination of the available set

of observations applied to the filter input.

Nonlinear

Neural Networks

8/12/2019 12 Adaptive Filtering

10/29

Linear Filter Structures

The operation of a linear adaptive filtering algorithm involves

two basic processes:

a f i l ter ing process designed to produce an output in response to a

sequence of input data

an adaptive process, the purpose of which is to provide mechanismfor the adaptive control of an adjustable set of parameters used in the

filtering process.

These two processes work interactively with each other.

There are three types of filter structures with finite memory : transversal filter,

lattice predictor,

and systolic array.

8/12/2019 12 Adaptive Filtering

11/29

Linear Filter Structures

For stationary inputs, the resulting solution is commonlyknown as the Wiener fi l ter, which is said to be optimum inthe mean-squaresense.

A plot of the mean-square value of the error signal vs. the

adjustable parameters of a linear filter is referred to as theerror-performance surface.

The minimum point of this surface represents the Wienersolution.

The Wiener filter is inadequate for dealing with situations inwhich non-stationarity of the signal and/or noise is intrinsicto the problem.

A highly successful solution to this more difficult problem isfound in the Kalman fi l ter, a powerful device with a wide

variety of engineering applications.

8/12/2019 12 Adaptive Filtering

12/29

8/12/2019 12 Adaptive Filtering

13/29

Lattice Predictor

It has the advantage of simplifying the computation

8/12/2019 12 Adaptive Filtering

14/29

Systolic Array

The use of systolic arrays has madeit possible to achieve a high throughput,

which is required for many advanced signal-

processing algorithms to operate

in real time

8/12/2019 12 Adaptive Filtering

15/29

Linear Adaptive Filtering Algorithms

Stochastic Gradient Approach

Least-Mean-Square (LMS) algorithm

Gradient Adaptive Lattice (GAL) algorithm

Least-Squares Estimation

Recursive least-squares (RLS) estimation

Standard RLS algorithm

Square-root RLS algorithms

Fast RLS algorithms

8/12/2019 12 Adaptive Filtering

16/29

Wiener Filters

The design of a Wiener filter requires a prioriinformation about the statistics of the data to be

processed.

The filter is optimum only when the statisticalcharacteristics of the input data match the a priori

information on which the design of the filter is

based.

When this information is not known completely,

however, it may not be possible to design the

Wiener filter or else the design may no longer be

optimum.

8/12/2019 12 Adaptive Filtering

17/29

The filter inputoutput relation is given by:

The Wiener filter error signal, e(m) is defined as thedifference between the desired signalx(m) and the filter

output signalx (m) :

error signal e(m) for N samples of the signals x(m) and y(m):

Wiener Filters: Least Square

Error Estimation

Wi Fil S

8/12/2019 12 Adaptive Filtering

18/29

Wiener Filters: Least Square

Error Estimation

The Wiener filter coefficients are obtained by minimising

an average squared error function E[e2(m)] with respect

to the filter coefficient vector w

Ryy=E [y(m)yT

(m)] is the autocorrelation matrix of theinput signal

rxy=E [x(m)y(m)] is the cross-correlation vector of the

input and the desired signals

Wi Filt L t S

8/12/2019 12 Adaptive Filtering

19/29

Wiener Filters: Least Square

Error Estimation

For example, for a filter with only two coefficients (w0,

w1), the mean square error function is a bowl-shaped

surface, with a single minimum point

Wi Filt L t S

8/12/2019 12 Adaptive Filtering

20/29

Wiener Filters: Least Square

Error Estimation

The gradient vector is defined as

Where the gradient of the mean square error function withrespect to the filter coefficient vector is given by

The minimum mean square error Wiener filter is obtained by

setting equation to zero

Wi Filt L t S

8/12/2019 12 Adaptive Filtering

21/29

Wiener Filters: Least Square

Error Estimation

The calculation of the Wiener filter coefficients requires the

autocorrelat ionmatrix of the input s ignaland the

crosscorrelat ionvector of the input and the desired

signals.

The optimum wvalue is wo= Ryy-1ryx

8/12/2019 12 Adaptive Filtering

22/29

8/12/2019 12 Adaptive Filtering

23/29

The LMS Filter

The LMS adaptation method is defined as

The instantaneous gradient of the squared error can beexpressed as

Substituting this equation into the recursion update equation

of the filter parameters, yields the LMS adaptation equation

8/12/2019 12 Adaptive Filtering

24/29

The LMS Filter

The main advantage of the LMS algorithm is its simplicity

both in terms of the memory requirement and the

computational complexity which is O(P), where P is the filter

length

Leaky LMS Algorithm The stability and the adaptability of the recursive LMS adaptation can

improved by introducing a so-calledleakage factor as

w(m +1) =.w(m) + .[y(m).e(m)]

When the parameter

8/12/2019 12 Adaptive Filtering

25/29

LMS Algorithm

Wk=zeros(1,L+1); % Vector Inicial de Pesos

yk=zeros(size(xk)); % Seal de salida inicial del FIR

ek=zeros(size(xk)); % Seal inicial de error

for i=L+1:N-1

for n=1:L+1

xk_i(1,n)=xk(i+1-n); % Vector x i-simo

end

yk(i)=xk_i*Wk'; % seal a la salida del FIR

ek(i)=dk(i)-yk(i); % Seal de error

Wk=Wk+2*mu*ek(i)*xk_i; % Vector de pesos i-simoend

8/12/2019 12 Adaptive Filtering

26/29

8/12/2019 12 Adaptive Filtering

27/29

Inverse modeling

Predictive deconvolution

Adaptive equalization

Blind equalization

8/12/2019 12 Adaptive Filtering

28/29

Prediction

Linear predictive codingAdaptive differential pulse-code modulation

Autoregressive spectrum analysis

Signal detection

8/12/2019 12 Adaptive Filtering

29/29