optimal and adaptive signal processinghcso/ee5410_13.pdf · optimal signal processing means finding...

Optimal and Adaptive Signal Processing Optimum signal processing is concerned with the design, analysis, and implementation of systems that extract information from sampled data in a manner that is ‘best’ or optimal in some sense. These systems are called optimal filters. Adaptive signal processing is concerned with the design, analysis, and implementation of systems whose structure changes in response to the incoming data.

H. C. So Summer, 2009-2010

Basic Classes of Optmal Filtering Applications

1.Prediction: use previous samples to predict current samples

Speech Modeling using Linear Predictive Coding (LPC)

Since speech signals are highly correlated, a speech signal ][ns can be accurately modeled by a linear combination of

its past samples:

H. C. So Summer, 2009-2010

][][̂][1

inswnsns iP

i−∑=≈

=

where }{ iw are known as the LPC coefficients. This corresponds a speech compression application of storing

instead of the raw data . Optimal signal processing can be used to determine }{ iw in an optimal way.

2. Identification

H. C. So Summer, 2009-2010

System Identification

Σ

ΣΣ

H(z)

Unknown system

Optimal filter

W(z)

+

++

+

+

-

s(k)

n (k)i

n (k)o

d(k)

r(k)

e(k)

x(k)

Given noisy input ][kx and/or noisy output ][kr , our aim is to determine the unknown system )(zH using )(zW

H. C. So Summer, 2009-2010

3. Inverse Filtering: find the inverse of the system

Signal Recovery Given a noisy discrete-time signal:

][][][][ kwkhkskx +⊗=

where ][ks , ][kh and ][kw represent signal of interest, unknown impulse response and noise. Optimal signal processing means finding ][ks in an optimal way.

H. C. So Summer, 2009-2010

4. Interference Canceling: Remove noise using an external reference

Interference Cancellation in Electrocardiogram (ECG) In biomedical engineering, the measured ECG signal is corrupted by the 50Hz power line interference:

][nr

][][][ ninsnr += where is noise-free ECG and represents 50Hz signal. External reference for is another 50Hz signal.

][ns ][ni][ni

H. C. So Summer, 2009-2010

Problem Statement for Optimal Filters

W(z)x(n) Σ

y(n)d(n)

EstimationError

+-

InputSignal

Output

e(n)

DesiredResponse

Given the input ][nx and the desired response , we want to find the transfer function

][nd)(zW or its impulse response

such that a statistical criterion or a cost function is optimized.

H. C. So Summer, 2009-2010

Some common optimization criteria in the literature are:

1.Least Squares : find )(zW that minimizes ∑−

=

1

0

2 ][N

nne where N

is the number of samples available. This corresponds to least-squares filter design.

2.Minimum Mean Square Error : find )(zW that minimizes

]}[{ 2 neE . This corresponds to Wiener filtering problem. Least Squares Filtering Consider that )(zW is a causal FIR filter of length L so that

∑=−

=

−1

0)(

L

i

ii zwzW (13.1)

H. C. So Summer, 2009-2010

The error function is thus given by ][ne

][][][ nyndne −= (13.2) where

][][][1

0nXWinxwny TL

ii =∑ −=

−

=,

T

TLL

LnxLnxnxnxnX

wwwwW

]]1[]2[]1[][[][

][ 1210

+−+−−=

= −−

L

L

The cost function is

21

0

1

0

1

0

2 ][][][)( ∑ ⎥⎦⎤

⎢⎣⎡

∑ −−=∑=−

=

−

=

−

=

N

n

L

ii

N

nLS inxwndneWJ (13.3)

which is a function of the filter coefficients }{ iw and N is the number of ][nx (and ). ][nd

H. C. So Summer, 2009-2010

The minimum of the least squares function can be found by differentiating )(WJ LS with respect to 110 ,,, −Lwww L and then setting the resultant expressions to zero as follows,

( )

∑ ∑ −−∑ =∑ −−=∑ −⇒

=−−∑ ⎟⎠⎞

⎜⎝⎛

∑ −−⇒

−==⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

∑ ⎟⎠⎞

⎜⎝⎛

∑ −−∂

∂=

∂∂

−

=

−

=

−

=

−

=

−

=

−

=

−

=

−

=

−

=

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

21

0

][][][][][][

0][][][2

1,,1,0,0][][)(

L

i

N

ni

N

n

L

ii

N

n

N

n

L

ii

N

n

L

ii

jj

LS

jnxinxwjnxinxwjnxnd

jnxinxwnd

Ljinxwndww

WJL

(13.4)

H. C. So Summer, 2009-2010

Denote

Tdxdxdxdxdx LRLRRRR )]1(ˆ)2(ˆ)1(ˆ)0(ˆ[ˆ +−+−−= L (13.5)

where

1,,1,0,][][)(ˆ 1

0−=∑ −=−

−

=LjjnxndjR

N

ndx L

and

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

+−+−+−−+−+−

−+−+−−

=

)1,1(ˆ)1,1(ˆ)1,0(ˆ)2,0(ˆ

)1,0(ˆ)0,1(ˆ)0,2(ˆ)0,1(ˆ)0,0(ˆ

ˆ

LLRLRLRLR

RLRLRRR

R

xxxxxx

xx

xx

xxxxxxxx

xx

O

MOL

O

L

with

∑ −−=−−−

=

1

0][][),(ˆ N

nxx jnxinxjiR

H. C. So Summer, 2009-2010

When LN >> , it is convenient to use:

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−−−

+−+−−

=

)0(ˆ)2(ˆ)1(ˆ)2(ˆ

)1(ˆ)1(ˆ)2(ˆ)1(ˆ)0(ˆ

ˆ

xxxxxx

xx

xx

xxxxxxxx

xx

RLRLRLR

RLRLRRR

RO

MOL

O

L

(13.6)

where

)(ˆ][][)(ˆ 1

0iRinxnxiR xx

iN

nxx −=+∑=

−−

=

As a result, we have

( ) dxxxLSLSxxdx RRWWRR ˆˆˆˆ 1⋅=⇒⋅=

− (13.7)

provided that xxR̂ is nonsingular. H. C. So Summer, 2009-2010

Example 13.1 A standard usage for least squares filtering is found in polynomial fitting. For example, line fitting corresponds to a first order polynomial and it is also called linear regression. Given ten measurement points which contain noises:

1 2 3 4 5 6 7 8 9 100

2

4

6

8

10

12

H. C. So Summer, 2009-2010

Suppose the desired form corresponds to a straight line, i.e., cmxy += . Hence we can write the “filter output” as:

cnmxny += ][][ where corresponds to the measurement values ][nd According to (13.3), the least squares cost function is:

( ) ( )210

1

210

1][][][),( cmnndcnmxndcmJ

nn−−∑=−−∑=

==

( )

∑∑ +∑=⇒

∑ =−−−=∂

∂

== =

=N

n

N

n

N

n

N

n

ncnmnnd

ncmnndm

cmJ

11 1

2

1

][

0)(][2),(

H. C. So Summer, 2009-2010

( )

∑∑ +∑=⇒

∑ =−−−=∂

∂

== =

=N

n

N

n

N

n

N

n

cnmnd

cmnndc

cmJ

11 1

1

1][

0)1(][2),(

⎥⎦

⎤⎢⎣

⎡=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

∑

∑

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

∑

∑∑=⎥⎦

⎤⎢⎣

⎡⇒⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

∑

∑=⎥⎦

⎤⎢⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

∑∑

∑∑

=

=

−

=

==

=

=

==

==3723.10091.1

][

][

][

][

11

1

1

1

11

2

1

1

11

11

2

N

n

N

nN

n

N

n

N

nN

n

N

nN

n

N

n

N

n

N

n

nd

nnd

Nn

nn

cm

nd

nnd

cm

n

nn

http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_1.m

H. C. So Summer, 2009-2010

http://www.ee.cityu.edu.hk/%7Ehcso/ee5410/ex13_1.m

1 2 3 4 5 6 7 8 9 100

2

4

6

8

10

12

H. C. So Summer, 2009-2010

Example 13.2

Σ h zi=0

4

i-i

Σ w zi=0

4

i-i

Σ

x(n)

d(n)

y(n) e(n)

+

-

Unknown System

Σq(n)+

+

In this example least squares filtering is applied in determining the impulse response of an unknown system. Assume that the unknown impulse response is causal and { }={1,2,3,2,1}. Given ih N samples of ][nx and ][nd where

is a measurement noise. ]n[q

H. C. So Summer, 2009-2010

We can use MATLAB to simulate the least squares filter for impulse response estimation. The MATLAB source code is http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_2.m %define the number of samples N=50; %define the noise and signal powers noise_power = 0.0; signal_power = 5.0; %define the unknown system impulse response h=[1 2 3 2 1]; %generate the input signal which is a Gaussian white noise with power 5 x=sqrt(signal_power).*randn(1,N);

H. C. So Summer, 2009-2010


%generate R_xx corr_xx=xcorr(x); for i=0:4 for j=0:4 R_xx(i+1,j+1)= corr_xx(N+i-j); end end

%generate the desired output plus noise d=conv(x,h); d=d(1:N)+sqrt(noise_power).*randn(1,N); %generate R_dx corr_xd = xcorr(x,d); for i=0:4 R_dx(i+1) = corr_xd(N-i); end %compute the estimate impulse response

H. C. So Summer, 2009-2010

W_ls = inv(R_xx)*(R_dx)'

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−−

−−

=

6413.2514044.244998.368182.115115.44044.246413.2514044.244998.368182.114998.364044.246413.2514044.244998.368182.114998.364044.246413.2514044.24

5115.48182.114998.364044.246413.251

ˆxxR

]9508.3340732.6163125.8738827.6588245.390[ˆ =dxR

TLSW ]8176.08501.18291.20151.21095.1[=

When N is increased to 500, we have

TLSW ]9828.09942.19927.29943.19975.0[=

When N is increased to 5000, we have

TLSW ]9984.09972.19976.29981.10000.1[=

For 500=N and the noise power is 0.5 (SNR=10 dB): H. C. So Summer, 2009-2010

TLSW ]9925.09773.19728.29826.10158.1[=

For 500=N and the noise power is 5.0 (SNR=0 dB):

TLSW ]0591.10249.29484.20138.20900.1[=

It is observed that

1. The estimation accuracy improves as N increases. It is reasonable because as N increases, the accuracy of xxR̂ and dxR̂ increases due to more samples are involved in their computation.

2. The estimation accuracy improves as the noise power decreases.

H. C. So Summer, 2009-2010

Example 13.3 Consider conversation using hand-free telephone between Users A and B. At the side of User A, the microphone will receive User A’s voice as well as the speaker output – User B’s voice. This corrupted signal can be modeled as:

][][][],[][][ ncndnininsny ⊗=+=

where is the desired speech (User A) and is due to speaker output (User B), which is equal to the convolution of speaker output ][nd and speaker-to-microphone channel impulse response . The reference signal (speaker output)

is highly correlated with such that:

][ns ][ni

][nc][nd ][ni

][][1

0lndcni l

L

l−∑=

−

=

Given and , the problem is to find . ][ny ][nd ][ns

H. C. So Summer, 2009-2010

load corrupted.dat; %this is y[n] plot(corrupted);

H. C. So Summer, 2009-2010

load ref.dat; %this is d[n] plot(ref);

H. C. So Summer, 2009-2010

Assume we know that 5=L so that

]4[]3[]2[]1[][

][][

43210

4

0

−+−+−+−+=

−∑==

ndcndcndcndcndc

lndcni ll

The error function is constructed as

][~][][4

0lndcnyke l

l−∑−=

=

where we hope ll cc →~ so that ][][ kske → The least squares cost function is then

21

0

4

0

1

0

2 ][~][][ ∑ ⎥⎦⎤

⎢⎣⎡

∑ −−=∑−

= =

−

=

N

n ll

N

nlndcnyne

H. C. So Summer, 2009-2010

Using (13.4)-(13.7), least squares solution is obtained from

( ) ydddLS RRW ˆˆ 1⋅=

−

where

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

=

)0(ˆ)1(ˆ)2(ˆ)3(ˆ)4(ˆ)1(ˆ)0(ˆ)1(ˆ)2(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ)1(ˆ)2(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ)1(ˆ)4(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ

ˆ

dddddddddd

dddddddddd

dddddddddd

dddddddddd

dddddddddd

dd

RRRRRRRRRRRRRRRRRRRRRRRRR

R

with

][][)(ˆ 1

0indndiR

iN

ndd +∑=

−−

=

and

H. C. So Summer, 2009-2010

Tydydydydydyd RRRRRR ])4(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ[ˆ −−−−=

with

4,3,2,1,0,][][)(ˆ 1

0=∑ −=−

−

=jjndnyjR

N

nyd

The MATLAB source code is http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_3.m It is found that

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

=

257.9152 207.4885 197.3658 166.4233144.6380 207.4885 257.9152 207.4885 197.3658 166.4233 197.3658 207.4885 257.9152 207.4885 197.3658 166.4233 197.3658 207.4885 257.9152 207.4885

144.6380 166.4233 197.3658 207.4885 257.9152

ˆ ddR

H. C. So Summer, 2009-2010


TydR ]867.0252 402.7274 714.0973 355.28504435.488[ˆ =

which gives

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

=

5.01393.9788-

3.01792.0122-

0.9588

ˆˆˆˆˆ

4

3

2

1

0

ccccc

W LS

The interference signal is estimated as

][ˆ][4

0lndcni l

l−∑=

=

while the source signal is estimated as

][][][ ninyns −=

H. C. So Summer, 2009-2010

The interference signal is plot(interference);

H. C. So Summer, 2009-2010

The source signal is plot(source);

H. C. So Summer, 2009-2010

Example 13.4 Suppose we want to find the position ),( yx of a target and we have range measurements between the target and M receivers with known positions ),( nn yx , Ni ,,2,1 L= . Without measurement noises, the range is:

( ) ( )22nnn yyxxr −+−=

H. C. So Summer, 2009-2010

Each range defines a circle where the target will lie and intersection of at 3 least circles provides the source location. In practice, The obtained range measurements are noisy:

Nnpyyxxr nnnn ,,2,1,)()( 22 L=+−+−=

Squaring both sides, we have:

222222 )()(2)()( nnnnnnn yyxxppyyxxr −+−++−+−= Now the noise component is

222 )()(2 nnnnn yyxxppq −+−+=

H. C. So Summer, 2009-2010

Expanding the equation and letting yields 22 yxR +=

222

22222

22222

222

22

,22

22

)()(

nnnnn

nnnnnn

nnnnnn

nnnn

ryxRyyxx

yxRqryxRyyxx

qyyyyxxxxr

qyyxxr

−+≈−+⇒

+=+−+=−+⇒

++−++−=⇒

+−+−=

Similar to Example 13.1, the inputs are now nx2 , and -1,

the desired output is , and the filter weights are

ny2222

nnnn ryxd −+=x, and y R. According to (13.3), the least squares cost

function is:

( )2

122),,( nnn

N

ndRyyxxRyxJ −−+∑=

=

H. C. So Summer, 2009-2010

( )

nnN

nn

N

nnn

N

nn

N

n

nnnnN

n

xdxRyxyxx

xdRyyxxx

RyxJ

∑=∑−∑+∑⇒

=−−+∑=∂

∂

====

=

111

2

1

1

22

0)2(22),,(

Similarly, we get:

nnN

nn

N

nn

N

nnn

N

nydyRyyyxx

yRyxJ

∑=∑−∑+∑⇒=∂

∂

==== 11

2

11220),,(

and

nN

nn

N

nn

N

ndRNyyxx

RRyxJ

∑=−∑+∑⇒=∂

∂

=== 111220),,(

H. C. So Summer, 2009-2010

In matrix form:

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

∑

∑

∑

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−∑∑

∑−∑∑

∑−∑∑

=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡⇒

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

∑

∑

∑

=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−∑∑

∑−∑∑

∑−∑∑

=

=

=

−

==

===

===

=

=

=

==

===

===

nN

n

nnN

n

nnN

n

nN

nn

N

n

nN

nn

N

nnn

N

n

nN

nnn

N

nn

N

n

nN

n

nnN

n

nnN

n

nN

nn

N

n

nN

nn

N

nnn

N

n

nN

nnn

N

nn

N

n

d

yd

xd

Nyx

yyyx

xyxx

Ryx

d

yd

xd

Ryx

Nyx

yyyx

xyxx

1

1

1

1

11

1

2

11

11

2

1

1

1

1

11

1

2

11

11

2

1

22

22

22

22

22

22

H. C. So Summer, 2009-2010

Suppose we have 4 receivers with positions (0,0), (0,10), (10,0) and (10,10) and the target is at (2,3). The noise-free ranges are:

( ) ( ) ( ) ( ) 3.6056==−+−=−+−= 130302 2221

211 yyxxr

( ) ( ) ( ) ( ) 7.2801==−+−=−+−= 5310302 2222

222 yyxxr

( ) ( ) ( ) ( ) 8.5440==−+−=−+−= 7303102 2223

233 yyxxr

( ) ( ) ( ) ( ) 10.6301==−+−=−+−= 113103102 2224

244 yyxxr

Under small range errors, , we get }6.10,6.8,3.7,6.3{}{ =nr

)(),,( 5318,13.4421.9983,3.0=Ryx .

Under large range errors, }11,9,7,4{}{ =nr , we get )(),,( 0750,14.2501.5750,3.1=Ryx .

http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_4.m H. C. So Summer, 2009-2010


Adaptive Signal Processing

Application areas are similar to those of optimal signal processing but now the environment is changing, the signals are nonstationary and/or the parameters to be estimated are time-varying. For example, Echo cancellation for Hand-Free Telephones (The speech echo is a nonstationary signal)

Equalization of Data Communication Channels (The channel impulse response is changing, particularly in mobile communications)

Time-Varying System Identification (the system transfer function to be estimated is non-stationary in some control applications)

H. C. So Summer, 2009-2010

Adaptive Filter Development

Year Application Developers

1959 Adaptive pattern recognition system Widrow et al

1960 Adaptive waveform recognition Jacowatz

1965 Adaptive equalizer for telephone channel

Lucky

1967 Adaptive antenna system Widrow et al

1970 Linear prediction for speech analysis Atal

Now numerous applications, structures, algorithms

H. C. So Summer, 2009-2010

Adaptive Filter Definition An adaptive filter is a time-variant filter whose coefficients are adjusted in a way to optimize a cost function or to satisfy some predetermined optimization criterion.

Characteristics:

Can automatically adapt (self-optimize) in changing environments and changing system requirements

Can be trained to perform specific filtering and decision-making tasks according to some updating equations

It can automatically operate in changing environments (e.g. signal detection in wireless channel)

nonstationary signal conditions (e.g. LPC of speech signal) time-varying parameter estimation (e.g. position tracking of moving source)

H. C. So Summer, 2009-2010

Block diagram of a typical adaptive filter is shown below:

AdaptiveFilter

AdaptiveAlgorithm

Σx(k)+-

{h(k)}

d(k)

e(k)

x(k) : input signal y(k) : filtered outputd(k) : desired responseh(k) : impulse response of adaptive filterThe cost function may be E{e (k)} or Σ e (k)2 2

k=0

N-1

y(k)

FIR or IIR adaptive filter filter can be realized in various structures adaptive algorithm depends on the optimization criterion

H. C. So Summer, 2009-2010

Basic Classes of Adaptive Filtering Applications 1. Prediction: signal encoding, linear prediction coding,

spectral analysis

H. C. So Summer, 2009-2010

2.Identification: adaptive control, layered earth modeling, vibration studies of mechanical system

H. C. So Summer, 2009-2010

3.Inverse Filtering: adaptive equalization for communication channel, deconvolution

H. C. So Summer, 2009-2010

4.Interference Canceling: adaptive noise canceling, echo cancellation

H. C. So Summer, 2009-2010

Widrow’s Least Mean Squares (LMS) Algorithm For simplicity, it is assumed that the adaptive filter is of causal FIR type and is implemented in direct form. Therefore, its system block diagram is

w (n)0

z-1

w (n)1

z-1

+

w (n)L-2

z-1

+

w (n)

+

L-1

...

...

x(n)

+d(n)

y(n)e(n) +

-

H. C. So Summer, 2009-2010

1. Optimization Criterion

To minimize the mean square error ]}[{ 2 neE where E denotes the expectation operator. The error signal

][ne at time n is:

][][][ nyndne −= where

)()(][][][1

0nXnWinxnwny TL

ii =∑ −=

−

=,

T

TLL

LnxLnx

W nwnwnwnwn

nxnxnX ]]1[]2[]1

]][][][][[)( 1210

+−+−

= −−

L

L

[][[)( −= H. C. So Summer, 2009-2010

2. Adaptation Procedure

ethod:

It is based on steepest descent m

)(nW ()()1 nWnW Δ+=+ where

⎟⎟⎠

⎞⎜⎜⎝

⎛

∂∂

μ−=Δ)(]}[{)(

2

nWneEnW (13.8)

and a step-size parameter which controls convergence ate and ensures stability

pectation operator is ignored:

μ is r In LMS algorithm, the ex

H. C. So Summer, 2009-2010

[ ])(][)(

)(nXnenW

AnW,)()(][][)(

)(][

][][

2)(

)(][

2)()1(

2

2

BBAnXnWndnenW

nWne

nenenW

nWnenWnW

TT=

⋅∂⋅−∂⋅μ−=

∂∂

⋅∂

∂μ−=

∂∂μ

−=+

r μ+=

∂∂

o 1,,1,0],[][][]1[ −=−μ+=+ Liinxnenwnw ii L (13.9) . Advantages

Low computational complexity Simple to implement Allow real-time operation

3

H. C. So Summer, 2009-2010

Example 13.5

This example considers an application of linear prediction. Suppose a signal ][nx is a second-order autoregressive (AR) process that satisfies the following difference equation:

][]2[81.0]1[558.1][ nvnxnxnx +−−−=

where white noise process such that

LMS filter to predict

][nv is a

⎩⎨⎧ =σ=+=

otherwise,00,]}[][{)(

2 mmnvnvEmR vvv

We want to use a two-coefficient

[][][][ˆ2

=∑ −= nwinxnwnx

)(nx by

]2[][]1[] 21 −+− nxnwnx 1=i

i

H. C. So Summer, 2009-2010

Upon convergence, we desire

558.1]}[{ 1and

→nwE

81.0]}[{ 2 −→nwE

z -1

w (n)1

z -1x(n)

w (n)

+d(n)

x(n)

e(n)+

-

2

++

+

H. C. So Summer, 2009-2010

The error function or prediction error

Thus the LMS algorithm for this problem is

][ne is given by

]2[][]1[][][

][][][][

21

2

1

−−−−=

∑ −−==

nxnwnxnwnx

inxnwndnei

i

]1[][][][

][][][

2][

][][

2][]1[

1

1

2

11

2

11

−μ+=∂∂

⋅∂

∂⋅

μ−=

∂∂

⋅μ

−=+

nxnenwnwne

nenenw

nwnenwnw

and

]2[][][][][

2][]1[ 2

2

2

22 −μ+=∂∂

⋅μ

−=+ nxnenwnwnenwnw

The computational requirement for each sampling interval is

Multiplications : 5 Addition/subtraction : 4

H. C. So Summer, 2009-2010

Two values of μ, 0.02 and 0.004, are investigated:

02.0=μ Convergence characteristics for LMS predictor with

H. C. So Summer, 2009-2010

004.0=μ Convergence characteristics for LMS predictor with

H. C. So Summer, 2009-2010

Observations: 1.When 02.0=μ , we had a fast convergence rate (the

parameters converged to the desired values in approximately 200 iterations) but large fluctuation existed in

2.When

][1 nw and ][n . 2w

004.0=μ , small fluctuation in ][1 nw and ][2 nw but the filter coefficients did not converge to the desired values of 1.558 and -0.81 respectively after the 300th iteration.

3.The learning behaviours of and agreed with

those of ][1 nw ][2 nw

]}[{ 1 nwE and ]}[{ 2 nwE . Notice that ]}[{ 1 nwE and ]}[{ 2 nwE can be derived using some advanced mathematics

H. C. So Summer, 2009-2010

We can use MATLAB to simulate this example

ww.ee.c

http://w ityu.edu.hk/~hcso/ee5410/ex13_5.m lear all Generate x[n]

] (2 ndn;

c%x(1)=randn; %v[n is white noise with power 1 x

xe

e % LMS alu = 0.001; %step-size=0.001

)=ra=10000;

for n=3:5000 (n)=1.558*x(n-1)-0.81*x(n-2)+randn; nd

N

for n=5001:N x(n)=1*x(n-1)-0.5*x(n-2)+randn; nd

gorithm mH. C. So Summer, 2009-2010

w1(1)=0; w1(2)=0; w1(3)=0; w2(1)=0; w2(2)=0;

for n=3:N n)*x(n-1)-w2(n)*x(n-2);

xlabel('number of iterations')

w2(3)=0;

e=x(n)-w1(w1(n+1)=w1(n)+mu*e*x(n-1); w2(n+1)=w2(n)+mu*e*x(n-2); end n=1:N; plot(n,w1(1:N),n,w2(1:N),'r');

nd('w1[n]','w2[n]') lege

H. C. So Summer, 2009-2010

1000 2000 3000 4000 5000 6000 7000 8000 9000 10000-1

-0.5

0

0.5

1

1.5 w1[n]

w2[n]

number of iterations

H. C. So Summer, 2009-2010

Example 13.6

This example considers an application of system identification. Given the input signal ][nx and output signal estimate the impulse response of the system or plant using the LMS algorithm.

Suppose the transfer function of the plant is

][nd , we can

∑=

−2

0i

ii zh which is

a causal FIR unknown system, then can be represented as

][nd

∑ −==

2

0][][

ii inxhnd

H. C. So Summer, 2009-2010

Assuming that the order the transfer function is unknown and we use a 2- coefficient LMS filter to model the system function as follows,

Σ h zi=0 i

-i2

Σ w z1

i=0 i-i

Σ

x(n) d(n)

e(n)

Plant

+

y(n) -

H. C. So Summer, 2009-2010

The error function is computed as

][][][][][][1∑ −−=−= inxnwndnyndne i

]1[][][][][ 10

0

−−−==

nxnwnxnwndi

Thus the LMS algorithm for this problem is

)()()(][

][][][

2][

][][

2][]1[

0

0

2

00

2

00

nxnenwnw

nenenenw

nwnenwnw

μ+=∂∂

⋅∂

∂⋅

μ−=

∂∂

⋅μ

−=+

and

]1[][][][][

2][]1[

1

1

2

11

−μ+=∂∂

⋅μ

−=+

nxnenwnwnenwnw

H. C. So Summer, 2009-2010

We can use MATLAB to simulate th

is example

http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_6.m % Generate signals N=5000; x=randn(1,N); %x[n] is white Gaussian h=[3,2,1]; %set system impulse response

d=d(1:length(x)); % LMS algorithm mu = 0.0025; %step-size=0.001 w0(1)=0; w0(2)=0; w0(3)=0; w1(1)=0; w1(2)=0; w1(3)=0;

n=3:N e=d(n)-w0(n)*x(n)-w1(n)*x(n-1); w0(n+1)=w0(n)+mu*e*x(n); w1(n+1)=w1(n)+mu*e*x(n-1); end

d=conv(h,x);

for

H. C. So Summer, 2009-2010

n=1:N;

terations')

plot(n,w0(1:N),n,w1(1:N),'r'); legend('w0[n]','w1[n]') xlabel('number of i

3.5

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000-0.5

0

0.5

1

1.5

2

2.5

3

w0[n]w1[n]


H. C. So Summer, 2009-2010

If we add:

e5410/ex13_6_2.mw2(n+1)=w2(n)+mu*e*x(n-2);

~hcso/ehttp://www.ee.cityu.edu.hk/

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000-0.5

0

0.5

1

1.5

2

2.5

3

w0[n]w1[n]w2[n]

number of iterations H. C. So Summer, 2009-2010

If we further add: W3(n+1)=w3(n)+mu*e*x(n-3); http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_6_3.m

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000-0.5

0

0.5

1

1.5

2

2.5

3


w0[n]w1[n]w2[n]w3[n]

H. C. So Summer, 2009-2010

http://www.ee.cityu.edu.hk/%7Ehcso/ee5410/ex13_6_2.m

Example 13.7

This example considers an application of interference cancellation. Given a received signal ][kr which consists of a source signal

][ks and a sinusoidal interference with task is to extract

known frequency. The ][ks from ][kr .

Notice that the amplitude and phase of the sinusoid are unknown. A well-known application is to remove 50/60 Hz power line interference in ECG recording.

H. C. So Summer, 2009-2010

Source Signal + Sinusoidal Interference

r(k)=s(k) + Acos(ω k + φ)0

Reference Signal

sin(ω k)0

90 Phase-Shift0cos(ω k)0

b0

b1

Σe(k)

+

-

-

The interference cancellation system consists of a phase-shifter and a two-weight adaptive filter. By properly adjusting the weights, the reference waveform can be changed in magnitude and phase in any way to model the

soid.

090

interfering sinuH. C. So Summer, 2009-2010

The filtered output is of the form

)cos(][)sin(][][][ 0100 kkbkkbkrke ω−ω−= The LMS algorithm is

)sin(][][][][2][2 00 kbkekb ∂∂∂

][][)(][][]1[

00

2

0

2

00

kkekw

kekekbkkbkb

ωμ+=

∂⋅

∂⋅

μ−=⋅−=+

nd

e∂μ

a

)cos(][][][][][]1[

2

11kekbkb ∂μ

−=+2

01

1

kkekbkb

ωμ+=∂

⋅

H. C. So Summer, 2009-2010

H. C. So Summer, 2009-2010

For sufficiently large number of iterations, we expect

)sin(][0 φ−→ Akb and

)cos(][1 φ→ Akb The filtered output is then approximated as

)cos(][)cos()cos()sin()sin(][][

0

00

kAkrkAkAkrke

φ+ω−=ωφ−ωφ+≈

][ks=

optimal and adaptive signal processinghcso/ee5410_13.pdf · optimal signal processing means finding...

Documents