ssp (meng pfiev) chapter3
TRANSCRIPT
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
1/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT
Chapter 3:
Spectrum Analysis
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
2/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT2
References
[1] Simon Haykin,Adaptive Filter Theory, Prentice Hall,1996 (3rdEd.), 2001 (4thEd.).
[2] Steven M. Kay, Fundamentals of Statistical Signal Processing:
Estimation Theory, Prentice Hall, 1993.
[3] Alan V. Oppenheim, Ronald W. Schafer,Discrete-Time Signal
Processing, Prentice Hall, 1989.
[4] Athanasios Papoulis,Probability, Random Variables, andStochastic Processes, McGraw-Hill, 1991 (3rdEd.), 2001 (4thEd.).
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
3/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT3
3. Power Spectrum (1)
( ) ( ) ,j tX x t e dt +
=
For a deterministic signalx(t), the spectrum is well defined: IfX()
represents its Fourier transform, i.e., if
then |X() |2 represents its energy spectrum. This follows from Parsevals
theorem since the signal energy is given by
Thus, |X() |2 represents the signal energy in the band (, + ).
2 2
12( ) | ( ) | .x t dt X d E + +
= =
t0
( )X t
0
2| ( )|X Energy in ( , ) +
+
(3-1)
(3-2)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
4/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT4
3. Power Spectrum (2)
( ) ( )T j t
T TX X t e dt
=
However for stochastic processes, a direct application of (3-1) generates
a sequence of random variables for every . Moreover, for a stochastic
process,E{|X(t) |2} represents the ensemble average power (instantaneousenergy) at the instant t.
To obtain the spectral distribution of power versus frequency for stochastic
processes, it is best to avoid infinite intervals to begin with, and start with a
finite interval ( T, T ) in (3-1). Formally, partial Fourier transform of a
processX(t) based on ( T, T ) is given by
so that
represents the power distribution associated with that realization based
on ( T, T ). Notice that (3-4) represents a random variable for every
and its ensemble average gives, the average power distribution based on
( T, T ). Thus
2 2| ( ) | 1( )
2 2
T j tT
T
XX t e dt
T T
=
(3-3)
(3-4)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
5/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT5
3. Power Spectrum (3)
1 2
1 2
2( )*
1 2 1 2
( )1 2 1 2
| ( ) | 1( ) { ( ) ( )}
2 2
1( , )
2
T
XX
T T j t tT
T T
T T j t t
T T
XP E E X t X t e dt dt
T T
R t t e dt dtT
= =
=
1 2 ( )
1 2 1 2
1( ) ( ) .
2T XX
T T j t t
T TP R t t e dt dt
T
=
2
2
2 | |
22
1( ) ( ) (2 | |)
2
( ) (1 ) 0
T XX
XX
T j
T
T j
TT
P R e T d T
R e d
=
=
(3-5)
represents the power distribution ofX(t) based on ( T, T ). IfX(t) is
assumed to be w.s.s, thenRXX(t1, t2) =RXX(t1- t2), and (3-5) simplifies to
Let = t1 t2 , we get
to be the power distribution of the w.s.s. processX(t) based on ( T, T ).
Finally letting T in (3-6), we obtain
(3-6)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
6/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT6
3. Power Spectrum (4)
( ) lim ( ) ( ) 0XX T XX
j
TS P R e d
+
= =
F T( ) ( ) 0.XX XX
R S
12( ) ( )XX XX
jR S e d
+
=
212 ( ) (0) {| ( ) | } ,XX XXS d R E X t P the total power. +
= = =
to be thepower spectral densityof the w.s.s processX(t). Notice that
i.e., the autocorrelation function and the power spectrum of a w.s.s process
form a Fourier transform pair, a relation known as the Wiener-Khinchin
Theorem. From (3-8), the inverse formula gives
(3-7)
and in particular for = 0, we get
From (3-10), the area under SXX() represents the total power of the
processX(t), and hence SXX() truly represents the power spectrum.
(3-8)
(3-9)
(3-10)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
7/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT7
3. Power Spectrum (5)
( )* *
1 1 1 1
2
1
12
12
( ) ( )
( ) 0.
i j
XX XX
iXX
n n n nj t t
i j i j i ji j i j
n j t
ii
a a R t t a a S e d
S a e d
+
= = = =
+
=
=
=
+ 0
represents the powerin the band( , ) +
( )XXS ( )XXS
( ) ( ) 0.XX XX
R nonnegative - definite S
The nonnegative-definiteness property of the autocorrelation functiontranslates into the nonnegative property for its Fourier transform
(power spectrum)
(3-11)
From (3-11), it follows that
(3-12)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
8/53
Dept. of Telecomm. Eng,Faculty of EEE
SSP2013DHT, HCMUT8
3. Power Spectrum (6)
0
( ) ( )
( )cos
2 ( )cos ( ) 0
XX XX
XX
XX XX
jS R e d
R d
R d S
+
+
=
=
= =
IfX(t) is a real w.s.s process, thenRXX() =RXX(-), so that
(3-13)
so that the power spectrum is an even function, (in addition to being realand nonnegative).
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
9/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT9
3. Power Spectra and Linear Systems (1)
* *( ) ( ) ( ), ( ) ( ) ( ) ( ).XY XX YY XX
R R h R R h h = =
( ) ( ), ( ) ( )f t F g t G
( ) ( ) ( ) ( )f t g t F G
{ ( ) ( )} ( ) ( ) j tf t g t f t g t e dt
+
=
h(t)X(t) Y(t)If a w.s.s processX(t) with autocorrelation
functionRXX() SXX() is applied to alinear system with impulse response h(t), then
the cross correlation functionRXY() and the output autocorrelation function
RYY() can be determined. From there
(3-14)
If (3-15)
then(3-16)
since
(3-17){ }
( )
{ ( ) ( )}= ( ) ( )
= ( ) ( ) ( )
= ( ) ( ).
j t
j j t
f t g t f g t d e dt
f e d g t e d t
F G
+ +
+ +
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
10/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT10
3. Power Spectra and Linear Systems (2)
* *
( ) { ( ) ( )} ( ) ( )XY XX XXS R h S H = =
( )* * *
( ) ( ) ( ),j j th e d h t e dt H
+ +
= =
( ) ( ) j tH h t e dt
+
=
Using (3-15)-(3-17) in (3-14) we get
2
( ) { ( )} ( ) ( )
( ) | ( ) | .
YY YY XY
XX
S R S H
S H
= =
=
(3-18)since
where
(3-19)
represents the transfer function of the system, and
(3-20)
From (3-18), the cross spectrum need not be real or nonnegative. However
the output power spectrum is real and nonnegative and is related to the
input spectrum and the system transfer function as in (3-20). Eq. (3-20) can
be used for system identification as well.
Example of Thermal noise (Example 11.1, p. 351, [4])
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
11/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT11
3. Power Spectra and Linear Systems (3)
( ) ( ) ( ) .WW WW R q S q = =
W.S.S White Noise Process: If W(t) is a w.s.s white noise process, then
Thus the spectrum of a white noise process is flat, thus justifying its
name. Notice that a white noise process is unrealizable since its total
power is indeterminate.
From (3-20), if the input to an unknown system is a white noise process,
then the output spectrum is given by
2( ) | ( ) |YY
S q H =
Notice that the output spectrum captures the system transfer function
characteristics entirely, and for rational systems Eq (3-22) may be used
to determine the pole/zero locations of the underlying system.
(3-21)
(3-22)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
12/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT12
3. Power Spectra and Linear Systems (4)
2 , | | / 2( ) | ( ) | .0, | | / 2
XX
q BS q H
B
= =
>
/ 2 / 2
/ 2 / 2
( ) ( )
sin( / 2)sinc( / 2)
( / 2)
XX XX
B Bj j
B BR S e d q e d
BqB qB B
B
= =
= =
2| ( )|H
/ 2B / 2B
1
qB
( )XX
R
Example 3.1: A w.s.s white noise process W(t) is passed through a low pass
filter (LPF) with bandwidthB/2. Find the autocorrelation function of the
output process.
Solution: LetX(t) represent the output of the LPF. Then from (3-22)
(3-23)
Inverse transform of SXX() gives the output autocorrelation function to
be(3-24)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
13/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT13
3. Power Spectra and Linear Systems (5)
1
2( ) ( )
t T
t TTY t X d
+
=
( ) ( ) ( ) ( ) ( )Y t h t X d h t X t
+
= =
2( ) ( ) | ( ) | .YY XX
S S H =
Eq (3-23) represents colored noise spectrum and (3-24) its autocorrelation
function.
Example 3.2: Let
T Tt
( )h t
1 / 2T
1
2( ) sinc( )
T j t
TTH e dt T
+
= =
represent a smoothing operation using a moving window on the input
processX(t). Find the spectrum of the output Y(t) in term of that ofX(t).
(3-25)
Solution: If we define an LTI system with impulse
response h(t) as in the figure, then in term of h(t),
(3-25) reduces to
(3-26)
so that (3-27)
where (3-28)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
14/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT14
3. Power Spectra and Linear Systems (6)
2( ) ( ) sinc ( ).
YY XX S S T =
( )XX
S 2sinc ( )T
T
( )YY
S
so that
(3-29)
Notice that the effect of the smoothing operation in (3-25) is to suppress
the high frequency components in the input (beyond / T), and the
equivalent linear system acts as a low-pass filter (continuous-time moving
average) with bandwidth 2/ T in this case.
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
15/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT15
3. Discrete-Time Processes (1)
( ) ( ) ( ),n
X t X nT t nT=
{ } ,kr +
( ) ( ).XX k
k
R r kT +
=
=
( ) 0,XX
j T
kk
S r e
+
=
=
( ) ( 2 / )XX XXS S T = +
22 .B
T
=
For discrete-time w.s.s stochastic processesX(nT) with autocorrelation
sequence (proceeding as above) or formally defining a continuous
time process we get the corresponding
autocorrelation function to be
Its Fourier transform is given by
(3-30)
and it defines the power spectrum of the discrete-time processX(nT).
From (3-30),
(3-31)
so that SXX() is a periodic function with period
(3-32)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
16/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT16
3. Discrete-Time Processes (2)
1
( )2 XX
B jk T
k Br S e d B
= 2
0
1{| ( ) | } ( )
2 XX
B
Br E X nT S d
B
= =
*( ) ( ) ( )XY XX
jS S H e
=
2( ) ( ) | ( ) |YY XX
jS S H e
=
( ) ( )j j nT
n
H e h nT e +
=
=
This gives the inverse relation
and
represents the total power of the discrete-time processX(nT). The
input-output relations for discrete-time system h(nT) translate into
(3-33)
(3-34)
(3-35)and
(3-36)
where (3-37)
represents the discrete-time system transfer function.
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
17/53
Dept. of Telecomm. Eng,Faculty of EEE SSP2013DHT, HCMUT17
3. Matched Filter (1)
0( ) ( ) ( ), 0r t s t w t t t = + < 0. ThusH(s) and its inverse 1/H(s)
can be chosen to be stableand causalin (3-58). Such a filter is known asthe Wiener factor, and since it has all its poles and zeros in the left half
plane, it represents a minimum phase factor. In the rational case, ifX(t)
represents a real process, then SXX() is even and hence (3-58) reads
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
27/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT27
3. Matched Filter (11)
2 20 ( ) ( ) | ( ) ( ) | .XX XX s j s jS S s H s H s = = = =
2 2 2
4
( 1)( 2)( )
( 1)XX
S
+ =
+
2 2 22
4
(1 )(2 )( ) .
1XX
s sS s
s
+ =
+
Example 18.3: Consider the spectrum
which translates into
2s j=
2s j=
1s=+1s =
1
2
js
+= 1
2
js
+=
1
2
js
=
1
2
js
=
The poles () and zeros ( ) of this
function are shown in the figure. From there to maintain the symmetry
condition in (3-59), we may group together the left half factors as
2
21 1
2 2
( 1)( 2 )( 2 ) ( 1)( 2)( )
2 1j js s
s s j s j s sH s
s s
+ + +
+ + + += =
+ +
(3-59)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
28/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT28
3. Matched Filter (12)
( )( ) ( ) jH A e =
0ln ( ) ln ( ) ( ) ( ) .j tH A j b t e dt
+ = =
and it represents the Wiener factor for the spectrum SXX() above.
Observe that the poles and zeros (if any) on thej-axis appear in even
multiples in SXX() and hence half of them may be paired withH(s) (andthe other half withH( s)) to preserve the factorization condition in (3-58).
Notice thatH(s) is stable, and so is its inverse.
More generally, ifH(s) is minimum phase, then lnH(s) is analytic on
the right half plane so that
gives
0
0
ln ( ) ( )cos
( ) ( )sin
t
t
A b t t dt
b t t dt
=
=
Thus
(3-60)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
29/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT29
3. Matched Filter (13)
( ) {ln ( )}.A = H
( ) .
XXS d
Since cost and sint are Hilbert transform pairs, it follows that the phase
function () in (3-60) is given by the Hilbert transform of lnA(). Thus
Eq. (3-60) may be used to generate the unknown phase function of a
minimum phase factor from its magnitude.
For discrete-time processes, the factorization conditions take the form
and
In that case 2( ) | ( ) |XX
jS H e =
0
( ) ( ) k
k
H z h k z
=
=
(3-61)
(3-62)
(3-63)
where the discrete-time system
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
30/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT30
3. Matched Filter (14)
2( ) ( ) | | ( ) | ( ).WWs jL s L s L j S = = =
is analytic together with its inverse in |z| >1. This unique minimum phase
function represents the Wiener factor in the discrete-case.
Matched Filter in Colored Noise: Returning back to the matched filter
problem in colored noise, the design can be completed as shown in figure
below
h0(t)=sg(t0 t)1( ) ( )G j L j =
0t t=
( ) ( )g
s t n t +( ) ( ) ( )r t s t w t = +
Whitening Filter Matched Filter
where G(j) represents the whitening filter associated with the noise
spectral density SWW() as in (3-55)-(3-58). Notice that G(s) is the inverse
of the Wiener factorL(s) corresponding to the spectrum SWW() i.e.,
(3-64)
The whitened output sg(t) + n(t) in the figure is similar to (3-38), and from
(3-45) the optimum receiver is given by 0 0( ) ( )gh t s t t =
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
31/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT31
3. Matched Filter (15)
1( ) ( ) ( ) ( ) ( ) ( ).g gs t S G j S L j S
= =where
If we insist on obtaining the receiver transfer functionH() for the original
colored noise problem, we can deduce it easily from the below figure
( )H 0
t t=
L-1(s) L(s) ( )H 0
t t=
( )r t( )r t
where0
0
1 1 *0
1 1 *
( ) ( ) ( ) ( ) ( )
( ){ ( ) ( )}
j t
g
j t
H L j H L S e
L L S e
= =
=(3-65)
0()H
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
32/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT32
3. AM/FM Noise Analysis (1)
0( ) ( ) cos( ) ( ),X t m t t n t = + +
0( ) cos( ( ) ) ( ),X t A t t n t = + + +
Consider the noisy AM signal
and the noisy FM signal
where
0( ) FM
( ) ( ) PM.
t
c m d
t cm t
=
(3-66)
(3-67)
(3-68)
Here m(t) represents the message signal and a random phase jitter in
the received signal. In the case of FM, (t) = (t) = c m(t) so that the
instantaneous frequency is proportional to the message signal. We will
assume that both the message process m(t) and the noise process n(t) arew.s.s with power spectra Smm() and Snn() respectively. We wish to
determine whether the AM and FM signals are w.s.s, and if so their
respective power spectral densities.
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
33/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT33
3. AM/FM Noise Analysis (2)
0
1( ) ( ) cos ( )2
XX mm nnR R R = +
Solution: AM signal: In this case from (3-66), if we assume U(0, 2),
then
so that (see figure below)
0 0( ) ( )( ) ( ).2
XX XX
XX nn
S SS S
+ += +
( )mmS
0
00
( )XX
S 0
( )mm
S
(a) (b)
(3-69)
(3-70)
Thus AM represents a stationary process under the above conditions.
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
34/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT34
3. AM/FM Noise Analysis (3)
FM signal: In this case (suppressing the additive noise component in (3-67),
we obtain2
0
0
2
0
0
2
0
( / 2, / 2) {cos( ( / 2) ( / 2) )
cos( ( / 2) ( / 2) )}
{cos[ ( / 2) ( / 2)]2
cos[2 ( / 2) ( / 2) 2 ]}
[ {cos( ( / 2) ( / 2))}cos2
{sin( ( / 2) ( / 2))
XXR t t A E t t
t t
AE t t
t t t
AE t t
E t t
+ = + + + +
+ +
= + +
+ + + + +
= +
+ 0}sin ]
since
0
0
0
{cos(2 ( / 2) ( / 2) 2 )}
{cos(2 ( / 2) ( / 2))} {cos 2 }
{sin(2 ( / 2) ( / 2))} {sin 2 } 0.
E t t t
E t t t E
E t t t E
+ + + +
= + + +
+ + + =
0
0
(3-71)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
35/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT35
3. AM/FM Noise Analysis (4)
2
0 0( / 2, / 2) [ ( , ) cos ( , ) sin ]2XXA
R t t a t b t + =
( , ) {cos( ( / 2) ( / 2))}a t E t t = +
( , ) {sin( ( / 2) ( / 2))}b t E t t = +
Eq (3-71) can be rewritten as
where
and
22
2
( )( ) ( )mm
d RR c R
d
= =
In general, a(t,) and b(t,) depend on both tand so that noisy FM is not
w.s.s in general, even if the message process m(t) is w.s.s. In the special
case when m(t) is a stationary Gaussian process, from (3-68), (t) is also a
stationary Gaussian process with autocorrelation function
for the FM case.
(3-72)
(3-73)
(3-74)
(3-75)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
36/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT36
3. AM/FM Noise Analysis (5)
2 2( (0) ( )).Y
R R
=
22 2 ( (0) ( ))/ 2{ } Y R Rj Y
E e e e = =
{ } {cos } {sin } ( , ) ( , ),jYE e E Y jE Y a t jb t = + = +
In that case the random variable2( / 2) ( / 2) ~ (0, )
YY t t N = +
where
Hence its characteristic function is given by
which for = 1 gives
where we have made use of (3-76) and (3-73)-(3-74). On comparing
(3-79) with (3-78), we get( (0) ( ))
( , ) R R
a t e
=
and
(3-76)
(3-77)
(3-78)
(3-79)
(3-80)
( , ) 0b t (3-81)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
37/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT37
3. AM/FM Noise Analysis (6)
so that the FM autocorrelation function in (3-72) simplifies into2
( (0) ( ))
0( ) cos .2XX
R RAR e
=
Notice that for stationary Gaussian message input m(t) (or (t)), the
nonlinear outputX(t) is indeed strict sense stationary with autocorrelation
function as in (3-82).
FindRXX() for narrowband FM and wideband FM?
(3-82)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
38/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT
Chapter 4:
Eigenanalysis
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
39/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT39
4. Eigenanalysis: Definitions (1)
Let Rdenote (MM)-correlation matrix of wide-senseDTStP represented by (M1)-observation vector u(n).Finding (M1)-vector qsuch that:
for some constant , rewriting in form:
(4.2) has nonzero solution in vector qif:
Equation (4.3) called characteristic equation, whose
roots 1, 2,, Mcalled eigenvalues.General, (4.3) has
Mdistinct roots Msolutions in the vector q.
qRq = (4.1)
( ) 0qIR = (4.2)
( ) 0det = IR (4.3)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
40/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT40
4. Eigenanalysis: Definitions (2)
Let I denote i-th eigenvalue of matrix R,qibe a
nonzero vector such that:
Vector qicalled eigenvectorassociated with i
Example: see pp. 161-162, [1]
iii qRq = (4.4)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
41/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT41
4. Eigenanalysis: Properties (1)
1, 2,, Mare eigenvalues of correlation matrix R, theneigenvalues of correlation matrix Rkequal 1
k, 2k,, M
kfor any k > 0.
Let q1, q2,, qMbe eigenvector corresponding to distincteigenvalues 1, 2,, Mof (MM)-correlation matrix R.Then, q1, q2,, qMare linearly independent.
Let1, 2,, Mbe eigenvalues of (MM)-correlationmatrix R. Then all these eigenvalues are real andnonnegative.
Let q1, q2,, qMbe eigenvector corresponding to distincteigenvalues 1, 2,, Mof (MM)-correlation matrix R.Then, q1, q2,, qMare orthogonal to each other.
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
42/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT42
4. Eigenanalysis: Properties (2)
Let q1, q2,, qMbe eigenvector corresponding to distincteigenvalues 1, 2,, Mof (MM)-correlation matrix R.
Define (MM)-matrix:
where
Define (MM)-diagonal matrix:
Then, original matrix Rmay be diagonalized as:
where Qis nonsingular with Q-1=QH Qis unitarymatrix.
[ ]MqqqQ ,,, 21 =
==
ji
jij
H
i
,0
,1qq
),,,(diag 21 M =
RQQ =H (4.5)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
43/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT43
4. Eigenanalysis: Properties (3)
(4.5) can be written:
Let1, 2,, Mbe eigenvalues of (MM)-correlation
matrix R. Then sum of these eigenvalues equals thetrace ofR:
traceof a square matrix defined as sum of diagonalelements of the matrix.
=
==M
i
H
iii
H
1
qqQQR (4.6)
[ ] =
=M
i
i
1
tr R (4.7)
4 i i i (4)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
44/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT44
4. Eigenanalysis: Properties (4)
Condition number = eigenvalue spread = eigenvalue
factor:
Eigenvalues of correlation matrix of a DTStP are
bounded by min and max values of power spectraldensity of the process:
Condition number is bounded by:
min
max)(
=R (4.8)
MiSS i ,...,2,1,maxmin = (4.9)
min
max
min
max)(S
S=
R
4 Ei l i P i (5)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
45/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT45
4. Eigenanalysis: Properties (5)
Minimax Theorem: Let (MM)-correlation matrix Rhave eigenvalues1, 2,, Mthat are arranged in
decreasing order as:
Then:
where Cis subspace of vector space of all(M1)-complex vector; dim(C): dimension of subspace C.
Subspace of dimension kdefined as set of complexvectors that can be written as a linear combination ofq1,q2,,qk
M 21
MkH
H
CkCk ,,2,1,maxmin)dim( ==
= xx
Rxx
0xx (4.10)
4 Ei l i P i (6)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
46/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT46
4. Eigenanalysis: Properties (6)
Karhunen-Loeve expansion: Let (M1)-vector u(n)drawn from a wide-sense stationary process of zero
mean and correlation matrix R. Letq1, q2,, qMbe
eigenvector corresponding toMeigenvalues of R.
Vectoru(n) may be expanded as linear combination of
these eigenvector:
Coefficients of the expansion are zero-mean,
uncorrelated random variables defined by:
=
=M
i
ii ncn1
)()( qu (4.11)
)()( nnc Hii uq=(4.12)
4 Ei l i P ti (7)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
47/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT47
4. Eigenanalysis: Properties (7)
where
Physical interpretation: viewing eigenvectors q1,q2,,qM as coordinates of anM-dimensional space,thus representing random vector u(n) by set of itsprojections c1(n), c2(n), , cM(n) on these axes.
From (4.11):
From (4.12), (4.13):
MincE i ,,2,1,0)]([ ==
==
jijincncE iji
0)]()([ (4.13)
2
1
2)()( nnc
M
i
i u==
MincE ii ,,2,1,)(2
==
(4.14)
(4.15)
4 Ei l i L R k M d li (1)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
48/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT48
4. Eigenanalysis: Low-Rank Modeling (1)
Dimensionality reduction: Transformation in a way that adata vector(in data space) can be represented by a reduced
number of effective feature (infeature space) and yetretain most of intrinsic content of input data.
ConsiderM-dimensional data vector u(n) (representingrealization of a wide sense stationary process) that may be
expanded as a linear combination of eigenvectors q1, q2,,qMcorresponding to distinct eigenvalues 1, 2,, Mof(MM)-correlation matrix R of u(n). See (4.11). If knowing
prior knowledgethat (M-p) eigenvalues p+1,, Mare allvery small, then truncating (4.11) at term i=p. Then, an
approximate reconstructionof data vector u(n) can bedefined:
4 Ei l i L R k M d li (2)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
49/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT49
4. Eigenanalysis: Low-Rank Modeling (2)
Vector u(n) has rankp
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
50/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT50
4. Eigenanalysis: Low-Rank Modeling (3)
Reconstruction error vector:
Mean-square error:
Error is small good data reconstruction ifeigenvalues
p+1
,, M
are all very small.
Example: Application of Low-Rank Modeling (see
pp.179-180, [1])
+===M
pi
ii ncnnn1
)()()()( quue (4.17)
[ ] +===M
pi
inE1
2)( e (4.18)
4 Ei l i Ei filt (1)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
51/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT51
4. Eigenanalysis: Eigenfilters (1)
Design optimum FIR filter with optimization criterion
of maximizing output signal-to-noise ratio
4 Ei l i Ei filt (2)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
52/53
Dept. of Telecomm. Eng,
Faculty of EEE
SSP2013
DHT, HCMUT52
4. Eigenanalysis: Eigenfilters (2)
Effects of signal only: Average power of signal at
filter output:
Effects of noise only: Average power of noise at filter
output:
Output signal-to-noise ratio:
RwwH
oP = (4.18)
wwHoN2= (4.19)
ww
RwwH
H
o
ooN
PSNR 2)( == (4.20)
4 Eigenanal sis: Eigenfilters (3)
-
8/13/2019 Ssp (Meng Pfiev) Chapter3
53/53
Dept of Telecomm Eng SSP2013
4. Eigenanalysis: Eigenfilters (3)
Optimum problem: max (SNR)osubject to wHw=1.
Applying minimax theorem (4.10):
Optimum FIR filterthat produces (4.21) having
coefficient vectorwo=qmax is called eigenfilter,where qmax: eigenvector associated with max.
Note: Eigen filtermaximizes the output SNR for a
random signal(w.s.s) in additive white noise,while a matched filtermaximizes the output SNR fora known signalin additive white noise.
2max
max,)(
=oSNR (4.21)