chapter 2 - discrete time signals and systems
TRANSCRIPT
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
1/812-1
2. Discrete Time Signals and Systems
We will review in this chapter the basic theories of discrete time signalsand systems. The relevant sections from our text are 2.0-2.5 and 2.7-2.10.
The only material that may be new to you in this chapter is the section onrandom signals (Section 2.10 of Text)
2.1 Discrete Time Signals
A discrete-time (DT) signal is signal that exists at specific time instants.The amplitude of a discrete-time signal can be continuous though.
When the amplitude of a DT signal is also discrete, then the signal is adigital signal.
A DT signal can be either real or complex. While a real signal carries onlyamplitude information about a physical phenomenon, a complex signal
carries both amplitude and phase information.
Throughout this course, we use square brackets [ ] to denote a DT signaland round brackets ( )g to denote a continuous time signal.
Example: If the n-th sample of the DT signal [ ]x n is the value of the
analog signal ( )ax t at t nT= , then
[ ] ( )a
x n x nT=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
2/812-2
Some common DT signals are1.Unit sample
1 0[ ]
0 otherwise
nn ==
2.Unit step1 0
[ ]
0 0
nu n
n
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
3/812-3
[ ] nx n A=
where both andA are real. If 0A > and 0 1< < , then [ ]x ndecreases as n increases; see figure below.
4.Sinusoidal( )[ ] cos ox n A n = +
where A is the amplitude, o is the frequency, and is the phase.
5.Periodic[ ] [ ]x n N x n+ =
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
4/812-4
for any time index n . Here N denotes the period.
Exercise: Is the sinusoidal signal defined above periodic in general?
Example: Express the unit step function in terms of the unit-impulseAnswers:
(a) [ ] [ 1] [ ]u n u n n =
(b)0
[ ] [ ]k
u n n k
=
=
Example: Express1 2,3,...,10
[ ]0 otherwise
nx n
==
in terms of the unit step function.
Answer:
[ ] [ 2] [ 11]x n u n u n=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
5/812-5
Example: Express the sinusoidal signal in terms of complex exponentialsignals
Answer:
( )( ) ( )
( ) ( )1 1 2 2
[ ] cos
2
o o
o
j n j n
n n
x n A n
e eA
A A
+ +
= +
+=
= +
where
*
1 22
jAeA A
= =
*
1 2oje
= =
It is also possible to express the signal as
( ){ }1 1[ ] 2 Ren
x n A =
Example: Express an arbitrary DT signal in terms of the unit impulseAnswer
[ ][ ] [ ]k
x n x k n k
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
6/812-6
2.2 Discrete Time Systems
Let [ ]T represents the transformation a discrete time system performedon its input [ ]x n . The corresponding output signal of the system is
[ ][ ] [ ]y n T x n= .
The system is linear if[ ]1 2 1 2[ ] [ ] [ ] [ ]T ax n bx n ay n by n+ = +
where 1 2[ ] and [ ]y n y n are the responses of the system to inputs of
1 2[ ] and [ ]x n x n respectively.
The above equation illustrates theprinciple of superposition.
Assume the system is linear and let [ ]hh n be the output of the system whenthe input is
[ ] [ ]kp n n k=
(i.e. a unit-impulse at time n k= ). Then according to the linearity property,
[ ]y n[ ]T [ ]x n
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
7/812-7
when the input is
[ ]
[ ]
[ ] [ ]
[ ]
k
k
k
x n x k n k
x k p n
=
=
=
=
,
the output will be
[ ][ ] [ ] kk
y n x k h n
=
=
The system is time-invariantif[ ] [ ]kh n h n k = ,
i.e. the output is delayed if the input is delayed. In this case
[ ][ ] [ ]
[ ] [ ]
k
y n x k h n k
x n h n
=
=
=
The signal [ ]h n is called the impulse response of the time-invariantsystem.
While the focus of this course is on linear, time-invariant (LTI) system,there are many real-life applications where the system is non-linear andtime-variant. A good example is a digital FM demodulator operating in the
mobile radio environment.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
8/812-8
Example: Provide a physical interpretation of a LTI system whose impulseresponse is
[ ] [ ]h n u n=
Answer: The output of the system is
[ ]
[ ]
1
[ ] [ ]
[ ]
[ ]
[ ] [ ]
[ 1] [ ]
k
k
n
k
n
k
y n x k h n k
x k u n k
x k
x k x n
y n x k
=
=
=
=
=
=
=
= +
= +
Thus the system is an integrator.
A system is casual if and only if the output at time n depends only on theinput up to time n . According to the equation
[ ][ ] [ ]ky n h k x n k
== ,
this means the impulse response [ ]h n is zero when 0n < .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
9/812-9
Example: A moving averager computes the signal2
11 2
1[ ] [ ]
1
M
k M
y n x n kM M =
= + +
from its input [ ]x n . Here 1M and 2M are positive integers. What is the
impulse response of the system? Is the system casual?
Answer:
The output can be rewritten as
1
2
1 2
1 2
1
1 2
1[ ] [ ]
1
1[ ] [ ]
1
n M
m n M
n M n M
m m
y n x mM M
x m x mM M
+
=
+
= =
=+ +
= + +
Compared to the output of the integrator, we can deduce that the impulse
response of the system is
[ ] [ ]( )1 21 2
1[ ] 1
1h n u n M u n M
M M= +
+ +
Since the impulse response is non-zero when 0n < , so the system is notcasual.
A real physical system can not be non-casual, i.e. it can not generate anoutput before there is an input. So in practice what a non-casual system
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
10/812-10
means is that there is a processing delay. For example, you can view themoving averager as a device that computes the local mean of the signal
[ ]x n at time n after it observes the sample 1[ ]x n M+ . So 1M is the delay.
A system is stable if a bounded input results in a bounded output. Therequirement for having a stable system can be derived from the
input/output relationship of a LTI system, which states that
[ ][ ] [ ]k
y n x k h n k
=
=
This means
[ ]
[ ]
[ ] [ ]max max
[ ] [ ]
[ ]
k
k
k m
y n x k h n k
x k h n k
x h n k x h m
=
=
= =
=
=
where is the absolute value operator and maxx is the largest magnitude of
the input signal.
So if the impulse response of the system is absolute-summable, i.e. when
[ ]k
S h k
=
= <
then the system is stable.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
11/812-11
Example: Is the integrator a stable system?Answer: Since [ ] 1h n = for 0n and zero otherwise, the impulse responseis not absolute-summable. Consequently the system is not stable.
Example: Is the moving averager a stable system?Answer: Yes, because the impulse response consists of only a finite
number of non-zero samples.
Finite Impulse Response (FIR) andInfinite Impulse Response (IIR):FIR => An impulse response of finite duration, hence a finite number of
non-zero samples. Always stable.
IIR => The impulse response is infinitely long. Can be unstable (for
example the integrator).
Example: Comment on the stability of a LTI system with the exponentialimpulse response
0[ ]
0 otherwise
na nh n
=
Solution:
0 0
[ ]kk
k k k
h k a a
= = =
= =
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
12/812-12
This is summable if 1a < . In this case,
1[ ]
1kS h k
a
=
= =
Cascadingof LTI systems serial connection of two or more systems; seethe example below.
As far as the input/output relationship is concerned, it really does not
matter what the order of the concatenation is. For the example above, bothpossibilities yields the same combined impulse response of
1 2[ ] [ ] [ ]h n h n h n=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
13/812-13
In many applications, we have to concatenate a system to an existing oneso that the combined system yields the desired response. A good example isthe equalizer used in a digital communication system.
Many communication channels introduces intersymbol interference (ISI)
ef. This means the received signal [ ]r n depends not only on the data bit[ ]b n , but also on some adjacent bits. For example,
1 1[ ] [0] [ ] [1] [ 1]r n h b n h b n= +
where 1[ ]h n represents the impulse response of the channel. The objective
of equalizer design is to find a digital filter with an impulse response 2 [ ]h n
so that the combined response of the channel and the equalizer,
1 2[ ] [ ] [ ]h n h n h n= , is the unit-impulse function. This means afterequalization, we have [ ] [ ]y n b n= , i.e. the ISI is removed.
Exercise: Consider an ISI channel with 1[ ] [ ]nh n a u n= , where 0 1a< < ,and [ ]u n is the unit step function. Determine the equalizer that completelyremoves the ISI.
Systems governed by the Linear Constant Coefficient Difference Equation(LCCDE):
1 0
[ ] [ ] [ ]N M
k j
k j
y n a y n k b x n j= =
= +
The above equation suggests that current output of the system depends onthe previous output as well as the current and previous input.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
14/812-14
In analyzing the above system, we assume the input is applied at time0n = (i.e. [ ] 0x n = for negative n ) and the initial state of the system is
defined as
( )[0] [ 1], [ 2],..., [ ]y y y N
= Y
(a) Zero State Response (ZSR) response of the system to an unitimpulse applied at time 0n = , under the condition that [0]Y is theall-zero vector.
(b) Zero Input Response (ZIR) response of the system due to a non-zero initial state but no input.
Example: [ ] [ 1] [ ]y n ay n x n= +Let the initial state be [0] [ 1]y b= =Y , then
2
3 2
[0] [0][1] [0] [1]
[2] [0] [1] [2]
y ab xy a b ax x
y a b a x ax x
= += + +
= + + +
or in general
1
0
[ ] [ ]n
n n k
k
y n a b a x k+
=
= +
The ZIR is1
1[ ] [ ]nh n a bu n+=
and the ZSR is
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
15/812-15
2[ ] [ ]nh n a u n=
It is clear that the ZIR corresponds to the bias term in [ ]y n . Since it is
independent of the input, the system can NOT be classified as a linearsystem. Note that the response of the system to
3 1 1 2 2[ ] [ ] [ ]x n w x n w x n= +
is
{ }
13
0
11 1 2 2
0
[ ] [ ]
[ ] [ ]
nn n k
k
nn n k
k
y n a b a x k
a b a w x k w x k
+
=
+
=
= +
= + +
,
which is different from
3 1 1 2 2[ ] [ ] [ ]y n w y n w y n= + ,where
1
1 1
0
12 2
0
[ ] [ ],
[ ] [ ]
nn n k
k
nn n k
k
y n a b a x k
y n a b a x k
+
=
+
=
= +
= +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
16/812-16
2.3 Fourier Transform of Discrete Time Signals
Consider the sinusoidal signal
( )[ ] cosx n A n = +It can be written in terms of two complex exponential functions as
( ) ( )
1 2[ ]
2
j n j n
j n j ne ex n A A e A e
+ +
+= = +
where
*
1 22
jAA e A= =
The complex signal j ne is an important signal in discrete time signalprocessing it is an eigenfunction of a linear system and it leads us to the
concept of Fourier Transform of a discrete-time signal.
Again let us use [ ]T to represent the operation a discrete time systemperforms on its input. A signal [ ]f n is an eigenfunction of the system if
[ ][ ] [ ]T f n a f n= ,
where the constant a is called an eigenvalue. This definition is consistent
with that in matrix theory where the eigenvector v and the eigenvalue b ofa matrix A is defined as
b=Av v .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
17/812-17
Here the matrix A is analogous to our linear system.
As shown in Section 2.2, the transformation performed by a LTI on itsinput [ ]x n is described by the convolution formula:
[ ] [ ] [ ]k
y n h k x n k
=
= ,
where [ ]h n is the impulse response of the system and [ ]y n is thetransformed signal or output of the system. If
[ ]j n
x n e= ,
then the output signal becomes
( )
( )
[ ] [ ]
[ ]
[ ]
j n k
k
j n j k
k
j n j k
k
j n j
y n h k e
h k e e
e h k e
e H e
=
=
=
=
=
=
=
,
where
( ) [ ]j j kk
H e h k e
=
= .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
18/812-18
It is clear from the above analysis thatj n
e
is indeed an eigenfunction of a
discrete-time LTI system with ( )jH e being the corresponding eigenvalue.
In the linear system literature, ( )jH e is called the frequency response of adiscrete-time LTI system.
In general, the expression
( ) [ ]j j kk
X e x k e
=
=
is called the Fourier Transform of the discrete-time signal [ ]x n .
One important property of the Fourier Transform of a discrete time signalis that it is periodic in with a period of 2 . This is quite different from
the Fourier Transform of a continuous time signal, which in general is notperiodic.
Example: Express the output of a LTI system in terms of its frequencyresponse when the input is the sinusoid ( )[ ] cosx n A n = + . Assume theimpulse response of the sytem is a real signal.
Solution:
- The sinusoidal input can be written as a weighted sum of two complexexponential functions as
1 2[ ]j n j n
x n A e A e = +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
19/812-19
where*
1 2/ 2j
A Ae A= = are the weighting coefficients.
- Since the system is linear, the sinusoidal response is1 1 2 2
[ ] [ ] [ ]y n A y n A y n= +where
( )1[ ]j j
y n H e e =
and
( )2[ ]j j
y n H e e = ,
are the outputs of the system when the inputs are 1[ ]j n
x n e
= and2[ ]
j nx n e = respectively.
- Since the impulse response is real,
( ) ( )*
*[ ] [ ]j j k j k j
k k
H e h k e h k e H e
= =
= = = .
This means we can write the output of the system as
( ) ( )
( ) ( )
( ){ }
( ) ( )
1 2
1 2
[ ] ]
* *
1 1
1
[ ]
2Re
cos ( )
j j n j j n
y n y n
j j n j j n
j j n
j
y n A H e e A H e e
A H e e A H e e
A H e e
A H e n
= +
= +
=
= + +
14243 1442443
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
20/812-20
Note that ( )jH e and ( ) are respectively the magnitude and phase of
the frequency response, i.e.
( ) ( ) ( )j j jH e H e e =
In conclusion, when the input is a sinusoid, the output is also a sinusoid
at the same frequency but with the amplitude scaled by ( )jH e and with
the phase shifted by an amount ( ) .
Example: Determine the frequency response of a delay element describedby the impulse response[ ] [ ]h n n d =
Solution
( ) [ ] [ ]j j n j n j d
n nH e h n e n d e e
= == = = This means
( ) 1jH e = (constant magnitude response)
and
( ) d = (linear phase)
Example: Determine the Fourier Transform of the one-sided exponentialsignal
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
21/812-21
[ ] [ ]nx n a u n=
where 0 1a< < and [ ]u n is the unit-step function.
Solution:
( ) ( )0 0
1[ ]
1
nj j n n j n j
jn n n
X e x n e a e aeae
= = =
= = = =
Since
( )
( )( ) ( )
( ) ( ) ( ){ }
( )
2 2 2
2 22 2 2 2
2 2 2
2
1 1 cos( ) sin( )
1 cos( ) sin( )1 cos( ) sin ( )
1 cos( ) sin ( ) 1 cos( ) sin ( )
1 cos( ) sin ( ) cos ( ) sin ( )
1 cos( )
jae a ja
a aa a j
a a a a
a a j
a a
= +
= + + + +
= + +
= + ( )2 2sin ( )exp ( )
j
where
sin( )( )
1 cos( )
a
a
=
,
this means the magnitude of the Fourier transform is
( ) ( )2 2 21
1 cos( ) sin ( )
jX e
a a
= +
and the phase is simply
( ) ( ) = .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
22/812-22
Existence of the Fourier Transform:- If we set the parameter a in the above example to unity, then the signal
becomes a unit-step. The Fourier Transform in this case, however, does
not exist in the finite magnitude sense.
- A sufficient condition for the existence of the Fourier transform (in thefinite-magnitude sense) is that the signal is absolute-summable, i.e.
[ ]k
S x k
=
= <
The proof is the same as that we used to proof the stability of a LTI
system.
- We can deduce from the above that the Fourier Transform always existsfor signals with finite duration.
Example: Determine the Fourier Transform of the signal1/( 1) 0
[ ]0 otherwise
M n Mx n
+ =
Solution
( )( )
( ) ( ) ( ){ }{ }
( )( )( )
1
0
1 / 2 1 /2 1 / 2
/2 /2 /2
12/2
2
1 1 1[ ]
1 1 1
1
1
sin1
1 sin
j MMj j n j n
jn n
j M j M j M
j j j
Mj M
eX e x n e e
M M e
e e e
M e e e
eM
+
= =
+ + +
+
= = =+ +
=
+
=+
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
23/812-23
The magnitude of the transform is
( )( )( )( )
12
2
sin1
1 sin
M
jX eM
+=
+
At a first glance, the phase of the Fourier Transform is ( ) / 2M = .However, the sin( ) / sin( )g function can take on either + or ve value.When there is a sign change in this function, that corresponds to an
additional 180 degree phase shift.
Plots of the magnitude and phase of the transform for the case of 4M =are shown below.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
24/812-24
The Inverse Fourier Transform is defined mathematically as( )
1[ ]
2
j j nx n X e e d
=
Proof:
( )
( )
( )
1 1[ ]
2 2
1[ ]
2
1[ ]
2
j j n j k j n
k
j n k
k
j n k
k
X e e d x k e e d
x k e d
x k e d
=
=
=
=
=
=
Since
( ) ( )
( )
( ) ( )
( ) ( )
1 1
2 2
1
2
1
2
sin sin
sinc
c c
c c
c
c
c c
j mj m
j m
j m j m
c cc
c
c c
e d e d j mj m
ej m
e e
m j
m m
m m
m
=
=
=
= = =
this means
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
25/812-25
( )
( )
( )1 1[ ]2 2
[ ] sinc
[ ] [ ]
[ ]
j j n j n k
k
k
k
X e e d x k e d
x k n k
x k n k
x n
=
=
=
=
=
=
=
Physically, the inverse Fourier transform states that the time domain signalis the sum of infinitesimally small complex sinuoids of the form( )j j nX e e d
where ( )jX e denotes the relative amount of each complex sinusoidalcomponent. Consequently, it is a synthesizingformula.
Example: Determine the impulse response of a LTI system whosefrequency response is given by
( ) 412
1
1
j
jH e
e
=
Solution:
We first rewrite the frequency response as
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
26/812-26
( )
( )
( )
412
412
0
412
0
1
1
j
j
kj
k
k j k
k
H ee
e
e
=
=
=
=
=
Comparing the above infinite series with the definition of a Fourier
transform, we come to the conclusion that
( )/ 4
12
0,4,8,12,...[ ]
0 otherwise
nn
h n ==
Example: The frequency response of an ideal low pass (i.e. brickwall)filter is
( )1
0 otherwise
j c
lpH e =
What is the corresponding impulse response?
c
c
1
( )j
lpH e
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
27/812-27
Solution:
( )
( )
1[ ]
2
1
2
sin
c
c
j j n
lp lp
j n
c
h n X e e d
e d
n
n
=
=
=
Observation: the impulse response of an ideal low pass filter is NOT
absolute summable.
Question: But then why does the frequency response exist?
Answer: The absolute-summability of a signal is a sufficient condition for
( )jH e < , not a necessary condition.
If we impose the constraint that the magnitude of a valid Fourier Transformmust be finite, is there any reason why we shouldnt impose the constraint
that the derivative(s) of a Fourier Transform should also be finite? After all,
any paremeter associated with a real world signal should be finite, right?
If we impose this additional constriant on the derivative, then the ideal lowpass filter is not a valid frequency response because of the discontinuities
in the spectrum.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
28/812-28
Exercise: Show that a sufficient requirement for( )j
dH e
d
<
is
( )k
k h k
=
<
Verify the result using the ideal low pass example.
It appears that if we were to be able to deal with a wide variety of signalsin our analysis, we should relax on the requirement that magnitude of avalid transform or its derivative(s) must be finite. This leads us to the
impulse function in the frequency domain ( ) . Some important
properties of this function are:
1. ( )0 is undefinied by infinitely large,2. ( ) 0 = for 0 , and
3. ( ) ( ) (0)f d f
=
With the introduction of this function, we can now have proper definitions
for the Fourier transforms of signals such as a DC signal, a complexsinusoidal, and the unit-step signal. The fact that these signals exist at
discrete frequencies is consistent with the above properties of the impulse
function.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
29/812-29
The Fourier Transform of the DC signal[ ] 1x n =
is
( )
( )
[ ]
2 2
j j n
n
j n
n
n
X e x n e
e
n
=
=
=
=
=
= +
Proof:
The function ( )jX e can be treated as an analog signal in . Since this
analog signal has a period of 2P = , it can be represented by thecomplex Fourier series
( )2
expj
k
k
X e X j kP
=
=
(1)
where
( )/ 2
/ 2
1 2exp
P
j
k
P
X X e j k dP P
= (2)
is the k-th complex Fourier coefficient. Substituting
( ) ( )2 2jn
X e n
=
= +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
30/812-30
into (2) yields
( )
( )
/ 2
/ 2
1 2exp
1 22 2 exp
2 2
1
P
j
k
P
n
X X e j k dP P
n j k d
=
=
= +
=
Substituting 1kX = into (1) yields
( )
( )
2exp
2exp
2
exp
j
kk
k
k
X e X j kP
j k
jk
=
=
=
=
=
=
The Fourier Transform of the complex sinuoid[ ] oj nx n e =
is
( )( )0
oj nj j n
n
j n
n
X e e e
e
=
=
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
31/812-31
This is simply the Fourier Transform of the DC signal shifted to the
frequency o = . Consequently,
( ) ( )2 2j on
X e n
=
= +
It can be shown that the Fourier Transform of the unit-step function is( ) ( )
12
1
j
jn
U e ne
=
= + +
2.4 Properties of Fourier Transforms
Linearity:If
( )
( )
1 1
2 2
[ ] ,
[ ]
j
j
x n X e
x n X e
then
( ) ( )1 2 1 2[ ] [ ]j j
ax n bx n aX e bX e
+ +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
32/812-32
Time Shi f ting and F requency Shi f ting
[ ] [ ]
[ ] ( )
[ ]
( )
j n
n
j n d j d
n
j m j d
m
j d j
x n d x n d e
x n d e e
x m e e
e X e
=
=
=
=
=
=
[ ] [ ]
[ ]( )
( )
( )
0
0
o oj n j n j n
n
j n
n
j
e x n x n e e
x n e
X e
=
=
=
=
Time reversal
[ ] [ ]
[ ]
[ ]
( )
( )( )
( )
j n
n
j n
n
j m
m
j
x n x n e
x n e
x m e
X e
=
=
=
=
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
33/812-33
We showed earlier that if [ ]x n is real, then
( ) ( )*j jX e X e = .
So if [ ]x n is real, then
[ ] ( )* jx n X e
Di ff erentiation in F requency
( ) [ ]
[ ]
[ ]( )
[ ]
j j k
k
j k
k
j k
k
j k
k
d dX e x k ed d
dx k e
d
x k jk e
j kx k e
=
=
=
=
=
=
=
=
The above implies
( )[ ] jd
nx n j X ed
Convolution:If 3 1 2[ ] [ ] [ ]x n x n x n= , then( ) ( ) ( )3 1 2
j j jX e X e X e =
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
34/812-34
Proof: Since
3 1 2
1 2
[ ] [ ] [ ]
[ ] [ ]k
x n x n x n
x k x n k
=
=
= ,then
( )
( )
( )
3 3
1 2
1 2
1 2
1 2
[ ]
[ ] [ ]
[ ] [ ]
[ ] [ ]
[ ] [ ]
j j n
n
j n
n k
j n k j k
n k
j n k j k
k n
j m
X e x n e
x k x n k e
x k x n k e e
x k x n k e e
x k x m e
=
= =
= =
= =
=
=
=
=
=
( )
( )
( ) ( )
1 2
2 1
1 2
[ ]
[ ]
j k
k m
j j k
k
j j k
k
j j
e
x k X e e
X e x k e
X e X e
= =
=
=
=
=
=
The result is known as the Convolution Theorem.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
35/812-35
Energy of a signal and the Parseval s TheoremThe energy of a discrete-time signal is defined as
( )22 1
[ ] 2j
nE x n X e d
= = =
The result is known as the Parsevals Theorem.
Proof:
Let [ ]
*[ ]h n x n=
This means
( ) ( )*j jH e X e = .
If
*
[ ] [ ] [ ]
[ ] [ ]
[ ] [ ]
k
k
y n x n h n
x k h n k
x k x k n
=
=
= =
=
,
then
( ) ( ) ( ) ( )2j j j j
Y e X e H e X e = =
Taking the inverse Fourier Transform of ( )jY e at 0n = yields
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
36/812-36
( )
( )
2
2
1[0] [ ]
2
1
2
j
k
j
y x k Y e d
X e d
=
= =
=
The Windowing Theorem
( ) ( )
( ) ( )
( ) ( ) ( )
( )
[ ] [ ] [ ]
1 1 2 2
1 1
2 2
1 1
2 2
1 1
2 2
j jn j jn
j jn j jn
jnj j
j
y n x n w n
X e e d W e e d
X e e d W e e d
X e W e e d d
X e W
+
= =
= =
=
=
=
=
=
( )( )
( ) ( )( )
( )
1 1
2 2
1
2
j jn
jj jn
j jn
e e d d
X e W e d e d
Y e e d
= =
=
=
=
where
( ) ( ) ( )( )1
2
jj jY e X e W e d
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
37/812-37
It is important to realize that the above is a circular convolution in thefrequency domain.
Example: Use the Windowing Theorem to determine the FourierTransform of the signal
( )sin
[ ]0 otherwise
cn
nM n M
y n
=
Solution
This signal can be considered as the product of the ideal low pass signal
( )sin[ ]
c
lp
nh n
n
=
and the rectangular window
1[ ]
0 otherwise
M n Mw n =
As shown earlier,
( )1
0 otherwise
j c
lpH e =
.
It can also be shown that (please verify)
( )( 1) ( 1)1 1
11 1
j M j Mj
j j
e eW e
e e
+ + +
+
= +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
38/812-38
Results for different values ofM are shown below
Note that ( )jMH e is this figure is equivalent to ( )jY e
Observations:
- the oscillation (also known as the Gibbs phenomenon) is more rapid forlargerM,
- the amount of ripples does not decrease though
Exercise: Plot ( )jW e and find out what causes the ripples in thesediagrams.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
39/812-39
2.5 Sampling of an Analog Signal
Reference: Sections 4.0-4.3, 4.6 of Text
Now that we know what the Fourier Transform of a discrete time signal[ ]x n is, we want to relate it to the Fourier transform of its continuous-time
counterpart ( )cx t , under the condition that
[ ] ( )c
x n x nT= ,
where T is the sampling period and
1sf
T=
is thesampling rate.
Let( ) ( ) j tc cX j x t e dt
=
be the continuous-time (CT) Fourier transform of ( )cx t , where is theanalog frequency. This means
( )1
( ) (inverse Fourier Transform)2
j t
c cx t X j e d
= .
Since [ ] ( )cx n x nT= , this means
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
40/812-40
( )
( )( )
( )
( )( )( )
( )
( )( )
212
212
2/
2 2
/
/
2
/
1[ ] ( )
2
12
1
2
1
2
T
T
T
j nT
c c
k
j nTc
k k
Tj k nT
c T T
k T
T
j nT
c T
T
x n x nT X j e d
X j e d
X j k e d k
X j k e d
+
=
+
=
= =
=
= + +
= +
( )( )
( )( )
( )( )
/
2
/
/
2
/
2
1
2
1
2
1 1
2
k
T
j nT
c T
kT
T
j nT
c T
kT
j n
c T T
k
X j k e d
X j k e d
X j k e d
T
=
=
=
=
= +
= +
= +
In comparing the above with the inverse Fourier transform of a DT signal,
we come to the conclusion that
Exercise: Show that the ( )jX e shown above is periodic in with aperiod of 2 .
( ) ( )( )21j
c T T
k
X e X j kT
=
= +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
41/812-41
To construct ( )jX e from ( )cX j , we can adopt the following procedure1.Divide ( )cX j into intervals of width 2 / T according to the formula
k-th interval:2 2
k kT T T T
< +
2.Define the spectral segment in the k-th interval mathematically as
( )( ) ( ) ( )1 12 22 / 2 /
0 otherwise
c
k
X j k T k TX j
< + =
3.Shift the different spectral segments to the center of the spectrum andadd them together according to
( ) ( )( )2k Tk
Y j X j k
=
= +
4.The Fourier transform of the DT signal is obtain by mapping the analogfrequency in ( )Y j to T = , i.e.
( )1
,j
X e Y jT T
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
42/812-42
Example: Sampling of a triangular analog spectrum with a one-sidedbandwidth of 480W= rad/s. The sampling frequency, in rad/s, is
2640s
T
= =
Fig: (a) ( )cX j , (b) 1( )X j , (c) 0( )X j , and (d) 1( )X j
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
43/812-43
Fig: (a) the various shifted spectra ( )kX j , (b) the summed spectrum ( )Y j ,and (c) the Fourier transform ( )jX e . Note that the x-variable in diagram (c)
is the digital frequency divided by .
It is observed that the summed spectrum ( )Y j is no longer triangular.Consequently the Fourier transform ( )jX e in the interval [ , ] is also not
triangular. This is known as the aliasing effectand is caused by using toosmall a sampling frequency.
The aliasing effect can be eliminated if the sampling frequency satisfies theNyquist criterion :
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
44/812-44
22
sW
T
=
where W is the (one-sided) bandwidth of the analog signal ( )cx t in rad/s.
Proof:
Consider the term ( )12 2 /k T for any 0k> . If the Nyquist samplingcriterion is satisfied, this means
1 2 1 12
2 2 2s
k k k W W
T
=
This means the spectral segments ( )kX j , k=1,2,, are all zero.Similarly, it can be shown that all the spectral segments with negative kare
zero. Consequently,
( )1j
cX e X j
T T
=
,
i.e. the shape of ( )cX j is preserved after sampling. In this case, theFourier transform of the sampled signal is simply a (frequency and
amplitude) scaled version of ( )cX j .
For the previous example involving the triangular ( )cX j , a samplingfrequency of 640 rad/s was used. This is less than the Nyquist frequency of
2 960W= rad/s. Consequently there is aliasing in the sampled signal.When the sampling frequency is at or above the Nyquist frequency, say
960, 2 960, 3 960s = rad/s, then the aliasing disappears and ( )jX e
becomes
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
45/812-45
Fig: Effect of varying the sampling frequency on ( )jX e : (a) the sampling frequency
is at the Nyquist rate of 960 rad/s, (b) the sampling frequency is twice the
Nyquist rate or 1920 rad/s, and (c) the sampling frequency is three times theNyquist rate or 2880 rad/s.
It is evident from the diagram that the spectral width of ( )jX e in theinterval [ , ] is proportional to the bandwidth to sampling-rate ratio
2
s
W =
The larger s is, the narrower the spectrum of the digital signal.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
46/812-46
2.5.1 Down Sampling by an I nteger Factor
Suppose ( )cx t was sampled at a rate of1/ T Hz to obtain
[ ] ( )cx n x nT=A down sampler will convert [ ]x n to another DT-signal [ ]dx n according to
[ ][ ]dx n x nM=
Example:M=2
[ ]dx n[ ]x n down sampler
M
0
n
[4]x
[2]x
[0]x
0 1 2 3 54
n
[2]d
x
[1]dx
[0]d
x
1 2
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
47/812-47
The samples [1], [3], [5],...x x x are not used in forming the signal [ ]dx n ,i.e. they are decimated. Consequently a down sampler is also known as a
decimator.
What is the relationship between ( )cX j , ( )jX e , and ( )jdX e ?From our earlier discussion we know
( ) ( )( )21j
c T Tk
X e X j kT
=
= +
Since the down-sampled signal [ ]dx n can be viewed as a sampled version
of ( )cx t with a sampling period of
'T MT= ,
so ( )jdX e must be
( ) ( )( )
( )( )
( )( )
( )( )
2' '
2
12
0
12 2
0
1
'
1
1 1
[ ]
1 1
j
d c T Tr
c MT MT r
M
c MT MT i k
M
c MT T MT i k
X e X j rT
X j rMT
X j kM iM T
X j k iM T
=
=
= =
= =
= +
= +
= + +
= + +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
48/812-48
( )( )
( )
12 21
0
1[ 2 ]/
0
1 1
1
Mi
c M T Ti k
Mj i M
i
X j kM T
X eM
+
= =
+
=
= +
=
This equation says that the FT of the down-sampled signal is the
superposition ofM frequency shifted and scaled copies of ( )jX e .
It should be emphasized that the above expression should only be used todetermine ( )jdX e
in the range < . For outside this range,
the value of ( )jdX e
can be determined by making use of the fact that it is
periodic with a period of 2 .
On the other hand if you blindly apply the formula above, you will end up
with a ( )
j
dX e
with a period of 2 .M
Example: for 2M = , we have( ) ( ) ( ){ }/ 2 ( 2 ) / 2
1
2
j j jdX e X e X e
= +
There will be no aliasing effect in [ ]dx n if the sampling frequency of [ ]x n
is at least twice the Nyquist frequency, i.e.
( )2
2 2s WT
= .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
49/812-49
If this is the case, then the effective sampling frequency of [ ]dx n is
( )' 22
ss W
=
which satisfies the Nyquist criterion for zero-aliasing. Alternatively you
can prove this using the diagram below.
Fig: (a) ( )jX e when the sampling frequency is twice the Nyquist frequency, (b)
( )/ 212 jX e , (c) ( )( 2 ) / 212 jX e , and (d) ( )j
dX e .
In general, the sampling frequency of [ ]x n must be at least M times theNyquist rate in order to avoid aliasing in [ ]dx n .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
50/812-50
2.5.2 Up Sampli ng by an I nteger Factor
When there is no aliasing in the sampled signal, then theoretically it ispossible to compute ( )cx t exactly at any value of t from [ ]x n . This
process is known as interpolation.
Argument:
( ) ( )[ ] ( )j c cx n X e X j x t=> => =>
Derivation:
( )
( )
( )
( )
/
/
/
/
/
/
1( )2
1
2
1
2
1
2
1[ ]
2
1[ ]2
j t
c c
T
j t
c
T
j t T
c
j j t T
j n j t T
n
j t T n
x t X j e d
X j e d
X j e d
T T
X e e d
x n e e d
x n e d
=
=
=
=
=
=
=
[ ]sinc
n
n
tx n n
T
=
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
51/812-51
In practice, it is not possible to calculate ( )cx t exactly from its samplesbecause the sinc function is, straightly speaking, infinitely long.
Interpolators that are commonly used in practice are:
- Lagrange,- Cubic spline
Note that sinc( / )t T is the inverse Fourier transform of the following ideallow pass spectrum:
Thus ideal interpolation is equivalent to feeding the discrete time signal
[ ]x n as a sequence of (continuous-time) impulses to an ideal low passfilter, i.e.
( ) [ ] ( ) sinccn
tx t x n t nT
T
=
=
Again, since the ideal low pass filter can never be implemented exactly
because of the abrupt transitions, we can never interpolate the analog signal
exactly from its samples.
/T /T
T
( )lpH j
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
52/812-52
Let[ ] ( ')i cx n x nT=
be the DT signal obtained by sampling ( )cx t at a rate of
( )' 1 1 Hz
' /s
Lf
T T L T = = =
Since this signal can be obtained from the signal
[ ] ( )c
x n x nT=
according
'[ ] [ ]sinc
[ ]sinc
[ ]sinc ,
i
k
k
k
nTx n x k k
T
nTx k k
LT
nx k k
L
=
=
=
= =
=
it is called an up-sampled signal.
[ ]i
x n[ ]x n
up sampler
L
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
53/812-53
What is the Fourier Transform of this up-sampled signal?- Since the sampling rate 1/sf T= used in generating [ ]x n satisfies the
Nyquist criterion ,
( )1j
cX e X j
T T
=
- Since 1/sf T= satisfies the Nyquist criterion, the new sampling rate' /sf L T= used in generating [ ]ix n will also satisfy the Nyquist
criterion. This means
( )
( )
1
' '
/
0 /
j
i c c
j L
L LX e X j X j
T T T T
LX e L
L
= =
= <
In otherword, ( )jiX e is simply a compressed version of ( )jX e .
The existence of the two distinct frequency bands in ( )jiX e suggests(again) a low pass filtering effect in interpolation. This can be traced back
to the sinc function
sinc n kL
in [ ]ix n .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
54/812-54
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
55/812-55
Exercise: What is the Fourier Transform of the signal
[ ] [ ]sincik
nx n x k k
L
=
=
in general? Do not assume that [ ]x k is obtained from sampling an analog
signal.
2.5.3 Changing the Sampli ng Rate by a Non-I nteger Factor
Suppose we want to change the sampling rate of the system by a factor off . How?
Approximate f as a rational numberL
fM
and then up-sample the signal by a factor of L , followed by down-sampling (the up-sampled signal) by a factor ofM .
[ ]y n[ ]x n
up sampler
L down sampler
M
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
56/812-56
2.6 Discrete Time Random Processes
A random variable is a parameter whose value can not be predicted exactly.
Associated with a random variable is its probability density function (pdf).
Example 1: Uniform variable x :
1/( )
( ) 0 otherwisexb a a v b
p v
=
Fig: pdf of a uniform random variable.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
57/812-57
Example 2: Gaussian random variable x :
( )( )
2
22
1exp
2
x
x
xx
v mp v
=
where
[ ]xm E= xand
( )22
x xE m = x
are respectively the mean and variance of x , with [ ]E beingthe average operator.
Fig:pdf of Gaussian random variables.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
58/812-58
Exercise: Determine the mean and variance of a uniform random variable.
Note that[ ]Pr ( )
b
x
a
a b p v dv = xwith
( ) 1x
p v dv
= .
A discrete time signal [ ]x n is a random process when every [ ]x n is arandom variable.
Notations:
1. [ ]n x n=x2.The pdf of nx is ( )nxp v3.The mean of nx is nxm4.The variance of nx is 2nx
If the random process [ ]x n is (wide-sense) stationary, then1. the pdf ( )nxp v is independent of the time index n , and2. the autocorrelation function, defined as
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
59/812-59
[ ] *, [ ] [ ]xx n m E x n x m = ,
depends only on the time difference n m , i.e.
[ ]
[ ]
*, [ ] [ ]
xx
xx
n m E x n x m
n m
= =
Basically what we are saying is that the first and second other statistics of a
(wide-sense) stationary random process do not depend on absolute time. In
this course, we will focus only on these random processes.
Exercise: Show that[ ] [ ]*xx xxn n =
Subsequently show that for a real random process
( ) ( )j je e =
The autocorrelation function at a time difference of 0n = is
[ ] 2*0 [ ] [ ] [ ]xx E x n x n E x n = =
and is called the average powerof the random process [ ]x n . If the random
process has zero mean, i.e. 0nxm = , then
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
60/812-60
[ ]22
2
0 [ ] [ ]
,
nxx x
x
E x n E x n m
= = =
where2
x is the variance of [ ]x n .
We will only focus on zero-mean processes in this course.
The autocorrelation function provides information as to how fast a randomprocess varies with time. A fast process will have a relatively narrow
autocorrelation function. The exact frequency contents of a random process
can be obtained from its power spectral density (psd) function, defined as
( ) [ ]j j nxx xxn
e n e
=
=
Since this equation is simply the Fourier transform of [ ]xx n , consequently
1[ ] ( )
2
j j n
xx xxn e e d
=
Note that
1[0] ( )
2
area under psd function
= average power
j
xx xxe d
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
61/812-61
White noise:A random process is white if there is no correlation between the values of
the process at different time instants. Mathematically this means
2[ ] [ ]xx x
n n =and
( ) 2
2
[ ] [ ]
j j n j n
xx xx x
n n
x
e n e n e
= =
= =
=
Exercise: Convince yourself that a white noise must have zero mean.
A commonly encountered random process is the white Gaussian noise.Remember, white refers to the spectral shape, and Gaussian refers to
the pdf.
Let the (stationary) random process [ ]x n be the input to a LTI system withan impulse response [ ]h n . Then the output of the system is
[ ] [ ] [ ]k
y n x k h n k
=
= .
We want to examine the statistical properties of the output process.
The mean value of [ ]y n is
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
62/812-62
[ ]
[ ]
( ) 0
[ ] [ ] [ ]
[ ] [ ]
[ ]
[ ]
ny
k
k
x
k
x
k
j
x
m E y n E x k h n k
E x k h n k
m h n k
m h n k
m H e
=
=
=
=
=
= =
=
=
=
=
,ym=where
( ) [ ]j j mm
H e h m e
=
=
is the frequency response of the system.
Basically, what the result is saying is that if the mean of the input isstationary, then the mean of the output is also stationary.
How about the autocorrelation function?By definition
[ ] *
*
, [ ] [ ]
[ ] [ ] [ ] [ ]
yy
k r
n m E y n y m
E h k x n k h r x m r
= =
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
63/812-63
* *
* *
* *
*
[ ] [ ] [ ] [ ]
[ ] [ ] [ ] [ ]
[ ] [ ] [ ] [ ]
[ ] [
k r
k r
k r
E h k x n k h r x m r
E h k h r x n k x m r
h k h r E x n k x m r
h k h
= =
= =
= =
=
=
=
=
[ ]]
[ ]
xx
k r
yy
r n m k r
n m
= =
+
=
Thus the autocorrelation function of the output process is also independent
of absolute time.
Since both the first and second order statistics of the output process areindependent of absolute time, the process is (wide-sense) stationary.
Let d n m= , then the autocorrelation function of the output process canbe written as
[ ] [ ]*
[ ] [ ]yy xxk r
d h k h r d k r
= == + Taking Fourier transform of both sides yields
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
64/812-64
( ) [ ]
[ ]
[ ]
[ ]
*
* ( )
*
[ ] [ ]
[ ] [ ]
[ ] [ ]
j j d
yy yy
d
j d
xx
d k r
j d k r j r j k
xx
k r d
j m
xx
m
e d e
h k h r d k r e
h k h r d k r e e e
h k h r m e
=
= = =
+
= = =
=
=
= +
= +
=
( )
( ) ( )
( ) ( ) ( )
( ) ( )
*
*
*
2
[ ] [ ]
[ ]
j r j k
k r
j j r j k
xx
k r
j j j k
xx
d
j j j
xx
j j
xx
e e
h k e h r e e
e H e h k e
e H e H e
e H e
= =
= =
=
=
=
=
=
According to an earlier result, if the input process [ ]x n has zero mean,then the output process [ ]y n will also have zero mean. This means thevariance of the output process equals the output power:
( )
( ) ( )
22
2
1[ ] [0]2
1
2
j
y yy yy
j j
xx
E y n e d
H e e d
= = =
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
65/812-65
If ( )jH e is a very narrow band filter centered at c = , then
( ) ( ) ( ){ }2
2 c c cj j j
y xx xxH e e e
+ ,
where 0 is the bandwidth of the filter.
For simiplicity assume a real [ ]x n 1. Then [ ] [ ]xx xxn n = (see an earlierexercise) and ( ) ( )j je e = . Consequently,
( ) ( )22 2 0c cj jy xxH e e
or simply
( ) 0cjxx e .
This property of ( )cjxx e is consistent with the fact that it represents a
power density.
Example: The input/output relationship of a LTI system is given by[ ] [ 1] [ ]y n ay n x n= +
where a is a positive constant less than 1. Determine the psd and the pdf of[ ]y n if [ ]x n is a white Gaussian process with zero mean and unit
variance.
1The proof is a bit lengthy for complex [ ]x n but the end result is the same.
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
66/812-66
Solution 1
- Since the input has zero mean, the output must also have zero mean.- Since the input is Gaussian, the output must also be Gaussian.- Taking the expectation of the square of both sides of [ ]y n yields
[ ]
2 2 2
2 2 2
2 2 2
2 2 2
[ ] ( [ 1] [ ])
[ 1] [ ] 2 [ 1] [ ]
[ 1] [ ] 2 [ 1] [ ]
2 [ 1
y
y x
E y n E ay n x n
E a y n x n ay n x n
a E y n E x n aE y n x n
a aE y n
= = + = + +
= + + = + + [ ] [ ]
2 2 2
] [ ]
y x
E x n
a = +or
22
2 2
1
1 1
xy
a a
= =
- The pdf of [ ]y n is simply
( )2
22
1exp
2y
yy
vp v
=
- The output [ ]y n can be expressed recursively as
( )
[ ] [ 1] [ ]
[ 2] [ 1] [ ]
y n ay n x n
a ay n x n x n
= +
= + +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
67/812-67
2
3 2
1
0
[ 2] [ 1] [ ]
[ 3] [ 2] [ 1] [ ]
[ ] [ ]m
m k
k
a y n ax n x n
a y n a x n ax n x n
a y n m a x n k
=
= + +
= + + +
= + Multiplying both sides by [ ]y n m , 0m > , and taking expectationyields
[ ]
[ ]
1
0
1
0
12
0
[ ] [ ] [ ] [ ] [ ]
[ ] [ ] [ ] [ ]
[ ] [ ] [ ]
mm k
k
mm k
k
mm k
k
E y n y n m E a y n m a x n k y n m
E a y n m y n m a x n k y n m
a E y n m a E x n k y n m
=
=
=
= +
= +
= +
2
[ ]
m
y
yy
a
m
=
=
Since the process [ ]y n is real, [ ] [ ]yy yym m = . So in general
2[ ]m
yy ym a =
- The psd is
( )
2
[ ]
j j myy yy
m
m j m
y
m
e m e
a e
=
=
=
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
68/812-68
( ) ( ) ( )( )( )
02 2 2
0
2 2 2
0 0
2 2
2
2 2 2
1 1
1 1 1 1
1 1
m mj m j m
y y y
m m
n j n m j m
y y y
n m
y y
yj j
j j j j
y y y
j
a e a e
a e a e
ae ae
ae ae ae ae
ae a
= =
= =
= +
= +
= +
+ =
( )
( )
( )
( )
2 2
2
2
1
1 2 cos
1
1 2 cos
j
y
e
a
a a
a a
=
+ =
+
Solution 2
-
The LTI has an impulse response of
[ ] [ ]nh n a u n=
where [ ]u n is the unit-step function.
- The corresponding frequency response is
( )0
1[ ]
1
j j n n j n
jn n
H e h n e a eae
= =
= = =
This means
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
69/812-69
( )
( ) ( )
2
2
2 2
2
1
1
1
1 cos( ) sin( )
1
1 2 cos( )
j
jH e
ae
a a
a a
=
=
+
=+
- Since [ ]x n is white and has a variance of unity, its psd is simply
( ) 2 1jxx xe = =
- The psd of [ ]y n is
( ) ( ) ( )2
2
1
1 2 cos( )
j j j
yy xxe e H e
a a
= =
+
2.7 Linear Predictiive Coding and the Autocorrelation Function
Consider the last example in Section 2.6 where we have a random process[ ]y n governed by the equation
[ ] [ 1] [ ]y n ay n x n= +
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
70/812-70
Here a is a positive constant less than 1 and [ ]x n is a white process withzero mean and unit variance. As shown in Section 2.6, the variance of
[ ]y n is related to the variance of [ ]x n according to the equation
2
22 2
11 1
xy
a a = =
Suppose [ ]y n is a sampled-speech signal and we want to send this speechsignal digitally through a communication channel. A simple way is to
quantize each sample into yB bits and feed the resultant bit stream to a
digital modulator to generate the transmitted signal
This encoding method, which is referred to as Pulse Code Modulation(PCM) in the literature, is not very efficient because [ ]y n has a relatively
large dynamic range, at least compared to [ ]x n .
Suppose the parameter parameter a (which will vary from speaker tospeaker) is known to both the encoder and the decoder. Then a more
efficient encoding scheme can be obtained by first subtracting
[ ] [ 1]py n ay n= from [ ]y n to obtain
[ ]y n bit
Quantizer
yB Digital
Modulator
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
71/812-71
[ ] [ ] [ ] [ ] [ 1]p
x n y n y n y n ay n= =
and then quantize [ ]x n using a xB -bit quantizer. The resultant bits,
together with the parameter a (also in quantized form), are then sent to the
receiver. Upon receiving these information, the decoder can approximate[ ]y n according to
[ ] [ 1] [ ]y n ay n x n= +% % ,
where [ ]x n% and a% are respectively the quantized versions of [ ]x n and a .
This encoding scheme, known as linear predictive coding(LPC), is more
efficient than PCM because [ ]x n has a smaller dynamic range than [ ]y n .
In the speech coding literature, [ ] [ 1]py n ay n= is referred to as the
predicted value of [ ]y n and [ ] [ ] [ ]px n y n y n= is called the residual orexcitation.
[ 1]ay n
+ [ ]x n[ ]x n
( )
Speech Model
1
1
j
jH e
ae
=
( )
Prediction Filter
j jF e ae =
[ ]y n
Eqv. response = 1j
ae
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
72/812-72
As shown earlier, the autocorrelation function of [ ]y n is 2[ ] myy xm a = .This means
[1] [0]yy yya =or
[1]=
[0]
yy
yy
a
So as long as [0] and [1]yy yy are known, then the parameter a will be
known to both the encoder and decoder. In practice, [0] and [1]yy yy can be
estimated according to
1
1 [ ] [ ] [ ].N d
yy
k
d y n y n d N
=
= +
This suggests that in order to implement the LPC encoder, the signal [ ]y n
must first be analyzed to obtain its autocorrelation function. Once the
autocorrelation function is estimated, then the information will be used to
obtain the prediction filter at the encoder and the synthesizing filter at the
decoder.
[ 1]ay n
[ ]y n[ ]x n%[ ]x n
[ ]Q
Quantizer1az
Synthesizing filter
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
73/812-73
As the parameter 1a , the variance of [ ]y n is going to much greater thanthat of [ ]x n . Consequently LPC will be much more efficient than PCM.
As 0a , the variance of [ ]y n approaches that of [ ]x n and LPC provideslittle improvement to the encoding efficiency.
Since 0a implies little correlation between samples in [ ]y n while1a corresponds to high correlation, we can conclude that LPC achieves
a higher encoding efficiency because it removes the redundancy in [ ]y n
through linear prediction.
The expression [ ] [ 1] [ ]y n ay n x n= + represents a first-order model. Ingeneral, speech signals can be modeled accurately using aN-th order model
(typical value ofNis 10):
1
[ ] [ ] [ ]N
k
k
y n a y n k x n=
= +
where [ ]x n is a white noise. The model also represents a system described
by a Linear Constant Coefficient Difference Equation with [ ]x n being the
excitation and [ ]y n being the output.
For theN-th order model, the prediction filter in the encoder computes
1
[ ] [ ]N
p k
k
y n a y n k=
=
and subtract it from [ ]y n to obtain the residual (excitation)
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
74/812-74
1
[ ] [ ] [ ] [ ] [ ]N
p k
k
x n y n y n y n a y n k=
= =
The residual, as well as the filter coefficients,
1
2
N
a
a
a
=
AM
are then quantized and transmitted to the receiver. Upon receiving these
information, the decoder synthesizes the original speech signal according to
1
[ ] [ ] [ ]N
k
k
y n a y n k x n=
= + % % ,
where [ ]x n% and ka% are respectively the quantized versions of [ ]x n and
ka .
How to determine linear predictorA? If we multiply both sides of [ ]y n by[ 1]y n and taking average, we obtain
[ ]
[ ] [ ]
1
1
1
[ ] [ 1] [ ] [ 1] [ ] [ 1]
[ ] [ 1] [ ] [ 1]
[ 1]
N
kk
N
k
k
N
k yy
k
E y n y n E a y n k y n x n y n
a E y n k y n E x n y n
a k
=
=
=
= +
= +
=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
75/812-75
or simply
1
[1] [ 1]N
yy k yy
k
a k =
=
Similarly if we multiply both sides of [ ]y n by [ 1]y n and takingaverage, we obtain
[ ]
[ ] [ ]
1
1
1
[ ] [ 2] [ ] [ 2] [ ] [ 2]
[ ] [ 2] [ ] [ 2]
[ 2]
N
k
k
N
kk
N
k yy
k
E y n y n E a y n k y n x n y n
a E y n k y n E x n y n
a k
=
=
=
= +
= +
=
or
1
[2] [ 2]N
yy k yy
k
a k =
=
It is evident that in general
1
1
[ ] [ ];
; 1,2,...,
N
yy k yy
k
N
k yy
k
m a k m
a k m m N
=
=
=
= =
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
76/812-76
These equations can be written in matrix form as2
1
2
3
1
[0] [1] [2] [ 2] [ 1] [1]
[1] [0] [1] [ 3] [ 2] [2]
[2] [1] [0] [ 4] [ 3] [3
[ 2] [ 3] [ 4] [0] [1]
[ 1] [ 2] [ 3] [1] [0]
N
N
aN N
aN N
aN N
aN N N
aN N N
=
L
L
LMM M M M M M
L
L
]
[ 1]
[ ]
N
N
M
or
N N N=U A V ,
where
[0] [1] [2] [ 2] [ 1]
[1] [0] [1] [ 3] [ 2]
[2] [1] [0] [ 4] [ 3]
[ 2] [ 3] [ 4] [0] [1]
[ 1] [ 2] [ 3] [1] [0]
N
N N
N N
N N
N N N
N N N
=
U
L
L
L
M M M M M M
L
L
,11
,22
,33
, 11
,
N
N
N
N
N NN
N NN
aa
aa
aa
aa
aa
=
=
AMM and
[1]
[2]
[3]
[ 1]
[ ]
N
N
N
=
VM
2For convenience, we drop the subscript yy in [ ]yy m .
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
77/812-77
So the coefficients for the prediction filter can be obtained by solving
1
N N N
=A U V ,
provided that the autocorrelation function [ ]m is known.
A brute-force computation of NA results in a complexity in the order of( )3O N . However, since the matrix is Toeplitz, i.e. all elements along any
diagonal are identical, a more efficient algorithm called the Levison and
Durbin (LD) algorithm can be used.
The LD algorithm only has a complexity of only ( )2O N .
The LD algorithm is a order-recursive algorithm, i.e. the m -th orderpredictor mA can be obtained from the ( 1)m -th predictor 1mA .
Specifically, we rewrite mA
as
,1
,2 11
,
0
m
m mm
m
m
m m
a
a
k
a
= = +
dAA
M (1)
where the vector 1md and the scaler mk are quantities to be determined.
The covariance matrix mU itself can be written in terms of 1mU and 1mVas:
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
78/812-78
( )
1 1
1[0]
r
m m
tm r
m
=
U VU
V(2)
where
1
[ 1]
[ 2]
[ 3]
[2]
[1]
r
m
m
m
m
=
VM (3)
is the correlation vector 1mV arranged in reverse order, and [ ]t
stands for
the transpose of a matrix.
Substituting (1)-(3) into the equation m m m=U A V implies
( )
1 1 11 1
10 [ ][0]
r
m m mm m
tr
mmk m
+ =
U V dA V
V(4)
One of the equation we can obtain from (4) is1 1 1 1 1 1
1 1 1 1
r
m m m m m m m
r
m m m m m
k
k
= + +
= + +
V U A U d V
V U d V
or
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
79/812-79
1
1 1 1
r
m m m mk
= d U V (5)
Since 1 1 1m m m =V U A ,
1 1
1 1
1 1
1 1
rm m
m m
r
m m
r
m m
==
=
=
V PV
PU A
PU PA
W A
, (6)
where
0 0 1
0 1 0
1 0 0
=
P
L
L
M M M M
L
is the permutation matrix representing vector reversal, and
1 1m m =W PU P
Because of the property of 1mU ,
1 1m m =W U (7)
Combining (5)-(7) implies
1
1 1 1
1
1 1 1
1
1 1 1
1
,
r
m m m m
r
m m m m
r
m m m m
r
m m
k
k
k
k
= =
=
=
d U V
U W A
U U A
A
(8)
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
80/812-80
i.e. the vector 1md is the vector containing the coefficients of the ( 1)m -th
predictor arranged in reverse order and scaled by the term mk .
The second equation we can derive from (4) is( ) ( )
( ) ( )
1 1 1 1
1 1 1 1
[ ] [0]
[0]
t tr r
m m m m m
t tr r r
m m m m m m
m k
k k
= + +
= +
V A V d
V A V A
or
( )
( )
( )1 1 1 1
11 1
[ ] [ ]
[0]
t tr r
m m m m
m tr r
mm m
m mk
= =
V A V A
V A(9)
where
( )
( )
1 1 1
1 1
[0]
[0]
tr r
m m m
t
m m
=
=
V A
V A (10)
In summary,1.At the end of the (m-1)-th iteration, the available information to the LPC
encoder are 1m and the (m-1)-th order predictor 1mA .
2.In the m-th iteration, the encoder computes mk according to (9) and setthe highest order term in the m-th order predictor to (see Eqn. (1))
,m m ma k=
-
7/29/2019 Chapter 2 - Discrete Time Signals and Systems
81/81
3.The other predictor coefficients are computed according to (1) and (8) as, 1, 1,
1, 1, ; 1,2,..., 1
m k m k m k
m k m m m k
a a d
a k a k m
= +
= = , (11)
where 1,m kd is the k-th component of the vector 1md in (8)
4.Update the term m and increase m by 1.
Exercise: Show that the term m can also be calculated recursively.
Equations (9)-(11) each has a computational complexity of 1m multiplication. Summing over all m from 1 toNyields a complexity of
( ) 21
3 1 3 ( 1) / 2N
m
m N N N =
=
Note that LPC-based speech codec can produce communication qualityspeech at a rate as low as 2.4 kbps (though 4.8-9.6 kbps are more typical).
Thi i h l th th 64 kb i d i PCM b d d