stationary processes - university of california, san...
TRANSCRIPT
LECTURE 9
Stationary Processes
9.1 STRICT-SENSE STATIONARITY
Stationarity refers to time invariance of some, or all, of the statistics of a random pro-
cess, such as mean, autocorrelation, and n-th order distribution. We define two types of
stationarity: strict-sense stationarity (SSS) and wide-sense stationarity (WSS).
A random process {X(t)} is said to be SSS (or just stationary) if all its finite-order
distributions are time invariant, i.e., the joint cdfs (pdfs, pmfs) of
X(t1), X(t2), . . . , X(tk)and
X(t1 + τ), X(t2 + τ), . . . , X(tk + τ)are the same for every k, every t1 , t2 , . . . , tk , and every time shi� τ. So for a SSS process,
the first-order distribution is independent of t , and the second-order distribution— the
distribution of any two samples X(t1) and X(t2)—depends only on τ = t2 − t1. To see this,
note that from the definition of stationarity, for any t , the joint distribution of X(t1) andX(t2) is the same as the joint distribution of X(t) = X(t1 + (t − t1)) and X(t2 + (t − t1)) =X(t + (t2 − t1)).Example 9.1. IID processes are SSS.
Example 9.2. �e random walk is not SSS. In fact, no independent increment process is
SSS.
Example 9.3. �eGauss–Markov process defined in Section ؏.؏.؏ is not SSS. However, if
we set X1 to the steady-state distribution of Xn, it becomes SSS (see Problem ؏.؏).
9.2 WIDE-SENSE STATIONARIY
A random process {X(t)} is said to be wide-sense stationary (WSS) if its mean and auto-
correlation functions are time invariant, i.e.,
Ȃ E[X(t)] = ̀ is independent of t and
Ȃ RX(t1 , t2) is a function only of the time difference t2 − t1.
2 Stationary Processes
As a technical condition, we also assume that
E[X(t)2] < ∞.
Since RX(t1 , t2) = RX(t2 , t1), for any wide-sense stationary process {X(t)}, the autocor-relation function RX(t1 , t2) is a function only of |t2 − t1|. Clearly SSS implies WSS. �e
converse is not necessarily true.
Example ؏.؏. Let
X(t) =.6666>6666F
+ sin t w.p. 1/4,− sin t w.p. 1/4,+ cos t w.p. 1/4,− cos t w.p. 1/4.
Note that E[X(t)] = 0 and RX(t1 , t2) = (1/2) cos(t2 − t1); thus X(t) is WSS. But X(0) andX(π/4) have different pmfs (i.e., the first-order pmf is not time-invariant) and the process
is not SSS.
For a Gaussian random process, WSS implies SSS, since every finite-order distribu-
tion of the process is completely specified by its mean and autocorrelation functions. �e
randomwalk is notWSS, since RX(n1 , n2) = min{n1 , n2} is not time-invariant. In fact, no
independent increment process can be WSS.
9.3 AUTOCORRELATION FUNCTION OF WSS PROCESSES
Let {X(t)} be a WSS process. We relabel RX(t1 , t2) as RX(τ), where τ = t1 − t2. �e auto-
correlation function RX(τ) satisfies the following properties.؏. RX(τ) is even, i.e., RX(τ) = RX(−τ) for every τ.؏. |RX(τ)| ≤ RX(0) = E[X2(t)], the “average power” of X(t).؏. If RX(T) = RX(0) for some T ̀= 0, thenRX(τ) is periodic with period T and so is X(t)
(with probability 1).
Example ؏.؏. Let X(t) = α cos(ωt +Θ) be periodic with random phase. �en
RX(τ) = α2
2cosωτ ,
which is also periodic.
�e above properties of RX(τ) are necessary but not sufficient for a function to qual-
ify as an autocorrelation function for a WSS process. �e necessary and sufficient con-
dition for a function R(τ) to be an autocorrelation function for a WSS process is that it
9.3 Autocorrelation Function of WSS Processes 3
be even and nonnegative definite, that is, for any n, any t1, t2 , . . . , tn and any real vector
a = (a1 , . . . , an),nHi=1
nHj=1
aia jR(ti − t j) ≥ 0.
To see why this is necessary, recall that the correlation matrix for a random vector must be
nonnegative definite, so if we take a set of n samples from theWSS random process, their
correlation matrix must be nonnegative definite. �e condition is sufficient since such
an R(τ) can specify a zero-mean stationary Gaussian random process. �e nonnegative
definite conditionmay be difficult to verify directly. It turns out, however, to be equivalent
to the condition that the Fourier transform of RX(τ), which is called the power spectral
density SX( f ), is nonnegative for all frequencies f .
Example ؏.؏. Consider the following functions.
(a) (b)
e−ατ
τ
e−α|τ|
τ
(c) (d)
τ
sinc τ
τ
(e) (f)
τ
T2
T
1
−1
2−|n|
n−4−3−2−1 1 2 3 4
4 Stationary Processes
(g) (h)
τ
1
−T Tτ
1
Here, the functions in (a), (c), and (g) are not autocorrelation functions, and the other
functions are autocorrelation functions of some WSS processes.
If RX(τ) drops quickly with τ, this means that samples become uncorrelated quickly
as we increase τ. Conversely, if RX (τ) drops slowly with τ, samples are highly corre-
lated. Figure ؏.؏ illustrates autocorrelation functions in these two cases. Hence, RX(τ) isa measure of the rate of change of X(t) with time t . It turns out that this is not just an
intuitive interpretation—as will be proved in Section ؏.؏, the Fourier transform of RX(τ)(the power spectral density) is in fact the average power density of X(t)) over frequency.
RX1(τ)
τ
RX2(τ)
τ
(a) (b)
Figure ؏.؏. Autocorrelation functions when (a) correlation is low and (b) correlationis high.
9.4 POWER SPECTRAL DENSITY
�e power spectral density (psd) of aWSS random process {X(t)} is the Fourier transformof RX(τ):
SX( f ) = F [RX(τ)] = X∞
−∞
RX(τ)e−i2πτ f dτ.
9.4 Power Spectral Density 5
For a discrete-time process {Xn}, the power spectral density is the discrete-time Fourier
transform of the sequence RX(n):SX( f ) = ∞H
n=−∞
RX(n)e−i2πn f , | f | < 1
2.
By taking the inverse Fourier transform, RX(τ) (or RX(n)) can be recovered from SX( f )as
RX(τ) = X∞
−∞
SX( f )ei2πτ f d f ,RX(n) = X 1
2
−1
2
SX( f )ei2πn f d f .�e power spectral density SX( f ) satisfies the following properties.
؏. SX( f ) is real and even.
؏. ∫∞
−∞SX( f )d f = RX(0) = E[X2(t)], that is, the area under SX( f ) is the average power.
؏. SX( f ) is the average power density, i.e., the average power of X(t) in the frequency
band [ f1 , f2] isX− f1
− f2
SX( f ) d f + X f2
f1
SX( f ) d f = 2X f2
f1
SX( f ) d f .؏. SX( f ) ≥ 0.
�e first two properties follow from the definition of power spectral density as the Fourier
transform of a real and even function RX(τ). �e third and fourth properties will be
proved later in Section ؏.؏. In general, a function S( f ) is a psd if and only if it is real,
even, nonnegative, and ∫∞
−∞S( f ) d f < ∞.
Example ؏.؏. We consider a few pairs of autocorrelation functions and power spectral
densities.
(a)
RX(τ) = e−2α|τ|
τ
SX( f ) = α
α2+(π f )2
f
6 Stationary Processes
(b)
RX (τ) = α2
2cos(ωτ)
τ
SX( f )α2
4
α2
4
f− ω2π
ω
2π
(c)
RX(n) = 2−|n|
n−4−3−2−1 1 2 3 4
SX( f ) = 3
5 − 4 cos 2π f
f− 1
2
1
2
Example 9.8 (Discrete-timewhitenoise process). Let X1 , X2 , . . . be zeromean, uncorre-
lated, with common variance N , for example, i.i.d. N(0,N). �e autocorrelation function
and the power spectral density are plotted in Figure ؏.؏.
RX(n) = ®N n = 0
0 otherwise
N
n
SX( f )N
f− 1
2+ 1
2
Figure؏.؏. Autocorrelation function and power spectral density of the discrete-timewhite noise process.
Example 9.9 (Band-limited white noise process). Let {X(t)} be a WSS zero-mean pro-
cess with autocorrelation function and power spectral density plotted in Figure ؏.؏. Note
that for any t , the samples X�t ± n
2B� for n = 0, 1, 2, . . . are uncorrelated.
9.4 Power Spectral Density 7
SX( f )N
2
f−B B
RX(τ) = NBsinc 2Bτ
1
2B2
2B
τ
Figure ؏.؏. Autocorrelation function and power spectral density of the band-limitedwhite noise process.
Example 9.10 (White noise process). If we letB Ă ∞ in the previous example, we obtain
a white noise process, which has
SX( f ) = N
2for all f ,
RX(τ) = N
2δ(τ).
If, in addition, {X(t)} isGaussian, thenweobtain the famouswhiteGaussian noise (WGN)
process. A sample path of the white noise process is depicted in Figure ؏.؏. For a white
noise process, all samples are uncorrelated. �e process is not physically realizable, since
it has infinite power. However, it plays a similar role in random processes to the role of
a point mass in physics and delta function in EE. �ermal noise and shot noise are well
modeled aswhiteGaussian noise, since they have very flat psd over verywide band (GHz).
Figure ؏.؏. A sample function of the white noise process.
8 Stationary Processes
9.5 WSS PROCESSES AND LTI SYSTEMS
Consider a linear time invariant (LTI) system with impulse response h(t) and transfer
functionH( f ) = F [h(t)]. Suppose that the input to the system is aWSSprocess {X(t) : t ∈Ă}, as depicted in Figure ؏.؏. We wish to characterize its output
Y(t) = X(t) ∗ h(t) = X∞
−∞
X(τ)h(t − τ) dτ.
X(t) Y(t)h(t)
Figure ؏.؏. An LTI system driven by a WSS process input.
Assuming that the system is in steady state, it turns out (not surprisingly) that the
output is also a WSS process. In fact, X(t) and Y(t) are jointly WSS, namely,
Ȃ X(t) and Y(t) are WSS, and
Ȃ their crosscorrelation function
RXY (t1 , t2) = E[X(t1)Y(t2)]is time invariant, RXY (t1 , t2) is a function of τ = t1 − t2.
Henceforth, we relabel RXY (t1 , t2) as RXY (τ), where τ = t1 − t2. Note that unlike RX(τ),RXY (τ) is not necessarily even, but satisfies
RXY (τ) = RYX(−τ).Example ؏.؏؏. Let Θ ∼ Unif[0, 2π]. Consider two processes
X(t) = α cos(ωt + Θ),Y(t) = α sin(ωt +Θ).
�ese processes are jointly WSS, since each is WSS (in fact SSS) and
RXY (t1 , t2) = E[α2 cos(ωt1 +Θ) sin(ωt2 + Θ)]= α2
4πX2π
0
sin(ω(t1 + t2) + 2θ) − sin(ω(t1 − t2)) dθ= −α2
2sin(ω(t1 − t2)),
which is a function only of t1 − t2.
9.5 WSS Processes and LTI Systems 9
We define the cross power spectral density for jointly WSS processes {X(t)} and {Y(t)}as
SXY ( f ) = F [RXY (τ)].Example ؏.؏؏. Let Y(t) = X(t) + Z(t), where {X(t)} and {Z(t)} are zero-mean uncorre-
lated WSS processes with power spectral densities SX( f ) and SZ( f ). �en, {X(t)} and{Y(t)} are jointly WSS. First, we show that {Y(t)} is WSS, since it is zero mean and
RY (t1 , t2) = E[(X(t1) + Z(t1))(X(t2) + Z(t2))]= E[X(t1)X(t2)] + E[Z(t1)Z(t2)]= RX(τ) + RZ(τ),where (a) follows since {X(t)} and {X(t)} and {Z(t)} are zero mean and uncorrelated.
Taking the Fourier transform of both sides,
SY ( f ) = SX( f ) + SZ( f ).To show that Y(t) and X(t) are jointly WSS, consider
RXY (t1 , t2) = E[X(t1)(X(t2) + Z(t2))]= E[X(t1)X(t2)] + E[X(t1)Z(t2)]= RX(t1 , t2),which is time invariant. Relabeling RX(t1 , t2) = RX(τ) and taking the Fourier transform,
SXY ( f ) = SX( f ).Let {X(t) : t ∈ Ă} be a WSS process input to a LTI system with impulse response h(t)
and transfer functionH( f ). If the system is stable, i.e.,
!!!!!!!! X∞
−∞
h(t) dt !!!!!!!! = |H(0)| < ∞ ,
then the input X(t) and the output Y(t) are jointly WSS with the following properties:
؏. E[Y(t)] = H(0) E[X(t)].؏. RYX (τ) = h(τ) ∗ RX(τ).؏. RY (τ) = h(τ) ∗ RX (τ) ∗ h(−τ).؏. SYX( f ) = H( f )SX( f ).؏. SY ( f ) = |H( f )|2SX( f ).�ese properties are summarized in Figure ؏.؏.
10 Stationary Processes
E[X(t)] E[Y(t)]H(0)
RX(τ) RY (τ)h(τ) h(−τ)RYX (τ)
PSfrag
SX( f ) SY ( f )H( f ) H(− f )SYX( f )
Figure ؏.؏. Properties of the output of an LTI system driven by aWSS process input.
To show the first property, consider
E[Y(t)] = E�X∞
−∞
X(τ)h(t − τ) dτ�= X∞
−∞
E[X(τ)] h(t − τ) dτ= E[X(t)] X∞
−∞
h(t − τ) dτ= E[X(t)]H(0).
For the second property,
RYX(τ) = E[Y(t + τ)X(t)]= E�X∞
−∞
h(s)X(t + τ − s)X(t) ds�= X∞
−∞
h(s)RX (τ − s) ds= h(τ) ∗ RX(τ).
For the third property, consider
RY (τ) = E[Y(t + τ)Y(t)]= E�Y(t + τ) X∞
−∞
h(s)X(t − s) ds�= X∞
−∞
h(s)RYX (τ + s) ds= RYX(τ) ∗ h(−τ).
In the above, we have shown that E[Y(t + τ)X(t)] and E[Y(t + τ)Y(t)] are functions of τ(not of t), confirming that {X(t)} and {Y(t)} are jointly WSS. �e fourth and fi�h prop-
erties follow by taking the Fourier transforms of RYX (τ) and RY (τ), respectively.
9.5 WSS Processes and LTI Systems 11
We can use the above properties to prove that SX( f ) is indeed the power spectral
density of X(t), that is, SX( f )d f is the average power contained in the frequency band[ f , f + d f ]. Consider an ideal band-pass filter shown in Figure ؏.؏ with output
Y(t) = h(t) ∗ X(t)driven by a WSS process {X(t)}. �en the average power of X(t) in the band [ f1 , f2] is
E[Y2(t)] = X∞
−∞
SY ( f ) d f= X∞
−∞
|H( f )|2SX( f ) d f= X− f1
− f2
SX( f ) d f + X f2
f1
SX( f ) d f= 2X f2
f1
SX( f ) d f .�is argument also shows that SX( f ) ≥ 0 for all f .
H( f )1
f− f2 − f1 f1 f2
Figure ؏.؏. �e transfer function of an ideal band-pass filter over [ f1 , f2].
Example ؏.؏؏ (kT/C noise). �e noise in a resistor R (in ohms) due to thermal noise is
modeled as aWGNvoltage sourceV (t) in series withR; see Figure ؏.؏. �epower spectral
density ofV (t) is SV ( f ) = 2kTRV2/Hz for all f , where k is the Boltzmann constant andT
is the temperature in degrees K.We find the average output noise power for an RC circuit
shown in Figure ؏.؏. First, note that the transfer function for the circuit is
H( f ) = 1
1 + i2π f RC,
which implies that
|H( f )|2 = 1
1 + (2π f RC)2 .
12 Stationary Processes
V (t)R
Figure ؏.؏. A model of the thermal noise in a resistor.
R
CV (t) V0(t)
Figure ؏.؏. An RC circuit driven by the thermal noise.
Now we write the output power spectral density in terms of the input power spectral
density as
SVo( f ) = SV ( f )|H( f )|2 = 2kTR
1
1 + (2π f RC)2 .
�us the average output power is
E[V 2
o (t)] = X∞
−∞
SVo( f )d f
= 2kTR
2πRCX∞
−∞
1
1 + (2π f RC)2 d(2π f RC)= kT
πCX∞
−∞
1
1 + x2dx
= kT
πCarctan x
!!!!!!!+∞
−∞
= kT
C,
which is independent of R.
9.6 Linear Estimation of Random Processes 13
9.6 LINEAR ESTIMATION OF RANDOM PROCESSES
Let {X(t)} and {Y(t)} be zero mean jointly WSS processes with known autocorrelation
and crosscorrelation functionsRX(τ),RY (τ), andRXY (τ). We observe the randomprocess
Y(s) for t − a ≤ s ≤ t + b (−a ≤ b) and wish to find the linearMMSE estimate of the signal
X(t), i.e., the estimate of the form
X̀(t) = Xa
−bh(s)Y(t − s) ds
that minimizes the mean square error
E[(X(t) − X̀(t))2].By the orthogonality principle, the linear MMSE estimate must satisfy
(X(t) − X̀(t)) ⊥ Y(t − s) , −b ≤ s ≤ a,
or equivalently,
E[(X(t) − X̀(t))Y(t − s)] = 0 , −b ≤ s ≤ a.
�us, for −b ≤ s ≤ a, the optimal estimation filter h(s)must satisfy
RXY (τ) = E[X(t)Y(t − τ)]= E[X̀(t)Y(t − τ)]= E�Xa
−bh(s)Y(t − s)Y(t − τ) ds�
= Xa
−bh(s)RY (τ − s) ds
for −b ≤ s ≤ a. To find h(s), we need to solve an infinite set of integral equations. Solving
these equations analytically is not possible in general. However, it can be done for two
important special cases:
Ȃ Infinite smoothing: a, b Ă ∞.
Ȃ Filtering: a Ă ∞ and b = 0
We discuss only the first case. �e second case leads to theWiener–Hopf equations from
which the famousWiener filter is derived.
When a, b Ă ∞, the integral equations for the linear MMSE estimate become
RXY (τ) = X∞
−∞
h(s)RY (τ − s) ds , −∞ < τ < +∞.
In other words,
RXY (τ) = h(τ) ∗ RY (τ).
14 Stationary Processes
By taking Fourier transforms, we have
SXY ( f ) = H( f )SY ( f ),which implies that the optimal infinite smoothing filter is
H( f ) = SXY ( f )SY ( f ) .
Observe the similarity between the optimal filter and the LMMSE estimate
X̀ = ΣTYXΣ
−1Y(Y − E[Y]) + E[X]
for the vector case discussed in Section ؏.؏.
�e LMMSE filter achieves the MSE
E[(X(t) − X̀(t))2] = E[(X(t) − X̀(t))X(t)] − E[(X(t) − X̀(t))X̀(t)](a)= E[(X(t) − X̀(t))X(t)]= E[(X(t)2] − E[X̀(t)X(t)],
where (a) follows by orthogonality. To evaluate the second term, consider
RXX̀(τ) = E[X(t + τ)X̀(t)]= E�X(t + τ) X∞
−∞
h(s)Y(t − s) ds�= X∞
−∞
h(s)RXY (τ + s) ds= RXY (τ) ∗ h(−τ),
which implies that
SXX̀( f ) = H(− f )SXY ( f ).�erefore,
E[X̀(t)X(t)] = RXX̀(0)= X∞
−∞
SXX̀( f ) d f= X∞
−∞
H(− f )SXY ( f ) d f= X∞
−∞
|SXY ( f )|2SY ( f ) d f
9.6 Linear Estimation of Random Processes 15
and the minimum MSE is
E[(X(t) − X̀(t))2] = E[(X(t)2] − E[X̀(t)X(t)]= X∞
−∞
SX( f ) d f − X∞
−∞
|SXY ( f )|2SY ( f ) d f
= X∞
−∞
�SX( f ) − |SXY ( f )|2SY ( f ) � d f .
�is MSE can be compared to
σ2X − ΣTYXΣ
−1YΣYX
for the vector case.
Example ؏.؏؏ (Additivewhite noise channel). Let {X(t)} and {Z(t)} be zero-mean un-
correlated WSS processes with
SX( f ) = ®P/2 | f | ≤ B ,
0 otherwise,
and
SZ( f ) = N
2for all f ,
as shown in Figure ؏.؏؏. In other words, the signal {X(t)} is band-limited white noise and
the noise {Z(t)} is white. We find the optimal infinite smoothing filter for estimating X(t)given
Y(τ) = X(τ) + Z(τ) , −∞ < τ < +∞.
�e transfer function of the optimal filter is
H( f ) = SXY ( f )SY ( f )
= SX( f )SX( f ) + SZ( f )
= ®P/(P + N) | f | ≤ B ,
0 otherwise,
as shown in Figure ؏.؏؏. �e MSE of the optimal filter is
X∞
−∞
SX( f ) − |SXY ( f )|2SY ( f ) d f = XB
−B
P
2− (P/2)2P/2 + N/2 d f
= PB − P2/4(P + N)/2 2B
= NPB
N + P.
16 Stationary Processes
SX( f )
SZ( f )
P
2
N2
f
f−B B
Figure ؏.؏؏. Power spectral densities SX ( f ) and SZ( f ).H( f )
PP+N
f−B B
Figure ؏.؏؏. �e transfer function H( f ) of the optimal infinite smoothing filter.
PROBLEMS
؏.؏. Moving average process. Let Z0 , Z1 , Z2 , . . . be i.i.d. ∼ N (0, 1).(a) Let Xn = 1
2Zn−1 + Zn for n ≥ 1. Find the mean and autocorrelation function of
Xn.
(b) Is {Xn} wide-sense stationary?(c) Is {Xn} Gaussian?(d) Is {Xn} strict-sense stationary?(e) Find E(X3|X1 , X2).(f) Find E(X3|X2).(g) Is {Xn}Markov?
(h) Is {Xn} independent increment?
(i) Let Yn = Zn−1 + Zn for n ≥ 1. Find the mean and autocorrelation functions of{Yn}.
Problems 17
(j) Is {Yn} wide-sense stationary?(k) Is {Yn} Gaussian?(l) Is {Yn} strict-sense stationary?(m)Find E(Y3|Y1 ,Y2).(n) Find E(Y3|Y2).(o) Is {Yn}Markov?
(p) Is {Yn} independent increment?
؏.؏. Random binary modulation. Let {Xn} be a zero-mean wide-sense stationary ran-
dom process with autocorrelation function RX(n), and Z1 , Z2 , . . . be i.i.d. Bern(p)random variables, i.e.,
Zi = ®1, with probability p,
0, with probability 1 − p.
Let
Yn = Xn ⋅ Zn , n = 1, 2, . . . .
(a) Find the mean and the autocorrelation function of {Yn} in terms of RX(n) andp.
(b) Is {Yn} jointly wide-sense stationary with {Xn}?؏.؏. Random binary waveform (؏؏ pts). Let {N(t)}∞t=0 be a Poisson process with rate λ,
and Z be independent of {N(t)} with P(Z = 1) = P(Z = −1) = 1/2. DefineX(t) = Z ⋅ (−1)N(t) , t ≥ 0.
(a) Find the mean and autocorrelation function of {X(t)}∞t=0.(b) Is {X(t)}∞t=0 wide-sense stationary?(c) Find the first-order pmf pX(t)(x) = P(X(t) = x).(d) Find the second-order pmf pX(t1),X(t2)(x1 , x2) = P(X(t1) = x1 , X(t2) = x2).(Hint: ∑k even x
k/k! = (ex + e−x)/2 and∑k odd xk/k! = (ex − e−x)/2.)
؏.؏. QAM random process. Consider the random process
X(t) = Z1 cosωt + Z2 sinωt , −∞ < t < ∞ ,
where Z1 and Z2 are i.i.d. discrete random variables such that pZi(+1) = pZi
(−1) =1
2.
(a) Is X(t) wide-sense stationary? Justify your answer.(b) Is X(t) strict-sense stationary? Justify your answer.
18 Stationary Processes
؏.؏. Mixture of twoWSS processes. Let X(t) andY(t) be two zero-meanWSS processes
with autocorrelation functions RX (τ) and RY (τ), respectively. Define the processZ(t) = ¦ X(t), with probability 1
2
Y(t), with probability 1
2.
Find the mean and autocorrelation functions for Z(t). Is Z(t) a WSS process?
؏.؏. Stationary Gauss–Markov process. Let
X0 ∼ N(0, a)Xn = 1
2Xn−1 + Zn , n ≥ 1 ,
where Z1 , Z2 , Z3 , . . . are i.i.d. N(0, 1) independent of X0.
(a) Find a such that Xn is stationary. Find themean and autocorrelation functions
of Xn.
(b) (Difficult.) Consider the sample mean Sn = 1
n∑n
i=1 Xi , n ≥ 1. Show that Snconverges to the process mean in probability even though the sequence Xn is
not i.i.d. (A stationary process for which the sample mean converges to the
process mean is called mean ergodic.)
؏.؏. AM modulation. Consider the AM modulated random process
X(t) = A(t) cos(2πt + Θ) ,where the amplitude A(t) is a zero-mean WSS process with autocorrelation func-
tion RA(τ) = e−1
2|τ|, the phase Θ is a Unif[0, 2π) random variable, and A(t) and Θ
are independent. Is X(t) a WSS process?
؏.؏. Random-delay mixture.
Let {X(t)}, −∞ < t < ∞, be a zero-mean wide-sense stationary process with au-
tocorrelation function RX(τ) = e−|τ|. Let
Y(t) = X(t −U ),whereU is a random delay, independent of {X(t)}. Suppose thatU ∼ Bern(1/2),that is,
Y(t) = ®X(t) with probability 1/2,X(t − 1) with probabiltiy 1/2.
(a) Find the mean and autocorrelation functions of {Y(t)}.(b) Is {Y(t)} wide-sense stationary? Justify your answer.(c) Find the average power E(Y(t)2) of {Y(t)}.(d) Now suppose that U ∼ Exp(1), i.e., fU (u) = e−u, u ≥ 0. Find the mean and
autocorrelation function of {Y(t)}.
Problems 19
؏.؏. Linear estimation. Let X(t) be a zero-mean WSS process with autocorrelation
function RX(τ) = e−|τ|. Let Y(t) be another zero-mean process which is jointly
WSS with X(t) with cross correlation function RXY (τ) = ∫1
0RX(s − τ)ds.
(a) Find the linearMMSE estimate ofY(t) given X(t). Leave your answer in terms
of e.
(b) Find the linear MMSE estimate of Y(t) given X(t) and X(t + 1).؏.؏؏. LTI system with WSS process input. Let Y(t) = h(t) ∗ X(t) and Z(t) = X(t) − Y(t)
as shown in the Figure ؏.؏؏.
(a) Find SZ( f ).(b) Find E(Z2(t)).Your answers should be in termsof SX ( f ) and the transfer functionH( f ) = F [h(t)].
X(t) Y(t)
+ -Z(t)
h(t)
Figure ؏.؏؏. LTI system.
؏.؏؏. Echo filtering. A signal X(t) and its echo arrive at the receiver as Y(t) = X(t) +X(t − ̀) + Z(t). Here the signal X(t) is a zero-mean WSS process with power
spectral density SX( f ) and the noise Z(t) is a zero-meanWSS with power spectral
density SZ( f ) = N0/2, uncorrelated with X(t).(a) Find SY ( f ) in terms of SX( f ), ̀, and N0.
(b) Find the best linear filter to estimate X(t) from {Y(s)}−∞<s<∞ .
؏.؏؏. Discrete-time LTI system with white noise input. Let {Xn : −∞ < n < ∞} be a
discrete-time white noise process, i.e., E(Xn) = 0, −∞ < n < ∞, and
RX(n) = ®1 n = 0,
0 otherwise.
�e process is filtered using a linear time invariant system with impulse response
h(n) = .66>66Fα n = 0,
β n = 1,
0 otherwise.
20 Stationary Processes
Find α and β such that the output process Yn has
RY (n) = .66>66F2 n = 0,
1 |n| = 1,
0 otherwise.
؏.؏؏. Finding time of flight. Finding the distance to an object is o�en done by sending
a signal and measuring the time of flight, the time it takes for the signal to return
(assuming speed of signal, e.g., light, is known). Let X(t) be the signal sent andY(t) = X(t − δ) + Z(t) be the signal received, where δ is the unknown time of
flight. Assume that X(t) and Z(t) (the sensor noise) are uncorrelated zero mean
WSS processes. �e estimated crosscorrelation function of Y(t) and X(t), RYX (t)is shown in Figure ؏.؏؏. Find the time of flight δ.
3
2 5 8t
RYX(t)
Figure ؏.؏؏. Crosscorrelation function.
؏.؏؏. Finding impulse response of LTI system. To find the impulse response h(t) of an LTIsystem (e.g., a concert hall), i.e., to identify the system, white noise X(t), −∞ <t < ∞, is applied to its input and the output Y(t) is measured. Given the input
and output sample functions, the crosscorrelation RYX(τ) is estimated. Show how
RYX (τ) can be used to find h(t).؏.؏؏. Generating a random process with a prescribed power spectral density. Let S( f ) ≥ 0,
for −∞ < f < ∞, be a real and even function such that
X∞
−∞
S( f )d f = 1.
Define the random process
X(t) = cos(2πFt + Θ),where F ∼ S( f ) and Θ ∼ U[−π , π) are independent. Find the power spectral den-sity of X(t). Interpret the result.
؏.؏؏. Integration. Let X(t) be a zero-mean WSS process with autocorrelation function
RX(τ) = e−|τ|. Let
Y(t) = Xt+1
tX(s)ds.
Problems 21
(a) Is Y(t)WSS?
(b) Is (X(t),Y(t)) jointly WSS?
(c) Find the linearMMSE estimate ofY(t) given X(t). Leave your answer in terms
of e.
(d) Find the linear MMSE estimate of Y(t) given X(t) and X(t + 1).؏.؏؏. Integrators. Let Y(t) be a short-term integration of a WSS process X(t):
Y(t) = 1
TXt
t−TX(u)du.
Find SY ( f ) in terms of SX( f ).؏.؏؏. Derivatives of stochastic processes. Let {X(t)} be a wide-sense stationary random
process with mean zero and autocorrelation function R(τ) = e−|τ|. Recall that a
random process {Y(t)} is continuous in mean square if E[(Y(t + є) − Y(t))2] Ă 0
as є Ă 0.
(a) Find the mean and the variance of X(t).(b) Is X(t) continuous in mean square? Justify your answer.
(c) Now let
Zє(t) = X(t + є) − X(t)є
be an є-approximation of the derivative X̀(t). Find the mean and the variance
of Zє(t).(d) Find the linear MMSE estimate of Zє(t) given (X(t), X(t + є)) and the associ-
ated MSE.
(e) Find the linear MMSE estimate of Zє(t) given X(t) and the associated MSE.
(f) Find the limiting mean and variance of Zє(t) as є Ă 0.