part 1 random processes for communications
TRANSCRIPT
Part 1 Random Processes for Communications
o A good mathematical model for a system is the basis of its analysis.
o Two models are often considered:n Deterministic model
o No uncertainty about its time-dependent behavior at any instance of time
n Random or stochastic modelo Uncertain about its time-dependent behavior at any
instance of timeo but certain on the statistical behavior at any instance of
time© Po-Ning [email protected] 1-2
System Models
© Po-Ning [email protected] 1-3
Examples of Stochastic Models
o Channel noise and interferenceo Source of information, such as voice
© Po-Ning [email protected] 1-4
Notion of Relative Frequency
o How to determine the probability of “head appearance” for a coin?
o Answer: Relative frequency.Specifically, by carrying out n coin-tossing experiments, the relative frequency of head appearance is equal to Nn(A)/n, where Nn(A) is the number of head appearance in these nrandom experiments.
© Po-Ning [email protected] 1-5
Notion of Relative Frequency
o Is relative frequency close to the true probability (of head appearance)?n It could occur that 4-out-of-10 tossing results are
“head” for a fair coin!o Can one guarantee that the true “head
appearance probability” remains unchanged(i.e., time-invariant) in each experiment performed at different time instance?
© Po-Ning [email protected] 1-6
Notion of Relative Frequency
o Similarly, the previous question can be extended to “In a communication system, can we estimate the noise by repetitive measurements at consecutive but different time instance?”
o Some assumptions on the statistical models are necessary!
© Po-Ning [email protected] 1-7
Conditional Probability
o Definition of conditional probability
o Independence of eventsn A knowledge of occurrence of event A tells us no
more about the probability of occurrence of event B than we knew without this knowledge.
n Hence, they are statistically independent.
)()(
)()( )|(
APBAP
ANBANABP
n
n !!=÷÷
ø
öççè
æ»
)()|( BPABP =
© Po-Ning [email protected] 1-8
Random Variable
o A non-negative function fX(x) satisfies
is called the probability density function (pdf) of random variable X.
o If the pdf of X exists, then
xxFxf X
X ¶¶
=)()(
𝐹! 𝑥 = Pr X ≤ 𝑥 = ("#
$𝑓! 𝑡 d𝑡
© Po-Ning [email protected] 1-9
Random Variable
o It is not necessarily true thatn If
then the pdf of X exists and equals fX(x).
,)()(xxFxf X
X ¶¶
=
© Po-Ning [email protected] 1-10
Random Vectorn If its joint density fX,Y(x,y) exists, then
n The conditional density of Y given that [X = x] is
provided that fX(x) ¹ 0.
fX,Y (x, y) =@2FX,Y (x,y)
@x@y
fY |X(y|x) = fX,Y (x,y)fX(x)
where 𝐹!,# 𝑥, 𝑦 = Pr[𝑋 ≤ 𝑥 and 𝑌 ≤ 𝑦]
© Po-Ning [email protected] 1-11
Random Process
o Random process: An extension of multi-dimensional random vectorsn Representation of two-dimensional random vector
o (X,Y) = (X(1), X(2)) = {X(j), jÎI}, where the index set I equals {1, 2}.
n Representation of m-dimensional random vectoro {X(j), jÎI}, where the index set I equals {1, 2,…, m}.
© Po-Ning [email protected] 1-12
Random Processn How about {X(t), tÎÂ}?
o It is no longer a random vector since the index set is continuous!
o This is a suitable model for, e.g., a noise because a noise often exists continuously in time.
© Po-Ning [email protected] 1-13
Stationarity
o The statistical property of a random process encountered in real world is often independent of the time at which the observation (or experiment) is initiated.
o Mathematically, this can be formulated as that for any t1, t2, …, tk and t:
),...,,(),...,,(
21)(),...,(),(
21)(),...,(),(
21
21
ktXtXtX
ktXtXtX
xxxFxxxF
k
k
=+++ ttt
o Why introducing “stationarity?”n With stationarity, we can be certain that the
observations made at different instances of time have the same distributions!
n For example, X(0), X(T), X(2T), X(3T), ….
n Suppose that Pr[X(0) = 0] = Pr[X(0)=1] = ½. Can we guarantee that the relative frequency of “1’s appearance” for experiments performed at severaldifferent instances of time approach ½ by stationarity? No, we need an additional assumption!
© Po-Ning [email protected] 1-14
Stationarity
© Po-Ning [email protected] 1-15
Mean Function
o The mean of a random process X(t) at time t is equal to:
where fX(t)(×) is the pdf of X(t) at time t.
o If X(t) is stationary, µX(t) is a constant for all t.
dxxfxtXEt tXX )()]([)( )(ò¥
¥-×==µ
© Po-Ning [email protected] 1-16
Autocorrelation
o The autocorrelation function of a (possibly complex) random process X(t) is given by:
o If X(t) is stationary, the autocorrelation function RX(t1, t2) is equal to RX(t1 - t2, 0).
© Po-Ning [email protected] 1-17
Autocorrelation
A short-hand for autocorrelation function of a stationary process
© Po-Ning [email protected] 1-18
Autocorrelation
o Conceptually, n Autocorrelation function = “power correlation”
between two time instances t1 and t2.n “Variance” is the degree of variation to the standard
value (i.e., mean).
© Po-Ning [email protected] 1-19
Autocovariance
© Po-Ning [email protected] 1-21
Wide-Sense Stationary (WSS)
o Since in most cases of practical interest, only the first two moments (i.e., µX(t) and CX(t1, t2)) are concerned, an alternative definition of stationarity is introduced.
o Definition (Wide-Sense Stationarity) A random process X(t) is WSS if
îíì
-==
)(),(constant;)(
2121 ttCttCt
XX
Xµ
îíì
-==
).(),(constant;)(
or 2121 ttRttR
t
XX
Xµ
© Po-Ning [email protected] 1-22
Wide-Sense Stationary (WSS)
o Alternative names for WSSn weakly stationaryn stationary in the weak sensen second-order stationary
o If the first two moments of a random process exist (i.e., are finite), then strictly stationaryimplies weakly stationary (but not vice versa).
© Po-Ning [email protected] 1-23
Cyclostationarity
o Definition (Cyclostationarity) A random process X(t) is cyclostationary if there exists a constant T such that
îíì
=++=+
).,(),(;)()(
2121 ttCTtTtCtTt
XX
XX µµ
© Po-Ning [email protected] 1-24
Properties of Autocorrelation Function for WSS Random Process
1. Mean Square Value: RX(0) = E[|X(t)|2]2. Conjugate Symmetry:
n Recall that autocorrelation function = “power correlation” between two time instances t1 and t2.
n For a WSS process, this “power correlation” only depends on time difference.
n Hence, we only need to deal with RX(t) here.
© Po-Ning [email protected] 1-25
Properties of Autocorrelation Function for WSS Random Process
3. Real Part Peaks at zero: |Re{RX(t)}| ≦ RX(0) Proof:
Hence,with equality holding when
© Po-Ning [email protected] 1-26
Properties of Autocorrelation Function for WSS Random Process
o Operational meaning of autocorrelation function:n The “power” correlation of a random process at t
seconds apart.n The smaller RX(t) is, the less the correlation
between X(t) and X(t+t). o Here, we assume 𝑋(𝑡) is a real-valued random process.
© Po-Ning [email protected] 1-27
Properties of Autocorrelation Function for WSS Random Process
n If RX(t) decreases faster, the correlation decreases faster.
RX(t)
t
© Po-Ning [email protected] 1-28
Example: Signal with Random Phase
o Let X(t) = A cos(2pfct + Q), where Q is uniformly distributed over [-p, p).n Application: A local carrier at the receiver side may
have a random “phase difference” with respect to the phase of the carrier at the transmitter side.
1-29
Example: Signal with Random Phase
ChannelEncoder
…0110Modulator
…,-m(t), m(t), m(t), -m(t)
m(t)
T
Ä
Carrier waveAccos(2pfct)
s(t)Å
w(t)=0
x(t)=A cos(2pfct)
Ä
Local carriercos(2pfct+Q)
òTdt
0
correlator
yT>< 00110…
X(t)=A cos(2pfct+Q)Local carriercos(2pfct)
An equivalent view:
© Po-Ning [email protected] 1-30
Example: Signal with Random Phase
Then
( )
( )
.0
)2sin()2sin(2
)2sin(2
)2cos(2
21)2cos(
)]2cos([)(
=
+--+=
+=
+=
+=
Q+=
-
-
-
ò
ò
tftfA
tfA
dtfA
dtfA
tfAEt
cc
c
c
c
cX
ppppp
pqp
qpqp
qp
qp
pµ
p
p
p
p
p
p
© Po-Ning [email protected] 1-31
Example: Signal with Random Phase
[ ](
[ ])
( ) ( )( )
( ).)(2cos2
)(2cos)(22cos21
2
)2()2(cos
)2()2(cos21
2
21)2cos()2cos(
)]2cos()2cos([),(
21
2
2121
2
21
21
2
212
2121
ttfA
dttfttfAdtftf
tftfA
dtftfA
tfAtfAEttR
c
cc
cc
cc
cc
ccX
-=
-+++=
+-++
+++=
++=
Q+×Q+=
ò
ò
ò
-
-
-
p
qppqp
qpqpq
pqpqp
qp
pqpq
pp
p
p
p
p
p
p
Hence, X(t) is WSS.
= 0
© Po-Ning [email protected] 1-33
Example: Signal with Random Delay
o Let
where …, I-2, I-1, I0, I1, I2, … are independent, and each Ij is either -1 or +1 with equal probability, and
)()( dn
n tnTtpIAtX --××= å¥
-¥=
îíì <£
=otherwise,00,1
)(Tt
tp
© Po-Ning [email protected] 1-34
Example: Signal with Random Delay
I0 I1 I2 I3I-1I-2I-3I-4
I0
I1 I2 I3I-1I-2
I-3
I-4
1-35
Example: Signal with Random Delay
ChannelEncoder
…0110Modulator
m(t) = p(t)
T
No (or ignore)carrier wave
s(t)Å
w(t)=0x(t) = A p(t)
correlator
yT>< 00110…
…,-m(t), m(t), m(t), -m(t)
R T0 dt X(t) = A p(t−td)
R T�td�td
dt
An equivalent view:
© Po-Ning [email protected] 1-36
Example: Signal with Random Delayn By assuming that td is uniformly distributed over [0,
T), we obtain:
0
)]([0
)]([][
)()(
=
--××=
--××=
úûù
êëé --××=
å
å
å
¥
-¥=
¥
-¥=
¥
-¥=
dn
dn
n
dn
nX
tnTtpEA
tnTtpEIEA
tnTtpIAEtµ
© Po-Ning [email protected] 1-37
Example: Signal with Random Delayn A useful probabilistic rule: E[X] = E[E[X|Y]]
[ ][ ]dttXtXEEtXtXE )()()]()([ 2121 =
So, we have:
Note:
���
��
E[X|Y ] =
�
Xx fX|Y (x|y)dx = g(y)
E�E[X|Y ]
�=
�
Yg(y) fY (y)dy
[ ]
)()(
)()(][
)()(][
]|)()([]|[
)()(
)()(
212
2122
212
212
21
21
dn
d
dn
dn
dn m
dmn
ddn m
ddmn
ddm
mdn
n
d
tnTtptnTtpA
tnTtptnTtpIEA
tmTtptnTtpIIEA
ttmTtptnTtpEtIIEA
ttmTtpIAtnTtpIAE
ttXtXE
----=
----=
----=
----=
úúû
ù
êêë
é÷ø
öçè
æ --××÷ø
öçè
æ --××=
å
å
åå
åå
åå
¥
-¥=
¥
-¥=
¥
-¥=
¥
-¥=
¥
-¥=
¥
-¥=
¥
-¥=
¥
-¥=
© Po-Ning [email protected]
.for 0][][][ Since mnIEIEIIE mnmn ¹==
1-38© Po-Ning [email protected] 1-38
© Po-Ning [email protected] 1-39
Among �� < n < �, there is at most one n that can make
p(t1 � nT � td)p(t2 � nT � td) = 1.
© Po-Ning [email protected] 1-39
© Po-Ning [email protected] 1-40© Po-Ning [email protected] 1-40
© Po-Ning [email protected] 1-43
Cross-Correlation
o The cross-correlation between two processes X(t) and Y(t) is:
o Sometimes, their correlation matrix is given instead for convenience:
úû
ùêë
é=
),(),(),(),(
),(,
,, utRutR
utRutRut
YXY
YXXYXR
© Po-Ning [email protected] 1-44
Cross-Correlation
o If X(t) and Y(t) are jointly WSS, then
úû
ùêë
é----
=
-=
)()()()(
)(),(
,
,
,,
utRutRutRutR
utut
YXY
YXX
YXYX RR
© Po-Ning [email protected] 1-45
Example: Quadrature-Modulated Random Delay Processes
o Consider a pair of quadrature decomposition of X(t) as:
where Q is independent of X(t) and is uniformly distributed over [0, 2p), and
( )îíì
Q+=Q+=)2sin()()(
2cos)()(tftXtXtftXtX
cQ
cI
pp
)()( dn
n tnTtpIAtX --××= å¥
-¥=.
© Po-Ning [email protected] 1-46
Example: Quadrature-Modulated Random Delay Processes
),())(2sin(21
2))(2sin()2)(2sin(),(
)]2cos()2[sin()]()([)]2sin()()2cos()([
)]()([),(,
utRutf
tufutfEutR
tfufEuXtXEufuXtftXE
uXtXEutR
Xc
ccX
cc
cc
QIXX QI
--=
úûù
êëé -+Q++
=
Q+Q+=Q+×Q+=
=
p
pppp
pp
=0
© Po-Ning [email protected] 1-47
Example: Quadrature-Modulated Random Delay Processes
© Po-Ning [email protected] 1-48
Example: Quadrature-Modulated Random Delay Processes
n Notably, if t = u, i.e., two quadrature components are synchronized, then
which indicates that simultaneous observations of the quadrature-modulated processes are “orthogonal” to each other! (See Slide 1-59 for a formal definition of orthogonality.)
0),(, =ttRQI XX
© Po-Ning [email protected] 1-49
Ergodicityo For a random-process-modeled noise (or random-
process-modeled source) X(t), how can we know its mean and variance?n Answer: Relative frequency. n How can we get the relative frequency?
o By measuring X(t1), X(t2), …, X(tn), and calculating their average, it is expected that this time average will be close to its mean.
o Question: Will this time average be close to its mean, if X(t) is stationary ?n Even if for a stationary process, the mean function µX(t) is
independent of time t, the answer is negative!
© Po-Ning [email protected] 1-50
Ergodicityn An additional ergodicity assumption is necessary
for time average converging to the ensemble average µX.
© Po-Ning [email protected] 1-51
Time Average versus Ensemble Average
o Examplen X(t) is stationary.n For any t, X(t) is uniformly distributed over {1, 2, 3,
4, 5, 6}.n Then, ensemble average is equal to:
5.3616
615
614
613
612
611 =×+×+×+×+×+×
© Po-Ning [email protected] 1-52
Time Average versus Ensemble Average
n We make a series of observations at time 0, T, 2T, …, 10T to obtain 1, 2, 3, 4, 3, 2, 5, 6, 4, 1. (These observations are deterministic!)
n Then, the time average is equal to:
1.310
1465234321=
+++++++++
© Po-Ning [email protected] 1-53
Ergodicity
o Definition. A stationary process X(t) is ergodic in the mean if
where
[ ][ ] 0)(Varlim .2
and 1,)(lim Pr.1
=
==
¥®
¥®
T
T
XT
XXT
µ
µµ
ò-=T
TX dttXT
T )(21)(µ
Time averageEnsembleaverage
© Po-Ning [email protected] 1-54
Ergodicity
o Definition. A stationary process X(t) is ergodic in the autocorrelation function if
where
[ ][ ] 0);(Varlim .2
and 1,)();(lim Pr.1
=
==
¥®
¥®
TR
RTR
XT
XXT
t
tt
Time averageEnsembleaverage
𝑅! 𝜏; 𝑇 =12𝑇("#
#𝑋 𝑡 + 𝜏 𝑋∗ 𝑡 d𝑡
© Po-Ning [email protected] 1-55
Ergodicity
o Experiments (or observations) on the same process can only be performed at different time.
o “Stationarity” only guarantees that the observations made at different time come from the same distribution.n Example. Rolling two different fair dices will get
two results but the two results have the same distribution.
© Po-Ning [email protected] 1-56
Statistical Average of Random Variables
o Alternative names of ensemble averagen Meann Expected value, or expectation valuen Sample average
o How about the sample average of a function g( ) of a random variable X ?
ò¥
¥-= dxxfxgXgE X )()()]([
© Po-Ning [email protected] 1-57
Statistical Average of Random Variableso The nth moment of random variable X
n The 2nd moment is also named mean-square value.o The nth central moment of random variable X
n The 2nd central moment is also named variance.n Square root of the 2nd central moment is also named
standard deviation.
E[Xn] =R1�1 xnfX(x)dx
© Po-Ning [email protected] 1-58
Joint Moments
o The joint moment of X and Y is given by:
n When i = j = 1, the joint moment is specifically named correlation.
n The correlation of centered random variables is specifically named covariance.
© Po-Ning [email protected] 1-59
Joint Momentsn Two random variables, X and Y, are uncorrelated if
Cov[X, Y] = 0.n Two random variables, X and Y, are orthogonal if
E[XY*] = 0.n The covariance, normalized by two standard
deviations, is named correlation coefficient of Xand Y.
YX
YXss
r ],[Cov=
© Po-Ning [email protected] 1-60
Stable Linear Time-Invariant (LTI) System
o Linearn Y(t) is a linear function of X(t).n Specifically, Y(t) is a weighted sum of X(t).
o Time-invariantn The weights are time-independent.
o Stable n Dirichlet’s condition (defined later) andn “Stability” implies that if the input is an energy function (i.e., finite
energy), the output is an energy function.
¥<ò¥
¥-tt dh 2|)(|
ImpulseResponse
h(t)X(t) Y(t)
© Po-Ning [email protected] 1-61
Example of LTI Filter: Mobile Radio Channel
Transmitter Receiver
),( 11 ta
),( 22 ta
),( 33 ta
)(tX )(tY
© Po-Ning [email protected] 1-62
Transmitter Receiver
ò¥
¥--= ttt dtXhtY )()()(
)(tX )(tY
Example of LTI Filter: Mobile Radio Channel
© Po-Ning [email protected] 1-63
o What are the mean and autocorrelation functions of the LTI filter output Y(t)?n Suppose X(t) is stationary and has finite mean.n Supposen Then
òò
ò¥
¥-
¥
¥-
¥
¥-
=-=
úûù
êëé -==
ttµttt
tttµ
dhdtXEh
dtXhEtYEt
X
Y
)()]([)(
)()()]([)(
¥<ò¥
¥-tt dh |)(|
Stable Linear Time-Invariant (LTI) System
© Po-Ning [email protected] 1-64
Zero-Frequency (ZF) or Direct Current (DC) Response
nThe mean of the LTI filter output process is equal to the mean of the stationary filter input multiplied by the DC response of the system.
ò¥
¥-= ttµµ dhXY )(
ImpulseResponse
h(t)1
© Po-Ning [email protected] 1-65
Autocorrelation Relation of LTI system
© Po-Ning [email protected] 1-66
Important Fact: WSS Input Induces WSS Outputo From the above derivations, we conclude:
n For a stable LTI filter, a WSS input guarantees to induce a WSS output.
n In general (not necessarily WSS),
n As the above two quantities also relate in the “convolution” form, a spectrum analysis is perhaps better in characterizing their relationship.
µY (t) =
Z 1
�1h(⌧)µX(t� ⌧)d⌧
RY (t, u) =
Z 1
�1
Z 1
�1h(⌧1)h
⇤(⌧2)RX(t� ⌧1, u� ⌧2)d⌧2d⌧1
© Po-Ning [email protected] 1-67
Summaryo Random variable, random vector and random processo Autocorrelation and crosscorrelationo Definition of WSSo Why ergodicity?
n Time average as a good “estimate” of ensemble average