c hapter 5 pairs of random variables
DESCRIPTION
C HAPTER 5 Pairs of Random Variables. Prof. Sang-Jo Yoo [email protected] http://multinet.inha.ac.kr. What we are going to study?. Extend the concepts to two random variables - PowerPoint PPT PresentationTRANSCRIPT
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA Universityhttp://multinet.inha.ac.krhttp://multinet.inha.ac.kr
Multimedia Network Lab.Multimedia Network Lab.
CHAPTER 5
Pairs of Random Variables
Prof. Sang-Jo [email protected]
http://multinet.inha.ac.kr
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
What we are going to study? Extend the concepts to two random variables
Joint pmf, cdf, and pdf to calculate the probabilities of events that involve the joint behavior two random variables.
Expected value to define joint moments Correlation when they are not independent Conditional probabilities involving a pair of random variables.
2
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
3
Multiple Random Variables Vector Random Variable
X is a function that assigns a vector of real numbers to each outcome in S (sample space of the random experiment).
Example The random experiment of selecting one student from a class,
define the following functions : H ( ) = height of student in inches
W ( ) = weight of student in pounds A ( ) = age of student in years. (H ( ), W ( ), A ( )) is vector random variable
A function that assigns a pair of real numbers to each out come in sample space S.
Example 5.1 Example 5.2
,X Y X
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Two Random Variables
4
S
A
X
B
x
y
The event involving a pair of RV (X,Y) are specified by conditions that we are interested in A={X+Y 10}, B={min(X,Y) 5}, C={X2+Y2 100}
To determine the probability that the pair X=(X,Y) in some region B in the plane, find the equivalent event for B in the underlying sample space S:
1 : , in A B X Y B X
in : , in P B P A P X Y B X
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
5
Events and Probabilities For -dimensional random variable X= , we
are particularly interested in events that have the product form
in in in where is a one-dimensional event that involves only Probability of product-form events in in in in in Probability of non-product form events B is approximated by the union of disjoint product-form
events
n 1 2 3( , , )X X X
1{A X 1}A 2X 2}A { nX }nAkA kX
1[ ] [{P A P X1}A 2X 2}A { nX }]nA
1[P X 1, , nA X ]nA
[ ] [ ] [ ]k kk k
P B P B P B
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
6
Example: Product-Form Events
X
y
Example 1
1 2 2{ } { }x X x Y y
1 2( , )x y 2 2( , )x y
x
y
x1 x2
y2
y1
Example 2
1 2 1 2{ } { }x X x y Y y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Pairs of Discrete Random Variables(1)
Vector random variable assumes values from some countable set SX,Y = {(xj, yk), j = 1, 2, • • •, k = 1, 2, • • •}. The joint probability mass function of specifies the probability of the product-form
event {X = xj} ∩ {Y= yk}, j = 1, 2, • • • ,k = 1, 2, • • •. This can be interpreted as the long-term relative frequency of the joint event {X =
xj} ∩ {Y= yk} in a sequence of repetitions of the random experiment.
The probability of any event B is the sum of the pmf over the outcomes in B
Note 5-1 (Example 5.5) How to show pmf graphically: Figure 5.5
7
,X YX
X
,( , )
[ ] ,j k
X Y j kx y B
P in B p x y
X
, ( , ) [( ) ( )] [ , ]X Y j k j k j kp x y P X x Y y P X x Y y
,1 1
( , ) 1X Y j kj k
p x y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Different Pairs of Random Variables
8
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Pairs of Discrete Random Variables(2)
Marginal probability mass functions The joint pmf of X=(X,Y) provides the information about the joint
behavior of X and Y. We are also interested in the probabilities of events involving
each of the RV in isolation.
Similarly,
9
,1
( ) [ ][ , ]
( , );
X j j
j
X Y j kk
p x P X xP X x Y anyting
p x y
,1
( ) ( , ).Y k X Y j kj
p y p x y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Pairs of Discrete Random Variables(3)
Example A urn contains 3 red, 4 white and 5 blue balls. Now, 3 balls are drawn. Let
X and Y be the number of red and white balls chosen, respectively, find the joint probability mass function of X and Y,
10
}].{}[{),(, kYjXPkjP YX
j k 0 1 2 3 PX(k)0 10/220 40/220 30/220 4/220 84/2201 30/220 60/220 18/220 0 108/22
02 15/220 12/220 0 0 27/2203 1/220 0 0 0 1/220
PY(k) 56/220 112/220
48/220 4/22022060)1,1(
312
151413,
CCCCP YX
22012)1,2(
312
1423,
CCCP YX
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Pairs of Discrete Random Variables(4)
Example 5.9 The number of bytes N in a message has a geometric
distribution with parameter p and range SN = {0, 1, 2, • • •}. Messages are broken into packets of maximum length M bytes. Let Q be the number of full packet in a message, and R be the number of bytes left over.
Find the joint pmf and the marginal pmf’s of Q and R. Solution
Q is the quotient of division of N by M, and R is the remainig bytes in the above division. Q takes on values in {0, 1, • • •}; that is, all non – negative integers.
R takes on values in {0, 1, • • • , M – 1}. Interestingly, the joint pmf is relatively easier to compute
11
.)1(][],[ rqMpprqMNPrRqQP
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Pairs of Discrete Random Variables(4)
Marginal pmf of Q is given by
Marginal pmf of R is found to be
12
.,2,1,0,))(1(1
1)1(
)1(
)}]1(,,1,{[][1
0
qppp
ppp
pp
MqMqMqMinNPqQP
qMMM
qM
M
k
kqM
0
[ ] [ { , , 2 , }]1(1 ) , 0,1, , 1.
1qM r r
Mq
P R r P N in r M r M rpp p p r M
p
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint cdf of X and Y (1) Defined as the probability of the product-form event
Joint cumulative distribution function of X and Y
Properties (i) , this is because is a subset of .(ii) This is because and are impossible events.
(iii) Marginal cumulative distribution functions
13
}{}{ 11 yYxX
},{),( 1111, yYxXPyxF YX
),(),( 22,11, yxFyxF YXYX 2121 yyandxx }{}{ 11 yYxX }{}{ 22 yYxX
0),(),( 1,1, xFyF YXYX
1{ } { }X Y y 1{ } { }X x Y 1),(, YXF
][),()(
][],[),()(
,
,
yYPyFyF
xXPYxXPxFxF
YXY
YXX
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint cdf of X and Y (2)(iv) Joint cdf is continuous from the ‘north’ and the ‘east’
This is a generalization of the right continuity property of the one-dimensional cdf
(v) The probability of the rectangle is given by
Example 5.12
Then
X and Y are exponentially distributed with respective parameter α and β
14
),(),(lim
),(),(lim
,,
,,
bxFyxF
yaFyxF
yXYXbx
yXYXax
,(1 )(1 ), 0, 0
( , )0
x y
X Ye e x y
F x yotherwise
,
,
( ) lim ( , ) 1 , 0
( ) lim ( , ) 1 , 0
xX X Yy
yy X Yx
F x F x y e x
F y F x y e y
1 2 1 2,x x x y y y
1 2 1 2 , 2 2 , 2 1 , 1 2 , 1 1, , , , ,X Y X Y X Y X YP x X x y Y y F x y F x y F x y F x y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint cdf of X and Y (3) The cdf can be used to find the probability of events that can
be expressed as the union and intersection of semi-infinite rectangles.
In particular,
15
),(),(),(
],[),(
21,11,12,
212122,
yxFyxFyxF
yYyxXxPyxF
YXYXYX
YX
,
, , , ,
3 5 3 2
5 2
(1 )(1 ), , 0Given ( , )
0
[1 3,2 5](3,5) (3,2) (1,5) (1,2)
(1 )(1 ) (1 )(1 )(1 )(1 ) (1 )(1 )
x y
X Y
X Y X Y X Y X Y
e e x yF x y
otherwise
P X YF F F F
e e e ee e e e
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint PDF of Two Continuous RVs Backgrounds
Joint cdf allows us to compute the probability of events that correspond to “rectangular” shapes in the plane.
To compute the probability of events corresponding to regions other than rectangles, any shape can be approximated by rectangles Bj,k.
The probability of the event can be approximated by the sum of the probabilities of infinitesimal rectangles, and if the cdf is sufficiently smooth, the probability of each rectangle can be expressed in terms of a density function.
16
1A X Y
2 2 1B X Y
, ,,
,j k
j k X Y j kj k x y B
P B P B f x y x y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint pdf of two jointly continuous RVs
When the random variables X and Y are jointly continuous, the probability of an event involving (X,Y) can be expressed as an integral of a joint probability density function. For every event B, which is a subset of the plane, we have
17
,[ ] ( , )X YB non negative
P X in B f x y dx dy
Joint cdf:
The pdf can be obtained from the cdf by differentiation:
.),(),( ,, ydxdyxfyxFy x
YXYX
.),(
),( ,2
, yxyxF
yxf YXYX
2 1
2 11 1 2 2 ,
, ,
[ , ] ( , )
and [ , ] ( , ) ( , ) .
b bX Y
a a
y dy x dxX Y X Y
y x
P a X b a Y b f x y dx dy
P x X x dx y Y y dy f x y dx dy f x y dxdy
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint pdf of two jointly continuous RVs
Since
18
,
[ , ] 1, we have
1 ( , ) .X Y
P X Y
f x y dx dy
The marginal pdf’s and are obtained by differentiating the marginal cdf’s.
Note 5-2 (Example 5.15, Example 5.16)
)(xf X )(yfY
, ,
, ,
,
( ) ( , ) ( ) ( , ),
( ) ( , ) ( , )
( ) ( , ) .
X X Y Y X Y
xX X Y X Y
Y X Y
From F x F x and F y F y wehavedf x f x y dy dx f x y dydx
f y f x y dy
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Jointly Gaussian random variables X and Y are Gaussian random variables with zero mean and unit variance.
The last integral is recognized as the Gaussian pdf with mean ρx and variance 1- ρ2, so the value of integral is one
Hence, fx(x) is the one-dim Gaussian pdf with zero mean and unit variance.
19
.2)1(22
12
12)(
arg
,12
1),(
2/
2
)1(2/)(2/
)1(2/])[(
2
)1(2/
)1(2/)2(
2
)1(2/
,)1(2/)2(
2,
2222
222222
2222
222
xxyx
xxyx
xyyx
X
yxyxYX
edyee
dyee
dyeexf
byobtainedisXofpdfinalmThe
yxeyxf
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Independence of two random variables
Two events are independent if the knowledge that one has occurred gives no clue to the likelihood that the other will occur.
Let X and Y be discrete random variables. Let A1 be the event that X=x, A2 be the event that Y=y. If X and Y are independent, then A1 and A2 are independent.
For continuous random variables: Example
X1= number of students attending the lecture on given day X2= number of tests within that week X3= number of students having a cold X4= number of students having hair cut
Which pair of random variables are independent?
20
1 2 1 2
,
[ ] [ ] [ ]( , ) [ ] [ ] [ ] ( ) ( )X Y X y
P A A P A P Aor p x y p X x and Y y P X x P Y y p x p y
, ( , ) ( ) ( )X Y X Yf x y f x f y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Independence Definition Let X and Y be random variables with joint density fXY and
marginal densities fX and fY, respectively. X and Y are independent if and only if
Remark By integrating the above equation, we have
So that
X and Y are independent if and only if their joint cdf is equal to the product of its marginal cdf’s.
Let X and Y be independent random variables, then the random variables defined by g(X ) and h(Y ) are also independent.
21
.,),()(),( yxallforyfxfyxf YXXY
y x
XY
y x
XY xdxfydyfydxdyxf )()(),(
.,)()(),( yxallforyFxFyxF YXXY
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example Consider the jointly distributed Gaussian random variables with
the joint pdf:
The product of the marginals equals the joint pdf if and only of ρ=0. Hence, X and Y are independent if and only if ρ=0.
What is the interpretation of ρ? It is related to a concept called correlation (to be discussed later).
22
2 2 2
2 2
2 2
( 2 )/2(1 ) ,,2
/2 /2
( )/2
1( , ) , .2 1
1 1( ) , ( )2 2
1( ) ( ) , , .2
x xy yX Y
x yX Y
x yX Y
f x y e x y
f x e f y e
f x f y e x y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint Moments and Expected Values of a Function of Two Random Variables
Remember: Expected value of X: indicates the center of mass of the
distribution of X Variance: expected value of (X-m)2, provides a measure of the
spread of the distribution.
We are interested in: How X and Y vary together? Whether the variation of X and Y are correlated?
If X increases does Y tend to increase or to decrease? The joint moments of X and Y, which are defined as expected
values of functions of X and Y, provide this information.
23
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
24
Expected value of functions of RVs Let Z=g(X,Y ), expected value of Z is given by
Sum of random variables (Example 5.24) Z=X+Y
The random variables do not have to be independent in order that the above formula holds.
( , ) ( , ) X,Y jointly continuous[ ] .
( , ) ( , ) X,Y dixcrete
XY
i j XY i ji j
g x y f x y dxdyE Z
g x y p x y
].[][][][ general,In
].[][')'('')'('
'')','()''(][
2121 nn
YX
XY
XEXEXEXXXE
YExEdyyfydxxfx
dydxyxfyxYXE
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Product of functions of independent random variables: Example 5.25 Suppose X and Y are independent random variables, and g(X,Y )
is separable where g(X,Y )=g1(X )g2(Y ), then
In general, if are independent random variables, then
25
1 2 1 2
1 2
1 2
[ ( ) ( )] ( ') ( ') ( ') ( ') ' '
( ') ( ') ' ( ') ( ') '
[ ( )] [ ( )].
X Y
X Y
E g X g Y g x g y f x f y dx dy
g x f x dx g y f y dy
E g X E g Y
nXX ,,1
)].([)]([)]([)]()()([ 22112211 nnnn XgEXgEXgEXgXgXgE
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Joint Moments, Correlation, and Covariance
The jkth joint moment of X and Y is defined by
Correlation of X and Y is defined by E[XY], specially in Electrical Engineering When E [XY ]=0, then X and Y are orthogonal.
The jkth central moment of X and Y is defined as
When j =2, k =0, it gives VAR(X); and when j=0, k=2, it gives VAR(Y). When j=k=1, it gives COV(X,Y)=E[X-E[X])(Y-E[Y])].
26
( , ) , X and Y jointly continuous[ ]
( , ) X,Y discrete
j k XYj k
j ki n XY i n
i n
x y f x y dxdyE X Y
x y p x y
].])[(])[[( kj YEYXEXE
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Covariance
1. COV[X,Y]=E[XY] if either of the random variables has mean zero.2. When X and Y are independent, then E [XY]=E [X]E[Y] so that
Correlation coefficient of X and Y
Where σX and σy are standard deviations of X and Y, respectively.
27
].[][][][][][][2][
]][][][][[),(
YEXEXYEYEXEYEXEXYE
YEXEXYEYXEXYEYXCOV
0),( YXCOV
,][][][),(
yXyXXY
YEXEXYEYXCOV
If XY=0 then X and Y are said to be uncorrelated.If X and Y are independent, then COV(X,Y)=0 so uncorrelated.
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Correlation Definitions E[XY]: maximize when X goes large Y also goes large. E[{X-E(X)}{Y-E(Y)}]: covariance, consider mean values of X and Y
Compute correlation at the equal conditions: conceptually zero mean If a positive (negative) value of (X-E(X)) tends to be accompanied by a
positive (negative) values of (Y-E(Y)); then COV will be positive. If they tend to have opposite signs, then COV(X,Y) will be negative. If they sometimes have the same sign and sometimes have opposite sign,
then COV(X,Y) will be close to zero. Correlation coefficient XY:
Multiplying either X or Y by a large number will increase the covariance, so need to normalize the covariance.
Properties of ρXY
and so -1 ≤ρXY ≤ 1.
28
),1(2121
])[(])[])([(2])[(
][][0
2
2
2
2
2
XYXY
YYXX
YX
YEYYEYXEXXEXE
YEYXEXE
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
29
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example 5.28 Suppose X and Y have the joint pdf:
The marginal pdf’s are found to be
Correlation coefficient,
30
.0
02),(
otherwisexyee
yxfyx
XY
0
0 0
0
2
0
0
.1)1(2
2][
.41)(
21][
;45][)1(2][
,23)1(2][
0,22)(
0),1(22)(
dxxeexe
dydxexyeXYE
YVARandYE
XedxeexXVAR
dxexeXE
xedxeeyf
xeedyeexf
xxx
x yx
xx
xx
yx
y
yxY
xxx yxX
.5
1
41
45
21
231
XY
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Conditional Probability and Conditional Expectation
Many random variables in practical interest Are not independent Output Y of a communication channel must depend on the input
X Consecutive samples of a waveform that varies slowly are likely
to be close in value.
We are interested in Computing the probability of event concerning the random
variable Y given that we know X=x. The expected value of Y given X=x.
31
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
32
Conditional Probability Recall the formula:
Case 1: X is a Discrete Random Variable For X and Y discrete random variables,
If X and Y are independent
How to calculate joint pmf from conditional marginal pmf’s
How to calculate
.][
],[]|[xXP
xXAinYPxXAinYP
in
[ in | ] |j
k Y j ky A
P Y A X x p y x
,
| k jY j k j Y j
k
P X x Y yp y x P Y y p y
P X x
, ,X Y k jp x y
, , | |X Y k j Y j k X k X k j Y jp x y p y x p x p x y p y
in in
in
,all all
all all
in , |
| in |k j A k j A
k j A k
X Y k j Y j k X kx y x y
X k Y j k k X kx y x
P Y A p x y p y x p x
p x p y x P Y A X x p x
Example 5.29Example 5.30
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Suppose X is a discrete RV and Y is a continuous RV Conditional cdf of Y given X=xk
Conditional pdf of Y given X=xk
If X and Y are independent
33
0][,][
],[)|(
k
k
kkY xXP
xXPxXyYPxyF
).|()|( kYkY xyFdydxyf
[ , ] [ ] [ ]so ( | ) ( ) and ( | ) ( )
k k
Y k Y Y k Y
P Y y X x P Y y P X xF y x F y f y x f y
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Case 2: X is a Continuous Random Variable If X is a continuous random variable, then Suppose X and
Y are jointly continuous random variables with a joint pdf that is continuous and non-zero over some region, then the conditional cdf of Y given X=x is defined by
Taking h 0,
The conditional pdf of Y given X=x is given by
If X and Y are independent, then so and .
34
.0)( xXP
).|(lim)|(0
hxXxyFxyF YhY
( ', ') ' ' ( , ') '[ , ][ ] ( )( ') '
when h is verysmall.
y x h yXY XY
xx h
XXx
f x y dx dy h f x y dyP Y y x X x hP x X x h hf xf x dx
.)(
')',()|(
xf
dyyxfxyF
X
y
XYY
.)(
),()|()|(xf
yxfxyFdydxyf
X
XYYY
)()(),( yfxfyxf YXXY )()|( yfxyf YY )()|( yFxyF YY
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example 5.32 Suppose the joint pdf of X and Y is given by
find and .
Solution
35
,0
02),(
otherwisexyee
yxfyx
XY
( | )Xf x y ( | )Yf y x
'0
' 2
( ) ( , ') ' 2 ' 2 (1 ), 0
( ) ( ', ) ' 2 ' 2 , 0 ,
xx y x xX XY
x y yY XY
f x f x y dy e e dy e e x
f y f x y dx e e dx e y
y
.01)1(2
2)(
),()|(
,02
2)(
),()|( )(2
xyfore
eee
eexf
yxfxyf
xyforee
eeyf
yxfyxf
x
y
xx
yx
X
XYY
yxy
yx
Y
XYX
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example X = input voltage, Y = output = input + noise voltage. Here, X=1 or -1 with equal probabilities, the noise is uniformly
distributed from -2 to 2 with equal probabilities. Whe X=1, Y becomes uniformly distributed in [-1,3] so
The conditional cdf of Y:
For example,
36
1/ 4 1 3( | 1) .
0 otherwiseYy
f y X
.
3,1
31,4
1'41
1,0
')1|'(]1|[)1|(
1
y
yydy
y
dyXyfXyYPXyF
y
y
YY
0
1
1[ 0 | 1] ( ' | 1) ' (0 | 1) ,4
3 2 1[1 2 | 1] (2 | 1) (1| 1) ,4 4 4
1 1 1[ 0, 1] [ 0 | 1] [ 1]4 2 8
Y Y
Y Y
P Y X f y X dy F X
P Y X F X F X
P Y X P Y X P X
11
134
330
)1|(
y
yyy
XyFY
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Conditional Expectation (1) The conditional expectation of Y given X=x is given by
When X and Y are both discrete random variables
On the other hand, E[Y|x] can be viewed as a function of x:
Correspondingly, this gives rise to the random variable:
37
.)|(]|[ dyxyyfxYE Y
[ | ] ( | )j
k j Y j ky
E Y x y p y x
].|[)( xYExg
].|[)( XYEXg
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Conditional Expectation(2) What is
Note that
Suppose X and Y are jointly continuous random variables
Generalization [in the above proof, change y to h(y)]; and in particular,
38
?]]|[[ XYEE
[ | ] ( ) , is continuous.[ [ | ]]
[ | ] ( ), is discrete.k
X
k X kx
E Y x f x dx XE E Y X
E Y x p x X
].[)(
),(
)()|(
)(]|[]]|[[
YEdyyyf
dxdyyxfy
dxxfdyxyyf
dxxfxYEXYEE
Y
XY
XY
X
]]|)(([)]([ XYhEEYhE ]].|[[][ XYEEYE kk
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example 5.37 Binary Communication System The input X to a communication channel assumes the values +1
or -1 with probability 1/3 and 2/3. The output Y of the channel is given by Y=X+N, where N is zero-mean, unit variance Gaussian RV.
Find the mean of the output Y.
Solution Since Y is Gaussian RV with mean +1 when X=+1, and -1 when X=-1,
the conditional expected value of Y given X are
Since ,
39
| 1 1 and | 1 1E Y X E Y X
|E Y E E Y X
0
|
= | 1 1/ 3 1 2 / 3 1/ 3
k
k X kx
k
E Y E Y x p x
E Y X k P X k
The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
40
Multiple Random Variables Joint cdf of
Joint pmf of
Marginal pmf's
Joint cdf of
Marginal pdf's
1 2, , , nX X X
1 2, , , nX X X
1 2, , , 1 2 1 1( , , , ) [ , , ]nX X X n n nF x x x P X x X x
1 2, , , 1 2 1 1( , , , ) [ , , ]nX X X n n np x x x P X x X x
1 2
1 1 1
, , , 1 2( ) [ ] ( , , , )j n
j j n
X j j j X X X nx x x x
p x P X x p x x x
1 2, , , nX X X1
1 2 1 1
' ' ' ', , , 1 2 , 1( , , , ) ( , )
n
n n
xx
X X X n X X n nF x x x f x x dx dx
1 2 1 1
' ', , , , 1 2 1 , , 1 1( , , , ) ( , , , )
n nX X X n X X n n nf x x x f x x x dx