1

Chapter 5 Multiple Random Variables§ 5.1 Joint Cumulative Distribution Function

The joint CDF FX,Y(x,y)=P[X<=x, Y<=y] is a complete probability model for any pair of random variables X and Y.

Definition 5.1 Joint Cumulative Distribution Function (CDF)

𝑭𝑿,𝒀 𝒙, 𝒚 = 𝑷[𝑿 ≤ 𝒙, 𝒀 ≤ 𝒚]

2

§ 5.1 Joint Cumulative Distribution Function

Theorem 5.1

𝑎. )0 ≤ 𝐹2,3 𝑥, 𝑦 ≤ 1

Foranypairofrandomvariables,X,Y,

𝑏. )𝐹2,3 ∞,∞ = 1

𝑐. )𝐹2 𝑥 = 𝐹2,3 𝑥, ∞ 𝑑. )𝐹3 𝑦 = 𝐹2,3 ∞, 𝑦

𝑒. )𝐹2,3 𝑥, −∞ =0 𝑓. )𝐹2,3 −∞, 𝑦 =0

g.)If x<=x1 andy<=y1,then

𝐹2,3 𝑥, 𝑦 ≤ 𝐹2,3(𝑥?, 𝑦?)

3

§ 5.1 Joint Cumulative Distribution Function

Example 5.2

Xyearsistheageofchildrenenteringfirstgradeinaschool.Yistheageofchildrenenteringsecondgrade.ThejointCDFofXandYis:

𝐹2,3 𝑥, 𝑦 =

0, 𝑥 < 50, 𝑦 < 6

𝑥 − 5 𝑦 − 6 , 5 ≤ 𝑥 < 6, 6 ≤ 𝑦 < 7𝑦 − 6, 𝑥 ≥ 6, 6 ≤ 𝑦 < 7𝑥 − 5, 5 ≤ 𝑥 < 6, 𝑦 ≥ 7

1, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

FinFX(x)andFY(y)

4

§ 5.1 Joint Cumulative Distribution Function

Theorem 5.2

𝑷 𝒙𝟏 < 𝑿 ≤ 𝒙𝟐, 𝒚𝟏 < 𝒀 ≤ 𝒚𝟐 = 𝑭𝑿,𝒀 𝒙𝟐, 𝒚𝟐 − 𝑭𝑿,𝒀 𝒙𝟐, 𝒚𝟏 − 𝑭𝑿,𝒀 𝒙𝟏, 𝒚𝟐 + 𝑭𝑿,𝒀 𝒙𝟏, 𝒚𝟏

Quiz 5.1

ExpressthefollowingextremevaluesofthejointCDF𝐹2,3 𝑥, 𝑦 asnumbersorintermsoftheCDF’s𝐹2 𝑥 and𝐹3 𝑦

𝑎. )𝐹2,3 −∞, 2 𝑏. )𝐹2,3 −∞,∞

𝑐. )𝐹2,3 ∞, 𝑦 𝑑. )𝐹2,3 ∞, −∞

5

§ 5.2 Joint Probability Mass Function (Discrete)

Definition 5.2 Joint Probability Mass Function (PMF)

ThejointprobabilitymassfunctionofdiscreterandomvariableXandYis

𝑃2,3 𝑥, 𝑦 = 𝑃[𝑋 = 𝑥, 𝑌 = 𝑦]

NecessaryandSufficientConditionsforaFunctiontobeadiscreteJointProbabilityMassFunction

1. 𝑃2,3 𝑥, 𝑦 ≥ 0

2. S S 𝑃2,3 𝑥, 𝑦 = 1�

UVVW

�

UVVX

6

§ 5.2 Joint Probability Mass Function (Discrete)

Example

Inanautomobileplanttwotasksareperformedbyrobots.Thefirstentailsweldingtwojoints;thesecond,tighteningthreebolts.LetXdenotethenumberofdefectiveweldsandYthenumberofimproperlytightenedboltsproducedpercar.SinceXandYareeachdiscrete,(X,Y)isatwo-dimensionaldiscreterandomvariable.PastdataindicatethatthejointmassfunctionisasshowninTable.

x/y

0

1

2

1 2 3 4

0.840

0.060

0.010

0.030

0.010

0.005

0.020

0.008

0.004

0.010

0.002

0.001

𝑃2,3 𝑥, 𝑦 =

0.840, 𝑥 = 0, 𝑦 = 00.030, 𝑥 = 0, 𝑦 = 10.020, 𝑥 = 0, 𝑦 = 20.010, 𝑥 = 0, 𝑦 = 3… , … . .

0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

7

§ 5.2 Joint Probability Mass Function (Discrete)

Theorem 5.3

FordiscreterandomvariableXandYandanysetBintheX,Yplane,theprobabilityoftheevent{(X,Y)∈ 𝐵}is

𝑃 𝐵 = S 𝑃2,3(𝑥, 𝑦)�

(X,W)∈_

8

§ 5.3 Marginal Probability Mass Function (Discrete)

Fordiscreterandomvariable,themarginalPMFsPX(x)andPY(y)areprobabilitymodelsfortheindividualrandomvariablesXandY

𝑃2 𝑥 = S 𝑃2,3(𝑥, 𝑦)�

W∈`a

Theorem 5.4

FordiscreterandomvariablesXandYwithjointPMFPX,Y(x,y)

𝑃3 𝑦 = S 𝑃2,3(𝑥, 𝑦)�

X∈`b

Ychooseallther.v.fromSY

X chooseallther.v.fromSX

9

§ 5.4 Joint Probability Density Function (Continuous)

Theorem 5.5

Definition 5.3 Joint Probability Density Function (PDF)

ThejointPDFofthecontinuousrandomvariablesXandYisafunctionfX,Y(x,y)withtheproperty

𝐹2,3 𝑥, 𝑦 = c c 𝑓2,3 𝑢, 𝑣 𝑑𝑢𝑑𝑣f

gf

f

gf

𝑓2,3 𝑥, 𝑦 =𝜕i𝐹2,3(𝑥, 𝑦)

𝜕𝑥𝜕𝑦

10

§ 5.6 Independent Random Variable

Example 5.12

Definition 5.4 Independent Random Variable

RandomvariablesX andY areindependent ifandonlyif

Discrete: 𝑃2,3 𝑥, 𝑦 = 𝑃2(𝑥)𝑃3(𝑦)

Continuous: 𝑓2,3 𝑥, 𝑦 = 𝑓2(𝑥)𝑓3(𝑦)

𝑓2,3 = j4𝑥𝑦, 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 10, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

AreXandYindependent?

11

§ 5.6 Independent Random Variable

Quiz 5.6

(A)RandomvariablesX andY,andRandomvariablesQandGhavejointPMFs:

PX,Y(x,y)

0

1

2

0 1 2

0.01

0.09

0

0

0.09

0

0

0

0.81

PQ,G(q,g)

0

1

0 1 2

0.06

0.04

0.18

0.12

0.12

0.8

AreXandYindependentAreQandG independent

(B)RandomvariablesX1andX2areindependentandidenticallydistributedwithprobabilitydensityfunction

𝑓2(𝑥) = j𝑥/2, 0 ≤ 𝑥 ≤ 20, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

WhatisthejointPDF 𝑓2l,2m 𝑥?, 𝑥i ?

12

§ 5.7 Expected Value of a Function of Two Random Variables

Theorem 5.9

ForRandomvariablesX andY,theexpectedvalueofW =g(X,Y)is

Discrete: 𝐸 𝑊 = S S 𝑔(𝑥, 𝑦)𝑃2,3(𝑥, 𝑦)�

W∈`a

�

X∈`b

Continuous: 𝐸 𝑊 = c c 𝑔 𝑥, 𝑦 𝑓2,3 𝑥, 𝑦 𝑑𝑥𝑑𝑦f

gf

f

gf

Theorem 5.10

𝐸 𝑎?𝑔? 𝑋, 𝑌 + ⋯+ 𝑎s𝑔s 𝑋, 𝑌 = 𝑎?𝐸 𝑔? 𝑋, 𝑌 + ⋯+ 𝑎s𝐸 𝑔s 𝑋, 𝑌

13

§ 5.7 Expected Value of a Function of Two Random Variables

Theorem 5.11

ForanytwoRandomvariablesX andY

Theorem 5.12

𝑉𝑎𝑟 𝑋 + 𝑌 = 𝑉𝑎𝑟 𝑋 + 𝑉𝑎𝑟 𝑌 + 2𝐸[(𝑋 − 𝜇2)(𝑌 − 𝜇3)]

E[X+Y]=E[X]+E[Y]

14

§ 5.7 Expected Value of a Function of Two Random Variables

Example

RandomvariablesXandYhavejointPDF

𝑓2,3(𝑥, 𝑦) = j5𝑥i/2, −1 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 𝑥i

0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

(a) WhatareE[X]andVar[X]?(b) WhatareE[Y]andVar[Y]?(c) WhatisCov[X,Y]?(d) WhatisE[X+Y]?(e) WhatisVar[X+Y]?

15

§ 5.8 Covariance, Correlation and Independence

Definition 5.5 Covariance

Thecovariance oftworandomvariablesX andY is

𝐶𝑜𝑣 𝑋, 𝑌 = 𝐸[(𝑋 − 𝜇2)(𝑌 − 𝜇3)]

Definition 5.6 Correlation Coefficient

ThecorrelationcoefficientoftworandomvariablesX andY is

𝜌2,3 =𝐶𝑜𝑣[𝑋, 𝑌]

𝑉𝑎𝑟 𝑋 𝑉𝑎𝑟[𝑌]�=𝐶𝑜𝑣[𝑋, 𝑌]𝜎2𝜎3

16

§ 5.8 Covariance, Correlation and Independence

Theorem 5.13 If𝑋y = 𝑎𝑋 + 𝑏 and𝑌y = 𝑐𝑌 + 𝑑,then

Theorem 5.14

−1 ≤ 𝜌2,3≤ 1

(a)𝜌2y,3y = 𝜌2,3 (b)𝐶𝑜𝑣[𝑋y, 𝑌y] = 𝑎𝑐𝐶𝑂𝑉[𝑋, 𝑌]

Theorem 5.15

IfXandYarerandomvariablessuchthatY=aX +b

𝜌2,3 = {−1 𝑎 < 00 𝑎 = 01 𝑎 > 0

17

§ 5.8 Covariance, Correlation and Independence

Definition 5.7

ThecorrelationofX andY is𝑟2,3 = 𝐸[𝑋𝑌]

Theorem 5.16(a)𝐶𝑜𝑣 𝑋, 𝑌 = 𝑟2,3 − 𝜇2𝜇3

(b)Var 𝑋 + 𝑌 = 𝑉𝑎𝑟 𝑋 + 𝑉𝑎𝑟 𝑌 + 2𝐶𝑜𝑣[𝑋, 𝑌]

Therelationbetweencovarianceandcorrelation

Therelationbetweenvarianceandcovariance

(c)IfX=Y,Cov 𝑋, 𝑌 = 𝑉𝑎𝑟 𝑋 = 𝑉𝑎𝑟 𝑌 ,and𝑟2,3 = 𝐸 𝑋i = 𝐸[𝑌i]

18

§ 5.8 Covariance, Correlation and Independence

Definition 5.8 Orthogonal Random Variables

RandomvariablesX andY areorthogonal if𝑟2,3 = 0

Definition 5.9 Uncorrelated Random Variables

RandomvariablesX andY areuncorrelated if𝐶𝑜𝑣[𝑋, 𝑌] = 0

19

§ 5.8 Covariance, Correlation and Independence

Theorem 5.17

(a)E 𝑔 𝑋 ℎ(𝑌) = 𝐸 𝑔 𝑋 𝐸[ℎ(𝑌)]

(b)𝑟2,3 = 𝐸 𝑋𝑌 = 𝐸 𝑋 𝐸[𝑌]

(c)Cov 𝑋, 𝑌 = 𝜌2,3 = 0

Forindependent randomvariablesXandY,

(c)Var 𝑋, 𝑌 = 𝑉𝑎𝑟 𝑋 + 𝑉𝑎𝑟[𝑌]

20

§ 5.9 Bivariate Gaussian Random Variables

Definition5.10BivariateGaussianRandomVariables

RandomvariableX and YhaveabivariateGaussianPDFwithparameters,,andsatisfyingif

fX,Y (x, y) =

exp

x −µXσ X

⎛

⎝⎜

⎞

⎠⎟2

−2ρX,Y (x −µX )(y−µY )

σ XσY+

y−µYσY

⎛

⎝⎜

⎞

⎠⎟2

2 1− ρX,Y2( )

⎡

⎣

⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥

2πσ XσY 1− ρX,Y2

µX µY σ X > 0σY > 0 ρX,Y −1< ρX,Y <1

21

§ 5.9 Bivariate Gaussian Random Variables

Theorem5.18

IfX and Y arethebivariateGaussianrandomvariableindefinition5.10,XistheGaussian(,)randomvariableandYistheGaussian(,)randomvariable:

fX (x) =1

σ X 2πexp

− x −µX( )2

2σ X2

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟

µX µYσ X σY

fY (y) =1

σY 2πexp

− y−µY( )2

2σY2

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟

Theorem5.19

BivariateGaussianrandomvariableX and Y indefinition5.10,havecorrelationcoefficientρX,Y

Theorem5.20

BivariateGaussianrandomvariableX and Yareuncorrelatedifandonlyiftheyareindependent

22

§ 5.9 Bivariate Gaussian Random Variables

Theorem5.21

IfX and Y arethebivariateGaussianrandomvariableindefinition5.10,andW1 andW2aregivenbythelinearlyindependentequations

W1 = a1X + b1Y

ThenW1 andW2 arebivariateGaussianrandomvariablesuchthat

W2 = a2X + b2Y

E[Wi ]= aiµX + biµY

Var[Wi ]= ai2σ X

2 + bi2σY

2 + 2aibiρX,Yσ XσY

Cov[W1,W2 ]= a1a2σ X2 + b1b2σY

2 + (a1b2 + a2b1)ρX,Yσ XσY

23

§ 5.10 Multivariate Probability Models

Definition5.11MultivariateJointCDFThejointCDF ofX1,X2,… Xn is

FX1,...,Xn(x1, x2,..., xn ) = P X1 ≤ x1,X2 ≤ x2,...,XN ≤ xn[ ]

Definition5.12MultivariateJointPMF

ThejointPMF ofthediscreterandomvariableX1,X2,… Xn is

PX1,...,Xn(x1, x2,..., xn ) = P X1 = x1,X2 = x2,...,XN = xn[ ]

Definition5.12MultivariateJointPDF

fX1,...,Xn(x1, x2,..., xn ) =

∂nFX1...Xn(x1, x2,..., xn )

∂x1∂x2...∂xn

ThejointPDF ofthecontinuousrandomvariableX1,X2,… Xn is

24

§ 5.10 Multivariate Probability Models

Theorem5.22IfX1,X2,… Xn arediscreterandomvariableswithjointPMF

PX1,...,Xn(x1, x2,..., xn ) ≥ 0

PX1,...,Xn(x1, x2,..., xn )

... PX1,...,Xn(x1, x2,..., xn ) =1

xn∈SXn

∑x1∈SX1

∑

Theorem5.23

IfX1,X2,… Xn arediscreterandomvariableswithjointPMF fX1,...,Xn(x1, x2,..., xn )

fX1,...,Xn(x1, x2,..., xn ) ≥ 0

...−∞

∞

∫ fX1,...,Xn(x1, x2,..., xn )dx1...dxn =1

−∞

∞

∫

FX1,...,Xn(x1,..., xn ) = ...

−∞

∞

∫ fX1,...,Xn(u1,u2,...,un )du1...dun

−∞

∞

∫

25

§ 5.10 Multivariate Probability Models

Theorem5.24TheprobabilityofaneventAexpressedintermsoftherandomvariablesX1,X2,… Xn

P[A]= PX1,...,Xn(x1, x2,..., xn )

(x1,...,xn )∈A∑

P[A]= ...A∫∫∫ fX1,...,Xn

(x1, x2,..., xn )dx1...dxn

Discrete

Continuous