PROBABILITY AND STATISTICS FOR ENGINEERING

Hossein Sameti

Department of Computer EngineeringSharif University of Technology

Two Random Variables

Introduction

Expressing observations using more than one quantity- height and weight of each person

- number of people and the total income in a family

Joint Probability Distribution Function

Let X and Y denote two RVs based on a probability model (, F, P). Then:

What about

So, Joint Probability Distribution Function of X and Y is defined to be

2

1

,)()()()( 1221

x

x XXX dxxfxFxFxXxP

.)()()()(2

11221

y

y YYY dyyfyFyFyYyP

?))(())(( 2121 yYyxXxP

,0),(

))(())((),(

yYxXP

yYxXPyxFXY

Properties(1)

1),(

0),(),(

XY

XYXY

F

xFyF

, )()(,)( XyYX

.0)(),( XPyFXY

,)(,)( YX

.1)(),( PFXY

Proof

Since

Since

Properties(2)

Proof

for

Since this is union of ME events,

Proof of second part is similar.

).,(),()(,)( 1221 yxFyxFyYxXxP XYXY

).,(),()(,)( 1221 yxFyxFyYyxXP XYXY

,12 xx yYxXxyYxX

yYxX

)(,)()(,)(

)(,)(

211

2

yYxXxPyYxXP

yYxXP

)(,)()(,)(

)(,)(

211

2

Properties(3)

Proof

So, using property 2,

we come to the desired result.

).,(),(

),(),()(,)(

1121

12222121

yxFyxF

yxFyxFyYyxXxP

XYXY

XYXY

.)(,)()(,)(

)(,)(

2121121

221

yYyxXxyYxXx

yYxXx

1y

2y

1x 2x

X

Y

0R

2121121

221

)(,)()(,)(

)(,)(

yYyxXxPyYxXxP

yYxXxP

Joint Probability Density Function (Joint p.d.f)

By definition, the joint p.d.f of X and Y is given by

Hence,

So, using the first property,

.

),(),(

2

yx

yxFyxf XY

XY

. ),(),(

dudvvufyxF

x y

XYXY

.1 ),(

dxdyyxf XY

Calculating Probability

How to find the probability that (X,Y) belongs to an arbitrary region D?

Using the third property,

.),(),(

),(),( ),(

),()(,)(

yxyxfdudvvuf

yxFyxxFyyxF

yyxxFyyYyxxXxP

XY

xx

x

yy

y XY

XYXYXY

XY

x

X

Y

yD

Dyx XY dxdyyxfDYXP),(

.),(),(

Marginal Statistics

Statistics of each individual ones are called Marginal Statistics.

marginal PDF of X,

the marginal p.d.f of X.

Can be obtained from the joint p.d.f.

.),()(

),()(

dxyxfyf

dyyxfxf

XYY

XYX

)(xFX

)(xf X

Proof

)()()( YxXxX

).,(,)( xFYxXPxXPxF XYX

Marginal Statistics

Using the formula for differentiation under integrals, taking derivative with respect to x we get:

dudyyufxFxFx

XYXYX ),(),()(

.),()(

dyyxfxf XYX

.),()(

),()(

dxyxfyf

dyyxfxf

XYY

XYX

Proof

Let

Then:

.),()()(

)( xb

xadyyxhxH

( )

( )

( ) ( ) ( ) ( , )( , ) ( , ) .

b x

a x

dH x db x da x h x yh x b h x a dy

dx dx dx x

Differentiation Under Integrals

Marginal p.d.fs for Discrete r.vs

j j

ijjii pyYxXPxXP ),()(

i i

ijjij pyYxXPyYP ),()(

X and Y are discrete r.vs Joint p.d.f:

Marginal p.d.fs:

When written in a tabular fashion,

to obtain one needs to add up all

entries in the i-th row. This suggests the name marginal densities.

),( jiij yYxXPp

mnmjmm

inijii

nj

nj

pppp

pppp

pppp

pppp

21

21

222221

111211

j

ijp

i

ijp

),( ixXP

Example

Given marginals, it may not be possible to compute the joint p.d.f.

Obtain the marginal p.d.fs and for:

. otherwise0,

,10constant,),(

yxyxf XY

Example

)(xf X)(yfY

Example

is constant in the shaded region We have: So:

Thus c = 2. Moreover,

Similarly,

Clearly, in this case given and , it will not be possible to obtain the original joint p.d.f in

Solution

0 1

1

X

Y

y

),( yxf XY

.122

),(1

0

21

0

1

0

0

ccycydydydxcdxdyyxf

yy

y

xXY

1 ),(

dxdyyxf XY

,10 ),1(22),()(1

xyXYX xxdydyyxfxf

.10 ,22),()(

0

y

xXYY yydxdxyxfyf

)(xf X)( yfY

Example

X and Y are said to be jointly normal (Gaussian) distributed, if their joint p.d.f has the following form:

By direct integration,

So the above distribution is denoted by

.1|| , ,

,12

1),(

2

2

2

2

2

)(

))((2

)(

)1(2

1

2

yx

eyxf Y

Y

YX

YX

X

X yyxx

YX

XY

Example

),,( 2

1),()( 22/)(

2

22

XXx

X

XYX Nedyyxfxf XX

),,( 2

1),()( 22/)(

2

22

YYy

Y

XYY Nedxyxfyf YY

).,,,,( 22 YXYXN

Example - continued

So, once again, knowing the marginals alone doesn’t tell us everything about the joint p.d.f

We will show that the only situation where the marginal p.d.fs can be used to recover the joint p.d.f is when the random variables are statistically independent.

Independence of r.vs

The random variables X and Y are said to be statistically independent if the events and are independent events for any two Borel sets A and B in x and y axes respectively.

For the events and if the r.vs X and Y are independent, then

i.e.,

or equivalently:

If X and Y are discrete-type r.vs then their independence implies

AX )( })({ BY

Definition

xX )( , )( yY

))(())(())(())(( yYPxXPyYxXP

)()(),( yFxFyxF YXXY

).()(),( yfxfyxf YXXY

., allfor )()(),( jiyYPxXPyYxXP jiji

Independence of r.vs

Given

obtain the marginal p.d.fs and

examine whether independence condition for discrete/continuous type r.vs are satisfied.

Example: Two jointly Gaussian r.vs as in (7-23) are independent if and only if the fifth parameter

Procedure to test for independence

)(xf X)( yfY

),,( yxf XY

.0

Example

Given

Determine whether X and Y are independent.

Similarly

In this case

and hence X and Y are independent random variables.

Solution

otherwise.,0

,10 ,0,),(

2

xyexy

yxfy

XY

.10 ,2 22

),()(

0 0

0

2

0

xxdyyeyex

dyeyxdyyxfxf

yy

yXYX

.0 ,2

),()(21

0 ye

ydxyxfyf y

XYY

),()(),( yfxfyxf YXXY