X

N

i ii=1

Continuouxf (x)dxE X = X =

x P(x ) Discre

s

t e

nX

nNn

ni i

i=1

Continx f (x)dxm = E X =

x P( Discrx ) et

u

e

ous

Operations on Multiple Random Variables

Previously we discussed operations on one Random Variable:

nX

nNn

n ni i

i=1

Con(x X) f (x)dxμ = E (X X) =

(x X) P(x ) Discret

tinu u

e

o s

Expected Value

Moment

Central moment

jωX jωxX XΦ (ω) = E e = f (x)e dx

jωx

X X

1f (x) = Φ (ω)e dω

2π

nn X

n n

ω = 0

d Φ (ω)m = ( j)

dω

11

Y X

dT (y)f (y) = f T (y)

dy

n

X nY

n

x = x

f (x )f (y) =

dT(x)dx

Characteristic Function

Moment Generation

Function of Random Variable

Monotone Transformation Nonmonotone Transformation

X,Y

i k X,Y i ki k

Cong(x,y)f (x,y)dxdyg = E g(X,Y) =

g(x ,y )P (x ,

tin

y Disc

uou

r

s

t) e e

1 n1 n 1 n X ,…,X 1 n 1 ng = E g(X ,…,X ) = g(x ,…,x )f (x ,…,x )dx dx

EXPECTED VALUE OF A FUNCTION OF Multiple RANDOM

VARIABLES

Two Random Variables

N Random Variables

EX 5.1-1

Joint Moment about the Origin

n k n knk X,Ym = E X Y = x y f (x,y)dxdy

nn0m = E[X ] the nth moment mn of the one random variable X

k0km = E[Y ] the kth moment mk of the one random variable Y

n + k is called the order of the moments

Thus m02 , m20 and m11 are all 2ed order moments of X and Y

The first order momentsare the expected values of X and Y and are the coordinates of the center of gravity of the function fXY(x,y)

10 01m = E[X] X and m = E[Y] Y

The second-order moment m11 = E[XY] is called the correlation of X and Y and given the symbol RXY hence

1XY X,Y1 = m = E[XY] = xyf (x,y)dxdR y

If the correlation can be written as XY = E[R X]E[Y]

Then the random variables X and Y are said to be uncorrelated.

Statistically independence of X and Y → X and Y are uncorrelated

The converse is not true in general

X,Y X Yf (x,y) = f (x)f (y)

XY X,Y11 = m = E[XY] = xyf (x,y)d dyR x

X Yxyf (x)f (y)dxdy

X YXY xf (x)dx yf (y)dyR

= E[X]E[Y]

For N random variables X1, X2, …,XN the (n1 + n2 + … nN) order joint moments are defined by

N1 2N1 2

N1

N1

n n nn n ...n N1 2

n nN X ,...,X N N1 1 1

m = E X X ...X

= ... X ...X f (x ,...,x )dx ...dx

where n1,n2 ,… nN are all integers 0, 1, 2,

Uncorrelated of X and Y does not imply that X and Y are Statistically independent in general, except for the Gaussian random variables as will be shown later

If RXY = 0 then the random variables X and Y are called orthogonal.

EX 5.1-2

Joint Central Moments

We define the joint central moments for two random variables X and Y as follows

n knkμ = E (X X) (Y Y)

n k

X,Y= (x X) (y Y) f (x,y)dxdy

The second-order ( n + k =2) central moments are

2 2X20μ = E (X X) = σ

The variance of X

2 2Y02μ = E (Y Y) = σ

The variance of Y

The second-order joint moment µ11 is very important. It is called the covariance of X and Y and is given the symbol CXY

1XY 1 = μ = E (X X)(YC Y)

The second-order joint moment µ11 is very important. It is called the covariance of X and Y and is given the symbol CXY

X,Y= (x X)(y Y)f (x,y)dxdy

By direct expansion of the product (X X)(Y Y) we get

XY 11C = μ = E (XY XY XY XY)

E[XY] E[X]Y XE[Y] XY = E[XY] XY XY XY

X XY XYY = XY= E[XR ] ]R [C E Y

Note on the 1-D , the variance was defined as2

2X = μ = E[(X X) ]

Which can be written as

2X = μ = E[(X X)(X X)]

2 2= E[X ] X

= E[XX] XX

XY

0 if X and Y are independents

E[X]E[Y] if X and Y are orthogo l C

na

XY = 0 if X=0 or Y=0C

X YXY Y X = R XY= R E[X] YC E[ ]

The normalized second order-moments defined as XY

X Y

Cρ = σ σ

is known as the correlation coefficient of X and Y

It can be shown that 1 ≤ ≤ 1

For N random variables X1, X2, …,XN the (n1 + n2 + … nN) order joint central moments are defined by

N1 2N1 2

n n nn n ...n N N1 1 2 2μ = E (X X ) (X X ) ... (X X )

N1

N1

n nN N X ,...,X N N1 1 1 1= ... (x X ) ...(x X ) f (x ,..., x )dx ... dx

,

2

Random variables and have the joint density

10 < x < 6 and 0 < y <4

( , ) 240

What is the expected value of the function g( , )=( ) ?

X Y

X Y

f x yelsewhere

X Y XY

4 62 2 2 2

,

0 0

1 E[( ) ] ( ) ( , ) = 64

24

y x

X Y

y x

XY xy f x y dxdy x y dxdy

Solution

Problem 5.1-1

4( ),

Random variables and have the joint density

( , ) ( ) ( )16

Find the mean value (Expected) of the function g( , )

1 15 0 < x and 0 < y

2 21

g( , ) 1 < x a2

x yX Y

X Y

f x y u x u y e

X Y

X Y

1

nd/or < y 2

0 all other x and y

Problem 5.1-3

1 15 0 < x and 0 < y

2 21 1

g( , ) 1 < x and/or < y 2 2

0 all other x and y

X Y

,

1 1 1

2 2 24( ) 4( )

10 0 02

E[g( , )] g( , ) ( , )

= 5 16 ( 1) 16 3.486

X Y

y x xy

x y x y

y x xy

X Y x y f x y dxdy

e dxdy e dxdy

Solution

4( ), ( , ) ( ) ( )16 x y

X Yf x y u x u y e

1 15 0 < x and 0 < y

2 21 1

g( , ) 1 < x and/or < y 2 2

0 all other x and y

X Y

,

E[g( , )] g( , ) ( , )X YX Y x y f x y dxdy

Solution

4( ), ( , ) ( ) ( )16 x y

X Yf x y u x u y e

1 1

2 24( )

0 0

= 5 16

y x

x y

y x

e dxdy

1

24( )

1 02

( 1) 16

xy

x y

xy

e dxdy

, ( , ) 0X Yf x y

4( )

102

( 1) 16y x

x y

y x

e dxdy

, ( , ) 0X Yf x y

3.486

, ( , ) 0X Yf x y

2

,

Two Random variables and have the joint density

2( 0.5 ) 0 < x < 2 and 0 < y <3

( , ) 430

( ) Find all the First and second order moments ?

X Y

X Y

x yf x y

elsewhere

a

( ) Find the covariance ?b

( ) Are and uncorrelated ?c X Y

Problem 5.1-19

jωX jωx jωxX X X X

1Φ (ω) = E e = f (x)e dx f (x) = Φ (ω)e dω

2π

nn X

n n

ω = 0

d Φ (ω)m = ( j)

dω

JOINT CHARACTERISTIC FUNTIONS

X() is the Fourier transform of fX(x) with the sign of is reverse

Let u first review the characteristic function for the single random variable

Let X be a random variable with probability density function fX(x)

We defined the characteristic function X() as follows

The moments mn can be found from the characteristic function as follows:

We now define the joint characteristic function of two random variables X and Y with joint probability density function fXY(x,y) as follows

1 2jω X + jω YX,Y 1 2Φ (ω ,ω ) = E e

1 2jω x + jω yX,Y 1 2 X,YΦ (ω ,ω ) = f (x,y) e dxdy

1 2jω x jω yX,Y X,Y 1 2 1 22

1f (x,y) = Φ (ω ,ω ) e dω dω

(2π)

were and are real numbers

X,Y(1,2) is the two-dimensional Fourier Transform of fXY(x,y) with reversal of sign of and

From the inverse two-dimensional Fourier Transform we have

By setting = 0 in the expression of X,Y() above we obtain the following

1jω x + j(0)yX,Y X,Y1Φ (ω ,0) = f (x,y) e dxdy

1 jω x

X,Y= f (x,y) e dxdy

1 jω x

X,Y= f (x,y)dy e dx

1 jω x

X= f (x) e dx

X 1= Φ (ω ) The characteristic function for the marginal probability density function fX(x)

Similarly

Y X,Y2 2Φ (ω ) = Φ (0,ω )

The joint moments can be found from the joint characteristic function

1 2

n+kX,Y 1 2n+k

nk n k1 2 ω =0, ω 0

Φ (ω ,ω )m = ( j)

ω ω

This expression is the two-dimensional extension of the one-dimension

nn X

n n

ω = 0

d Φ (ω)m = ( j)

dω

The joint characteristic function for N random variables 1 nX ,…,X

1 1 N N

1 N

jω x + ... + jω xX ,...,X 1 NΦ (ω ,...,ω ) = E e

The joint moments are obtained from

N

1 2 N N1 2

i

RX1,....X 1 NR

n n ...n nn n1 2 N all ω =0

(ω ,...,ω )m = ( j)

ω ω ... ω

1 2 Nwere R = n n n

The Gaussian Random Variable

2 ( varinace) ( mean)X Xwhere the and thea22 2

2

( )1( )

2X xa

x

xXf ex

The “spread” about the point x = ax is related to σx

A random variable X is called Gaussian if its density function has the form

JOINTLY GAUSSINA RANDOM VARIABLES

Two random variables X and Y are said to be jointly gaussian orBivariate gaussian density if their joint density function is of the form

X,Y

Y

2Y Y

X

2

2X X

2

2

ρ

ρ

ρ

xσ

(x X)

1 f ( , ) =

2π 1

1 2 (x X)

yσ

(y Y) (y Y exp +

2(1 σ σ σ σ

)

)

X = E X Y = E Y2 2Xσ = E (X X)

2 2Yσ = E (Y Y)

X Y(Y = (X X)E σY) σρ

were

X,Y X,Y 2X Y

1f (x,y) f (X,Y) =

2πσ σ 1 ρ

The joint Gaussian density function and its maximum is located at the point (X,Y)

The maximum value is obtained from

The locus of constant values of will be an ellipse

X,Yf (x,y)This is equivalent to the intersection of with the plane parallel to the xy-plane

X,Yf (x,y)

If then the joint Gaussian density ρ = 0

X,Y

Y

2Y Y

X

2

2X X

2

2

ρ

ρ

ρ

xσ

(x X)

1 f ( , ) =

2π 1

1 2 (x X)

yσ

(y Y) (y Y exp +

2(1 σ σ σ σ

)

)

X,Y 2Y Y

2

2X X

(x 1 1 f ( , ) = exp +

(y Y)y

σ σ

2π

X)x

σ σ2

YX

2

2

2

22XX

f (x

2YY

f (y))

1 1= exp

(y exp

(x X

2π 2π

Y)

2σσ

)

2σσ

YX= f( ) f x (y)

1 2 N 1 2 N

1 1Y ,Y ,...,Y 1 N X ,X ,...,X 1 1 N Nf (y ,...,y ) f (x = T ,...,x = T ) J

1 11 1

1 N

1 1N N

1 N

T T

Y Y

J =

T T

Y Y

11

Y X

dT (y)f (y) = f T (y)

dy

n

X nY

n

x = x

f (x )f (y) =

dT(x)dx

i i 1 2 NY = T (X ,X ,...,X ) i = 1,2, ..., N1

j j 1 2 NX = T (Y ,Y ,...,Y ) j = 1,2, ..., N

,

,

u h x y x y

v g x y x y

1

1

1, ( )

21

, ( )2

x h u v u v

v g u v u v

Example

, ,( , ) ( , )X Y U V

A B

f x y dxdy f u v dudv

1 1, ,

Comparing it with the Right side

( , ) , , , | ( , ) |U V X Yf u v f h u v g u v J x y

1 1, ,

1 1

1 1

( , ) , , , | ( , ) |

were

, ,

( , ) is the Jacobian, ,

| ( , ) | is magnitude absolute v

X Y X Y

A B

f x y dxdy f h u v g u v J x y dudv

h u v h u vx x

u v u vJ x yy y g u v g u vu v u v

J x y

Left side

alue of the determinant of the Jacobian

1 1, , ( , ) , , , | ( , ) |U V X Yf u v f h u v g u v J x y

,

,

u h x y x y

v g x y x y

1

1

1, ( )

21

, ( )2

x h u v u v

v g u v u v

Example

1 1

1 1

, ,

( , ) , ,

h u v h u vx x

u v u vJ x yy y g u v g u vu v u v

2 2

2 2

u v u v

u vu v u v

u v

1 1

2 2 1 1

2 2

1( , )

2J x y

Example Let X and Y are two zero mean Gaussian independent random variables

22 2

2

1( )

2

xXf x e

22 2

2

1( )

2

yYf y e

2 2 1 Rectangular to Polar Let =tan

Transformation

YR X Y

X

Find ( , ) ?Rf r

Solution Since X and Y are independent

( , ) ( ) ( )X X Yf x y f x f y 2 22( 2)2

1 2

x ye

Assuming 0 and 0 2r

2 2 1The Transformation =tany

r x yx

1 1

has the inverse Transformation

Polar to Rectangular ( , ) cos( ) ( , ) sin( )

Transformationx h r r y g r r

1 1, , ( , ) , , , | ( , ) |R X Yf r f h r g r J x y

1 1, ,( , ) , , , | ( , ) |R X Yf r f h r g r J x y

( cos ) ( cos )( , )

( sin ) ( sin )

x xr r

r rJ x yy y

r rr r

cos sin( )

sin( ) cos

r

r

cos sin( ) | ( , ) |

sin( ) cos

rJ x y r

r

1 1, ,( , ) , , , | ( , ) |R X Yf r f h r g r J x y

1( , ) cos( )x h r r

1( , ) sin( )y g r r

r

1 1, ,( , ) , , , | ( , ) |R X Yf r f h r g r J x y

1( , ) cos( )x h r r

1( , ) sin( )y g r r

r

, ( , )Rf r

2 22( cos ) ( n ) 2si

, 2

1( , )

2

r r

Rf r e r

2 22

2 2

rre

2 2

2,2( , )

2r

R

rf r e

, ( , )Rf r

22 21

2rre

Uniform density between 0,2

Rayleigh density function

Let and be independent uniform random variables over(0,1).

Find the pdf of , ( ) ?Z

X Y

Z XY f z

We seek the density of , ( ) ?ZZ f zSolution

If we can find a joint density between and another random variable say , then we can

find the marginal density ( ) by integrating the joint density ( , ) with respect to Z ZW

Z W

f z f z w w

( ) ( , )Z ZWf z f z w dw

How can we find the joint density ( , ) ?ZWf z wQ :

1 1 ( , ) , , , ( , )ZW XYf z w f x h z w y g z w J x y A : Using the Jacobian method

Since  and are independent then ( , ) ( ) ( ) 

1 0 1 , 0 1 ( , )

0 otherwise

XY X Y

XY

X Y f x y f x f y

x yf x y

Let and Z XY W X

The transformation , has the inverse transformation /z xy w x x w y z w

2

0 1( , ) 1

x x

z wJ x y zy y

w wz w

1( , )J x y

w

1 ( , ) ,ZW XY

zf z w f x w y

w w

1 0 1 , 0 1( , )

0 otherwiseXY

zwz

f w ww

1 0 1

0 otherwise

z w

10 1

( , )0 otherwise

ZW

z wf z w w

( ) ( , )Z ZWf z f z w dw

1 1 ( , ) , , , ( , )ZW XYf z w f x h z w y g z w J x y

1 0 1 , 0 1( , )

0 otherwiseXY

x yf x y

Z XY

we seek another

transformation W

( ) ( , )Z ZWf z f z w dw

1 1

ln 0 1z

dw z zw