Functions of Random VariablesLecture 19-20

Starting November 6

The probability that we may fail in the struggle ought not to deter us from the support of a cause we believe to be just.

Abraham Lincoln

Outline

• Four Methods for Function of Random Variables

1) Discrete case: calculate directly. 2) Method of m.g.f3) Method of c.d.f4) Method of transforms

Functions of Random Variable

You know distribution of X and Y. Want to know distribution of function of X and Y like X+Y, X2, eX+Y, g(X,Y)

Many ways to attach the problem1) Discrete case: calculate directly. 2) Method of m.g.f3) Method of c.d.f4) Method of transforms

Discrete Problems

Given X has pdf,

What is the distribution of U=2XWhat is the distribution of V=(X-3)2

x 1 2 3 4 5

f(x) .1 .3 .3 .2 .1

Discrete Problems

U=2X has pdf,

U 2 4 6 8 10

f(u) .1 .3 .3 .2 .1

Discrete Problems

What is the distribution of V=(X-3)2

What is value set of V? {0,1,4}

x 1 2 3 4 5

V 4 1 0 1 4

Steps

What is Value Set of V

x 1 2 3 4 5

v 4 1 0 1 4

( 0) ( 3) .3( 1) ( 2) ( 4) .3 .2

P V P XP V P X P X

= = = == = = + = = +

Calculate pdf

v 0 1 4

f(v) .3 .5 .2

( 0) ( 3).3( 1) ( 2) ( 4) .3 .2( 4) ( 1) ( 5) .2

P V P XP V P X P XP V P X P X

= = == = = + = = += = = + = =

General Steps

a) Identify value sets of X

and value set of V=g(X)

a) Calculate pdf

{1, 2, 3, 4, 5}S =

' {0,1, 4}S =

For each ( ) ( ) ( )

{ | ( ( ) )

V Xx A

f v P V v f x

A x g x v∈

= = =

= =

∑

Works for X continuous and U discrete as well

Let X be uniform continuous RV on interval [0,20]

Let 1 0 2( ) 15 2 8

20 8 20

xU g x x

x

< ≤⎧⎪= = < ≤⎨⎪ < <⎩

General Steps

a) Identify value sets of X

and value set of U=g(X)

a) Calculate pdf

(0, 20)S =

' {0,15, 20}S =

2

0

( 1) (0 2) 1/ 20 1/10P U P X dx= = < ≤ = =∫

Calculate pdf

U 1 15 20

f(u) 1/10 6/20 12/20

MGF Method

Moment generating function can be used to calculate the distribution of sums of independent random variables.

If are independent random variables with mgf then

has mgf

1 2, , , nX X X…( )

iXm t

1

n

ii

W X=

=∑1

( ) ( )i

m

W Xi

m t m t=

=∏

ExampleLet X be the number of heads in ten tosses. Let Y be the number of heads in 20 tosses. What is the distribution of X +Y?

X~binomial (n=10,p=0.5)Y~binomial(n=20,p=0.5)The mgf of binomial is So the mgf of X+Y is

( (1 ))t npe p+ −

10 20 30( ) ( ) ( ) (0.5 0.5) (0.5 0.5) (0.5 0.5)t t tX Y X Ym t m t m t e e e+ = = + + = +

Thus X+Y is binomial n=30 p=0.5

Example

Ten families in a neighborhood have children until they get a girl. What is the distribution of the total number of kids in the ten families?

ExampleThe number of kids in a family is

Xi~geometric (p=0.5)The mgf of a geometric is

The mgf of sum of 10 geometrics is

This is a negative binomial r=10 p=p!

( )1 (1 )

t

t

pem tp e

=− −

10

( )1 (1 )i

t

tX

pem tp e

⎡ ⎤= ⎢ ⎥∑ − −⎣ ⎦

Notes on MGF method

MGF is quite valuable for deriving many important results.

Try the ones on the next page for practice.But it only works in limited cases

Let X and Y be independent

distr X Y X+Y

binomial n1,p n2,p Binomial(n1+n2,p)

Poisson λ1 λ2 Poisson(λ1+λ2 )

Negbinomial

(r1,p) (r2,p) Neg Bin.(r1+r2,p)

Gamma (θ,κ1) (θ,κ2) Gamma(θ,κ1+ κ2))

X+Y frequently not same type of distribution as X and Y

If Xi, i=1..n are independent and identically distributed geometric(p) then the distribution of the sum of Xi’s is negative binomial (n,p)

If Xi, i=1..n are independent and identically distributed exponential(θ) then the distribution of the sum of Xi’s is gamma (θ,n.)

Method of the CDF

• Intuitively• Formally

– 1-1 function

Continuous case

X~uniform(0,2)U=X2

What is value set of U?

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

0.5

1

1.5

2

2.5

3

3.5

4

x

x2

Continuous case

X~uniform(0,2)U=X2

What is value set of U?

What is P(U≤1)? P(U≤1.5)?P(U≤c)?

What is cdf of U?

Continuous case

X~uniform(0,2)U=X2

What is P(U≤1)?P(U ≤3)?

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

0.5

1

1.5

2

2.5

3

3.5

4

x

x2

1

0

1 1( 1) ( 1) (1)2 2X

x

P U P X F dx=

≤ = ≤ = = =∫3

0

1 3( 3) ( 3) ( 3) 0.8862 2X

x

P U P X F dx=

≤ = ≤ = = = =∫

Continuous case

What is cdf of U?

Recall

What is pdf of U?

( ) '( )Uf u F u=

Continuous case

What is cdf of U?Calculate F(c)=P(U≤c)

What is pdf of U?

0

0 0

1( ) ( ) ( ) ( ) 0 42 2

1 4

c

U X

c

cF c P U c P X c F c dx c

c

<=⎧⎪⎪= ≤ = ≤ = = = < <⎨⎪

≥⎪⎩

∫

1/ 2 1/ 2( / 2)( ) '( )4U U

d u uf u F udu

−

= = = 1 0 4( ) 4

0 . .U

uf u u

o w

⎧ < <⎪= ⎨⎪⎩

Method of cdf

Basic idea: If U=g(x), Find cdf of U as a function of cdf of X. Differentiate.

Simplifications possible:

Consider case of 1 to 1 functions.

1 to 1 increasing functionsIf U is 1 to 1 increasing function

u=g(x) →x=g-1(u)=h(u)Then the cdf of U is given by

And the pdf of U is given byby chain rule

No need to explicitly make cdf!

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

0.5

1

1.5

2

2.5

3

3.5

4

x

x2

1( ) ( ( ) ) ( ( )) ( ( ))U XF c P g X u P X g u F h u−= ≤ = ≤ =

( ( )) ( ( ))( ) ( ( ))XU X

dF h u d h uf u f h udu du

= =

1 to 1 increasing

X~uniform(0,2)U=X2

Inverse

pdf0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0

0.5

1

1.5

2

2.5

3

3.5

4

x

x2

1

1/ 2

( ) ( )( ( ))

2

g u h u ud h u u

du

−

−

= =

=

1/ 2 1/ 2( ( )) 1( ) ( ( )) 0 42 2 4U X

d h u u uf u f h u udu

− −

= = = < <

u

1( )g u−

You try

Let Z be a standard normal with mean 0 and std dev. 1.

Find pdf of U=2Z+3, using the method just shown. Is this a 1-1 increasing function?( )( ( ))

( ( ))( ) ( ( ))u Z

h ud h u

dud h uf u f h u

du

=

=

= =

Decreasing function

Let Z be a standard normal with mean 0 and std dev. 1.

Find pdf of U=-2Z+3, using the method just shown. This is a 1-1 decreasing function!

1 to 1 decreasing functions

If U is 1 to 1 decreasing functionu=g(x) →x=g-1(u)=h(u)

Then the cdf of U is given by

And the pdf of U is given by

by chain rule

1( ) ( ( ) ) ( ( )) 1 ( ( ))U XF c P g X u P X g u F h u−= ≤ = ≥ = −

(1 ( ( ))) ( ( ))( ) ( ( ))XU

d F h u d h uf u f h udu du

− ⎡ ⎤= = −⎢ ⎥⎣ ⎦

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

1

2

3

4

5

6

7

x

1/x

In general

Increasing functions

Decreasing functions

So for general 1 to 1 functions, the pdf is( ( )) ( ( ))( ) ( ( ))U

Ud F u d h uf u f h u

du du= =

( ( )) 0d h udu

≥

( ( )) 0d h udu

≤

Example

Let Find pdf ofValue set U in (1,∞)

2( ) 3 0 1Xf x x x= < <3U X −=

1/3

4/3

1/3 2 4/3 2

( )( ( )) 1

3( ( )) 1( ) ( ( )) 3( ) 1

3u

h u ud h u u

dud h uf u f h u u u u u

du

−

−

− − −

=

=−

= = = <

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

100

200

300

400

500

600

700

800

900

x

x-3

Method of Joint Transforms

If X is a vector of continuous R.V.

If U=G(X) is 1 to 1 and the inverse function x=g-1(u)=h(u) is differentiable, then the pdfof U is given by

Where |J(u)| is thedeterminant of the Jacobian

1 2( , , , )nx x x…

( ) ( ( )) ( )U Xf u f h u J u=1 1

1 2

2 2

1 2

| |

x xu u

Jx xu u

∂ ∂∂ ∂

=∂ ∂∂ ∂

Example

Let

Find pdf

( )( , ) 0 0X YXYf x y e X Y− += < < ∞ < < ∞

U X Y= +

• Step 0 – figure out value set 0<U• Step 1: find 1 to 1 transform

U=X+Y V=Y

In general you can use any function (simple as possible) such that U=a(X,Y), V=b(X,Y) is 1 to 1

• Step 2 – Solve for (x,y) in terms of (u,v)U=X+Y V=Y

Y=V X=U-V• Step 3 Calculate the range of U and V0<X<∞ → 0<U-V<∞0<Y<∞→ 0<V<∞

0<V<U< ∞

• Step 4 – Calculate Jacobian of Y=V X=U-V

1 1| | | (1)(1) (0)( 1) | |1|

0 1

xxu v

Jy yu v

∂∂∂ ∂ −

= = = − − =∂ ∂∂ ∂

Step 5 and 6

• Use formula to calculate the joint of U and V

• Calculate marginal of U

( ), ( , ) ( , ) | | (1) 0u v v

U V Xf u v f u v v J e v u− − += − = < < < ∞

( )

0

( ) 0U

u uUf u e dv ue u− −= = < < ∞∫

Problem

The joint density of X1 and X2 are given by

Find a) The joint density of Y=X1+X2 and Z=X1.b) The marginal density of Y

1 21 2

1 0 1,0 1( , )

0x x

f x xotherwise

< < < <⎧= ⎨⎩

Solution

Solving for y and z

The Jacobian is

The new range is

1 2 2

1 2

, , we get, .

y x x z xx y z x z= + == − =

1 11

0 1J

−= =

1 0 1z y z and z< < + < <

continued

The joint pdf of y and z isf(y,z)=1*|1| = 1

To get pdf of z. Integrate out y

1 0 1z y z and z< < + < <

0

1

1

1 0 10 0

1 0 1( )

1 2 1 2

0 2

y

y

z y z a n d zy

d z y yf y

d z y y

f o r y−

< < + < <

≤⎧⎪⎪ = < <⎪⎪= ⎨⎪ = − < <⎪⎪⎪ ≥⎩

∫

∫