moment generating functions
DESCRIPTION
Moment Generating Functions. Contents. Review of Continuous Distribution Functions. Continuous Distributions. The Uniform distribution from a to b. The Normal distribution (mean m , standard deviation s ). The Exponential distribution. The Gamma distribution. - PowerPoint PPT PresentationTRANSCRIPT
Moment Generating Functions
1/33
Contents
Review of Continuous Distribution Functions
2/33
0
0.1
0.2
0.3
0.4
0 5 10 15
1
b a
a b
f x
x0
0.1
0.2
0.3
0.4
0 5 10 15
1
b a
a b
f x
x
1
b a
a b
f x
x
Continuous Distributions
The Uniform distribution from a to b
1
0 otherwise
a x bf x b a
The Normal distribution (mean m, standard deviation s)
2
221
2
x
f x e
0
0.1
0.2
-2 0 2 4 6 8 10
The Exponential distribution
0
0 0
xe xf x
x
The Gamma distribution
Let the continuous random variable X have density function:
1 0
0 0
xx e xf x
x
Then X is said to have a Gamma distribution with parameters a and l.
Moment Generating function of a Random Variable X
if is discrete
if is continuous
tx
xtX
Xtx
e p x X
m t E ee f x dx X
Moment Generating function of a Random Variable X
Examples
1. The Binomial distribution (parameters p, n)
tX txX
x
m t E e e p x
0
1n
n xtx x
x
ne p p
x
0 0
1n nx n xt x n x
x x
n ne p p a b
x x
1nn ta b e p p
0,1,2,!
x
p x e xx
The moment generating function of X , mX(t) is:
2. The Poisson distribution (parameter l)
tX txX
x
m t E e e p x 0 !
xntx
x
e ex
0 0
using ! !
t
xt xe u
x x
e ue e e e
x x
1te
e
Moment Generating function of a Random Variable X
0
0 0
xe xf x
x
The moment generating function of X , mX(t) is:
3. The Exponential distribution (parameter l)
0
tX tx tx xXm t E e e f x dx e e dx
0 0
t xt x e
e dxt
undefined
tt
t
Moment Generating function of a Random Variable X
2
21
2
x
f x e
The moment generating function of X , mX(t) is:
4. The Standard Normal distribution (m = 0, s = 1)
tX txXm t E e e f x dx
2 22
1
2
x tx
e dx
2
21
2
xtxe e dx
Moment Generating function of a Random Variable X
2 2 2 22 2
2 2 21 1
2 2
x tx t x tx t
Xm t e dx e e dx
We will now use the fact that
222
11 for all 0,
2
x b
ae dx a ba
We have
completed the
square
22 2
2 2 21
2
x tt t
e e dx e
This is 1
Moment Generating function of a Random Variable X
1 0
0 0
xx e xf x
x
The moment generating function of X , mX(t) is:
4. The Gamma distribution (parameters a, l)
tX txXm t E e e f x dx
1
0
tx xe x e dx
1
0
t xx e dx
Moment Generating function of a Random Variable X
We use the fact
1
0
1 for all 0, 0a
a bxbx e dx a b
a
1
0
t xXm t x e dx
1
0
t xtx e dx
tt
Equal to 1
Moment Generating function of a Random Variable X
Properties of Moment Generating Functions
1. mX(0) = 1 0, hence 0 1 1tX X
X Xm t E e m E e E
v) Gamma Dist'n Xm tt
2
2iv) Std Normal Dist'n t
Xm t e
iii) Exponential Dist'n Xm tt
1ii) Poisson Dist'n
te
Xm t e
i) Binomial Dist'n 1nt
Xm t e p p
Note: the moment generating functions of the following distributions satisfy the property mX(0) =
1
2 33212. 1
2! 3! !kk
Xm t t t t tk
We use the expansion of the exponential function:2 3
12! 3! !
ku u u u
e uk
tXXm t E e
2 32 31
2! 3! !
kkt t t
E tX X X Xk
2 3
2 312! 3! !
kkt t t
tE X E X E X E Xk
2 3
1 2 312! 3! !
k
k
t t tt
k
Properties of Moment Generating Functions
0
3. 0k
kX X kk
t
dm m t
dt
Now
2 33211
2! 3! !kk
Xm t t t t tk
2 1321 2 3
2! 3! !kk
Xm t t t ktk
2 13
1 2 2! 1 !kkt t t
k
Properties of Moment Generating Functions
1and 0Xm
24
2 3 2! 2 !kk
Xm t t t tk
2and 0Xm
continuing we find 0kX km
Properties ofMoment Generating Functions
i) Binomial Dist'n 1nt
Xm t e p p
Property 3 is very useful in determining the moments of a random variable X.
Examples
11
nt tXm t n e p p pe
10 010 1
n
Xm n e p p pe np
Properties ofMoment Generating Functions
2 11 1 1
n nt t t t tXm t np n e p p e p e e p p e
2
2
1 1 1
1 1
nt t t t
nt t t
npe e p p n e p e p p
npe e p p ne p p
2 221np np p np np q n p npq
Properties of Moment Generating Functions
1ii) Poisson Dist'n
te
Xm t e
1 1t te e ttXm t e e e
1 1 2 121t t te t e t e tt
Xm t e e e e
1 2 12 2 1t te t e tt t
Xm t e e e e
1 2 12 3t te t e tte e e
1 3 1 2 13 23t t te t e t e t
e e e
Properties of Moment Generating Functions
0 1 0
1 0e
Xm e
0 01 0 1 02 22 0
e e
Xm e e
3 0 2 0 0 3 23 0 3 3t
Xm e e e
To find the moments we set t = 0.
Properties of Moment Generating Functions
iii) Exponential Dist'n Xm tt
1
X
d tdm t
dt t dt
2 21 1t t
Properties of Moment Generating Functions
3 32 1 2Xm t t t
4 42 3 1 2 3Xm t t t
5 54 2 3 4 1 4!Xm t t t
1
!kk
Xm t k t
Properties of Moment Generating Functions
Thus
2
1
10Xm
3
2 2
20 2Xm
1 !
0 !kk
k X k
km k
Properties of Moment Generating Functions
2 33211
2! 3! !kk
Xm t t t t tk
The moments for the exponential distribution can be calculated in an alternative way. This is note by expanding mX(t)
in powers of t and equating the coefficients of tk to the coefficients in:
2 31 11
11Xm t u u u
tt u
2 3
2 31
t t t
Properties of Moment Generating Functions
Equating the coefficients of tk we get:
1 ! or
!k
kk k
k
k
Properties of Moment Generating Functions
2
2t
Xm t e
The moments for the standard normal distribution
We use the expansion of eu.2 3
0
1! 2! 3! !
k ku
k
u u u ue u
k k
2 2 2
22
2
2 3
2 2 2
212! 3! !
t
kt t t
tXm t e
k
2 4 6 212 2 3
1 1 11
2 2! 2 3! 2 !k
kt t t t
k
The moments for the standard normal distribution
We now equate the coefficients tk in:
2 222
112! ! 2 !
k kk kXm t t t t t
k k
Properties of Moment Generating Functions
If k is odd: mk = 0.
2 1
2 ! 2 !k
kk k
For even 2k:
2
2 !or
2 !k k
k
k
1 2 3 4 2
2! 4!Thus 0, 1, 0, 3
2 2 2!
The log of Moment Generating Functions
Let lX (t) = ln mX(t) = the log of the moment generating function
Then 0 ln 0 ln1 0X Xl m
1 X
X XX X
m tl t m t
m t m t
1
0 0
0X
XX
ml
m
The log of Moment Generating Functions
2
2 X X X
X
X
m t m t m tl t
m t
2
2 22 12
0 0 0 0
0
X X X
X
X
m m ml
m
The log of Moment Generating Functions
Thus lX (t) = ln mX(t) is very useful for calculating the mean and variance of a random variable
1. 0Xl
2 2. 0Xl
The log of Moment Generating Functions
Examples
1. The Binomial distribution (parameters p, n)
1n nt t
Xm t e p p e p q
ln ln tX Xl t m t n e p q
1 tX t
l t n e pe p q
10Xl n p np
p q
The log of Moment Generating Functions
2
t t t t
Xt
e p e p q e p e pl t n
e p q
220X
p p q p pl n npq
p q
2. The Poisson distribution (parameter l)
1te
Xm t e
ln 1tX Xl t m t e
tXl t e
tXl t e
0Xl
2 0Xl
The log of Moment Generating Functions
3. The Exponential distribution (parameter l)
undefined
X
tm t t
t
ln ln ln if X Xl t m t t t
The log of Moment Generating Functions
11Xl t t
t
2
2
11 1Xl t t
t
22
1 1Thus 0 and 0X Xl l
The log of Moment Generating Functions
4. The Standard Normal distribution (m = 0, s = 1)
2
2t
Xm t e
2
2ln tX Xl t m t
, 1X Xl t t l t
2Thus 0 0 and 0 1X Xl l
The log of Moment Generating Functions
Summary of Discrete Distributions
Name
probability function p(x)
Mean
Variance
Moment generating
function MX(t)
Discrete Uniform p(x) =
1N x=1,2,...,N
N+12
N2-112
et
N etN-1et-1
Bernoulli p(x) =
p x=1q x=0
p pq q + pet
Binomial p(x) =
N
x pxqN-x Np Npq (q + pet)N
Geometric p(x) =pqx-1 x=1,2,... 1p
qp2
pet
1-qet
Negative Binomial p(x) =
x-1
k-1 pkqx-k
x=k,k+1,...
kp
kqp2
pet
1-qet k
Poisson p(x) =
x
x! e- x=1,2,... e(et-1)
Hypergeometric
p(x) =
A
x
N-A
n-x
N
n
n A
N n A
N
1-AN
N-n
N-1 not useful
Summary
Summary of Continuous Distributions
Name
probability density function f(x)
Mean
Variance
Moment generating function MX(t)
Continuous Uniform
otherwise
bxaabxf
0
1)(
a+b2
(b-a)2
12 ebt-eat
[b-a]t
Exponential
00
0)(
x
xlexf
lx
1
12
t
for t <
Gamma
f(x) =
00
0)( f(x)
1
x
xexaG
l lxaa
2
t
for t <
2
d.f.
f(x) = (1/2)
(/2) x e-(1/2)x x ? 0
0 x < 0
1
1-2t /2
for t < 1/2
Normal f(x) =
1
2 e-(x-)2/22
2 et+(1/2)t22
Weibull
f(x) = x e-x x ? 0
0 x < 0
( )+1
( )+2 -[ ]( )+1
not avail.
Summary
Expectation of functions of Random Variables
i ix i
E g X g x p x g x p x
E g X g x f x dx
X is discrete
X is continuous
Moments of Random Variables
kk E X
-
if is discrete
if is continuous
k
x
k
x p x X
x f x dx X
The kth
moment of X
Moments of Random Variables
The 1th
moment of X
)(
)(
)(1
1
11
XfX
Xpx
XEX
The kth central moment of X
0 k
k E X
-
if is discrete
if is continuous
k
x
k
x p x X
x f x dx X
where m = m1 = E(X) = the first moment of X .
Moments of Random Variables
Rules for expectation
Rules:
1. where is a constantE c c c
2. where , are constantsE aX b aE X b a b
2023. var X E X
22 22 1E X E X
24. var varaX b a X