moment generating function
DESCRIPTION
MathematicsTRANSCRIPT
EDWIN OKOAMPA BOADU
DEFINITIONDefinition 2.3.3. Let X be a random variable
with cdf FX. The moment generating function (mgf) of X (or FX), denoted MX(t), is
provided that the expectation exists for t in some neighborhood of 0. That is, there is an h>0 such that, for all t in –h<t<h, E[etX] exists.
tXX eEtM
If the expectation does not exist in a neighborhood of 0, we say that the moment generating function does not exist.
More explicitly, the moment generating function can be defined as:
variablesrandom discretefor
and , variablesrandom continuousfor
x
txX
txX
xXPetM
dxxfetM
Theorem 2.3.2: If X has mgf MX(t), then
where we define
0
)( 0
t
Xn
nnX tM
dt
dM
0nnXE X M
First note that etx can be approximated around zero using a Taylor series expansion:
2 30 0 2 0 3 0
2 32 3
1 10 0 0
2 6
12 6
tx t t tXM t E e E e te x t e x t e x
t tE x t E x E x
Note for any moment n:
Thus, as t0
1 2 2n
n n n nX Xn
dM M t E x E x t E x t
dt
nnX xEM 0
Leibnitz’s Rule: If f(x,θ), a(θ), and b(θ) are differentiable with respect to θ, then
b
a
b
a
dxxf
bd
dafa
d
dbfdxxf
d
d
,
,,,
Berger and Casella proof: Assume that we can differentiate under the integral using Leibnitz’s rule, we have
dxxfxe
dxxfedt
d
dxxfedt
dtM
dt
d
tx
tx
txX
Letting t->0, this integral simply becomes
This proof can be extended for any moment of the distribution function.
xf x dx E x
Moment Generating Functions for Specific DistributionsApplication to the Uniform Distribution:
abt
eee
tabdxab
etM
atbtb
a
txb
a
tx
X
11
Following the expansion developed earlier, we have:
222
3222
333222
333222
6
1
2
11
6
1
2
11
621
61
21
11
tbabatba
t
t
ab
aabbab
t
t
ab
abab
tab
tab
tab
tab
tab
tabtabtabtM X
Letting b=1 and a=0, the last expression becomes:
The first three moments of the uniform distribution are then:
32
24
1
6
1
2
11 ttttM X
4
16
24
10
3
12
6
10
2
10
3
2
1
X
X
X
M
M
M
Application to the Univariate Normal Distribution
dxx
tx
dxeetMx
txX
2
2
2
1
2
1exp
2
1
2
1 2
2
Focusing on the term in the exponent, we have
2
222
2
222
2
222
2
22
2
2
2
2
1
2
2
1
22
2
1
2
2
1
2
1
txx
txxx
txxx
txxxtx
The next state is to complete the square in the numerator.
422
42222
22
222
2
022
0
02
ttc
tttxx
tx
ctxx
The complete expression then becomes:
2 2 2 4 2
2 2
2
2 22
21 1
2 2
1 1
2 2
x t t txtx
x tt t
The moment generating function then becomes:
22
2
222
2
1exp
2
1exp
2
1
2
1exp
tt
dxtx
tttM X
Taking the first derivative with respect to t, we get:
Letting t->0, this becomes:
2221
2
1exp ttttM X
01XM
The second derivative of the moment generating function with respect to t yields:
Again, letting t->0 yields
2222
2222
2
1exp
2
1exp
tttt
tttM X
222 0 XM
Let X and Y be independent random variables with moment generating functions MX(t) and MY(t). Consider their sum Z=X+Y and its moment generating function:
t x ytz tx tyZ
tx tyX Y
M t E e E e E e e
E e E e M t M t
We conclude that the moment generating function for two independent random variables is equal to the product of the moment generating functions of each variable.
Skipping ahead slightly, the multivariate normal distribution function can be written as:
where Σ is the variance matrix and μ is a vector of means.
xxxf 1'
2
1exp
2
1
In order to derive the moment generating function, we now need a vector t. The moment generating function can then be defined as:
1exp ' '
2XM t t t t
Normal variables are independent if the variance matrix is a diagonal matrix.
Note that if the variance matrix is diagonal, the moment generating function for the normal can be written as:
1 2 3
21
22
23
2 2 2 2 2 21 1 2 2 3 3 1 1 2 2 3 3
2 2 2 21 1 1 1 2 2 2 3 3 3
0 01
exp ' ' 0 02
0 0
1exp
2
1 1 1exp
2 2 2
X
X X X
M t t t t
t t t t t t
t t t t
M t M t M t