discrete distributionsmath.usask.ca/~laverty/s241/s241 lectures pdf/09 s241 expectatio… · x =...
TRANSCRIPT
Discrete distributions
The Bernoulli distribution
Discrete distributions
1 0
1
q p xp x P X x
p x
0
0.2
0.4
0.6
0.8
1
0 1
1 Bernoulli trial =
0 Bernoulli trial = X
S
F
The Binomial distribution
0,1,2, ,x n xn
p x P X x p q x nx
-
0.0500
0.1000
0.1500
0.2000
0.2500
0.3000
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
p(x)
x
X = the number of successes in n repetitions of
a Bernoulli trial
p = the probability of success
-
0.02
0.04
0.06
0.08
0.10
0.12
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
The Poisson distribution
Events are occurring randomly and uniformly in time.
X = the number of events occuring in a fixed period of time.
0,1,2,3,4,!
x
p x e xx
The Negative Binomial distribution
the Bernoulli trials are repeated independently until a fixed number, k, of successes has occurred and X = the trial on which the kth success occurred.
The Geometric distribution
the Bernoulli trials are repeated independently the first success occurs (,k = 1) and X = the trial on which the 1st success occurred.
P[X = x] = p(x) = p(1 – p)x – 1 = pqx – 1
1
, 1, 2,1
k x kx
p x P X x p q x k k kk
Geometric ≡ Negative Binomial with k = 1
The Hypergeometric distribution
Suppose we have a population containing N objects.
The population are partitioned into two groups.
• a = the number of elements in group A
• b = the number of elements in the other group (group B).
Note N = a + b.
• n elements are selected from the population at random.
• X = the elements from group A. (n – X will be the number of
elements from group B.)
a b
x n xp x P X x
N
n
Continuous distributions
Uniform distribution from a to b
1
0 otherwise
a x bf x b a
0
0.1
0.2
0.3
0.4
0 5 10 15
1
b a
a b
f x
x0
0.5
1
0 5 10 15 a b x
F x
0
1
x a
x aF x P X x a x b
b a
x b
Normal distribution mean m and standard deviation s
2
221
2
x
f x em
s
s
m
s
0
0 0
xe xf x F x
x
the exponential distribution with parameter
f(x) =
0
0.5
1
-2 0 2 4 6
f(x)
The Weibull distribution - parameters a and b.
1and 0x
f x F x x e xba
b ba
f(x) =
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 1 2 3 4 5
(a = 0.5, b = 2)
(a = 0.7, b = 2)
(a = 0.9, b = 2)
The gamma distribution - parameters a and
0
0.1
0.2
0.3
0.4
0 2 4 6 8 10
(a = 2, = 0.9)
(a = 2, = 0.6)
(a = 3, = 0.6)
1 0
0 0
xx e xf x
x
aa
a
The c2 distribution with n degrees of freedom
0
0.1
0.2
0 4 8 12 16
(n = 4)
(n = 5)
(n = 6)
21
2 2
112
2
0
0 0
xx e x
f x
x
n
n
n
Expectation
Let X denote a discrete random variable with probability function p(x) (probability density function f(x) if X is continuous) then the expected value of X, E(X) is defined to be:
i i
x i
E X xp x x p x
E X xf x dx
and if X is continuous with probability density function f(x)
Example: Suppose we are observing a seven game series where the teams are evenly matched and the games are independent. Let X denote the length of the series. Find:
1. The distribution of X.
2. the expected value of X, E(X).
Solution: Let A denote the event that team A,
wins and B denote the event that team B wins.
Then the sample space for this experiment
(together with probabilities and values of X)
would be (next slide):
outcome AAAA BBBB BAAAA ABAAA AABAA AAABA
Prob (½ )4 (½ )4 (½ )5 (½ )5 (½ )5 (½ )5
X 4 4 5 5 5 5
outcome ABBBB BABBB BBABB BBBAB BBAAAA BABAAA
Prob (½ )5 (½ )5 (½ )5 (½ )5 (½ )6 (½ )6
X 5 5 5 5 6 6
outcome BAABAA BAAABA ABBAAA ABABAA ABAABA AABBAA
Prob (½ )6 (½ )6 (½ )6 (½ )6 (½ )6 (½ )6
X 6 6 6 6 6 6
outcome AABABA AAABBA AABBBB ABABBB ABBABB ABBBAB
Prob (½ )6 (½ )6 (½ )6 (½ )6 (½ )6 (½ )6
X 6 6 6 6 6 6
- continued
At this stage it is recognized that it might be easier to determine the distribution of X using counting techniques
• The possible values of X are {4, 5, 6, 7}
• The probability of a sequence of length x is (½)x
• The series can either be won by A or B.
• If the series is of length x and won by one of the teams (A say) then the number of such series is:
1
4
x
x
In a series of that lasts x games, the winning team wins 4 games and the losing team wins x - 4 games. The winning team has to win the last games. The no. of ways of choosing the games that
the losing team wins is:
1
4
x
x
Thus
1
1 11 12
4 42 2
x xx x
p x P X xx x
The no. of ways of choosing the games that the losing team wins
The no. of ways of choosing the winning team
The probability of a series of length x.
x 4 5 6 7
p(x) 3
3 1 1
0 2 8
44 1 1
1 2 4
55 1 5
2 2 16
66 1 5
3 2 16
i i
x i
E X xp x x p x
1316
1 1 5 54 5 6 7
8 4 16 16
8 20 30 35 935
16 16
0
0.1
0.2
0.3
0.4
0 1 2 3 4 5 6 7
Interpretation of E(X)
1. The expected value of X, E(X), is the centre of gravity of the probability distribution of X.
2. The expected value of X, E(X), is the long-run average value of X. (shown later –Law of Large Numbers)
E(X)
Example: The Binomal distribution
Let X be a discrete random variable having the Binomial
distribution. i. e. X = the number of successes in n
independent repetitions of a Bernoulli trial. Find the
expected value of X, E(X).
1 0,1,2,3, ,n xx
np x p p x n
x
0 0
1n n
n xx
x x
nE X xp x x p p
x
1
1
1
!1
! !
nn xx
x
nn xx
x
nx p p
x
nx p p
x n x
Solution:
0 0
1n n
n xx
x x
nE X xp x x p p
x
1
1
1
!1
! !
nn xx
x
nn xx
x
nx p p
x
nx p p
x n x
1
!1
1 ! !
nn xx
x
np p
x n x
1 21 2! !1 1
0! 1 ! 1! 2 !
n nn np p p p
n n
1! !
12 !1! 1 !0!
n nn np p p
n n
1 20 11 ! 1 !
1 10! 1 ! 1! 2 !
n nn nnp p p p p
n n
2 1
1 ! 1 !1
2 !1! 1 !0!
n nn n
p p pn n
1 20 1
1 11 1
0 1
n nn nnp p p p p
2 11 1
12 1
n nn n
p p pn n
1 1
1 1n n
np p p np np
Example: A continuous random variable
The Exponential distribution
Let X have an exponential distribution with parameter .
This will be the case if:
1. P[X ≥ 0] = 1, and
2. P[ x ≤ X ≤ x + dx| X ≥ x] = dx.
The probability density function of X is:
0
0 0
xe xf x
x
The expected value of X is:
0
xE X xf x dx x e dx
We will determine
udv uv vdu
0
xE X xf x dx x e dx
xx e dx
using integration by parts.
In this case and xu x dv e dx
Hence and xdu dx v e
Thus x x xx e dx xe e dx 1
x xxe e
0
00
1 x x xE X x e dx xe e
1 1
0 0 0
Summary:
1
E X
If X has an exponential distribution with parameter
then:
Example:
The Uniform distribution
Suppose X has a uniform distribution from a to b.
Then:
1
0 ,
b aa x b
f xx a x b
The expected value of X is:
1
b
b a
a
E X xf x dx x dx
2 2 2
1
2 2 2
b
b a
a
x b a a b
b a
Expectation
Let X denote a discrete random variable with probability function p(x) (probability density function f(x) if X is continuous) then the expected value of X, E(X) is defined to be:
i i
x i
E X xp x x p x
E X xf x dx
and if X is continuous with probability density function f(x)
The Binomial distribution
1 0,1,2,3, ,n xx
np x p p x n
x
0 0
1n n
n xx
x x
nE X xp x x p p np
x
The Exponential distribution
0
0 0
xe xf x
x
0
1xE X xf x dx x e dx
The Uniform distribution
Suppose X has a uniform distribution from a to b.
Then:
1
0 ,
b aa x b
f xx a x b
The expected value of X is:
1
b
b a
a
E X xf x dx x dx
2 2 2
1
2 2 2
b
b a
a
x b a a b
b a
Example:
The Normal distribution
Suppose X has a Normal distribution with parameters m
and s.
Then:
2
221
2
x
f x em
s
s
The expected value of X is:
2
221
2
x
E X xf x dx x e dxm
s
s
xz
m
s
Make the substitution:
1 and dz dx x zm s
s
Hence
Now
2
21
2
z
E X z e dzm s
2 2
2 21
2 2
z z
e dz ze dzs
m
2 2
2 21
1 and 02
z z
e dz ze dz
Thus E X m
Example:
The Gamma distribution
Suppose X has a Gamma distribution with parameters a
and .
Then:
1 0
0 0
xx e xf x
x
aa
a
Note:
This is a very useful formula when working with the
Gamma distribution.
1
0
1 if 0,xf x dx x e dxa
a a
a
The expected value of X is:
1
0
xE X xf x dx x x e dxa
a
a
This is now
equal to 1. 0
xx e dxa
a
a
1
1
0
1
1
xx e dxa a
a
a
a
a a
1a a a a
a a
Thus if X has a Gamma (a ,) distribution then the
expected value of X is:
E Xa
Special Cases: (a ,) distribution then the expected
value of X is:
1. Exponential () distribution: a = 1, arbitrary
1
E X
2. Chi-square (n) distribution: a = n/2, = ½.
21
2
E X
nn
The Gamma distribution
0
0.05
0.1
0.15
0.2
0.25
0.3
0 2 4 6 8 10
E Xa
0
0.05
0.1
0.15
0.2
0.25
0 5 10 15 20 25
The Exponential distribution
1
E X
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0 5 10 15 20 25
The Chi-square (c2) distribution
E X n
Expectation of functions of
Random Variables
Definition
Let X denote a discrete random variable with probability function p(x) (probability density function f(x) if X is continuous) then the expected value of g(X), E[g(X)] is defined to be:
i i
x i
E g X g x p x g x p x
E g X g x f x dx
and if X is continuous with probability density function f(x)
Example:
The Uniform distribution
Suppose X has a uniform distribution from 0 to b.
Then:
1 0
0 0,
bx b
f xx x b
Find the expected value of A = X2 . If X is the length of
a side of a square (chosen at random form 0 to b) then A
is the area of the square
2 2 2 1
b
b a
a
E X x f x dx x dx
3 3 3 2
1
0
0
3 3 3
b
b
x b b
b
= 1/3 the maximum area of the square
Example:
The Geometric distribution
Suppose X (discrete) has a geometric distribution with
parameter p.
Then:
1
1 for 1,2,3,x
p x p p x
Find the expected value of X and the expected value of
X2.
2 32 2 2 2 2 2
1
1 2 1 3 1 4 1x
E X x p x p p p p p p p
2 3
1
1 2 1 3 1 4 1x
E X xp x p p p p p p p
Recall: The sum of a geometric Series
Differentiating both sides with respect to r we get:
2 3
1
aa ar ar ar
r
2 3 1or with 1, 1
1a r r r
r
22 3
2
11 2 3 4 1 1 1
1r r r r
r
Differentiating both sides with respect to r we get:
Thus
This formula could also be developed by noting:
2 3
2
11 2 3 4
1r r r
r
2 31 2 3 4r r r
2 31 r r r
2 3r r r
2 3r r
3r
2 31
1 1 1 1
r r r
r r r r
2 3
2
1 1 1 11
1 1 1 1r r r
r r r r
This formula can be used to calculate:
2 3
1
1 2 1 3 1 4 1x
E X xp x p p p p p p p
2 31 2 3 4 where 1p r r r r p
2 31 2 3 4 where 1p r r r r p
221 1 1
= = 1
p pr p p
To compute the expected value of X2.
2 32 2 2 2 2 2
1
1 2 1 3 1 4 1x
E X x p x p p p p p p p
2 2 2 2 2 31 2 3 4p r r r
we need to find a formula for
2 2 2 2 2 3 2 1
1
1 2 3 4 x
x
r r r x r
Note 2 3
1
0
11
1
x
x
r r r r S rr
Differentiating with respect to r we get
1 2 3 1 1
1 20 1
11 2 3 4
1
x x
x x
S r r r r xr xrr
Differentiating again with respect to r we get
2 3
1 2 3 2 4 3 5 4S r r r r
32 2
31 2
21 1 2 1 1
1
x x
x x
x x r x x r rr
2
32
21
1
x
x
x x rr
Thus
2 2 2
32 2
2
1
x x
x x
x r xrr
2 2 2
32 2
2
1
x x
x x
x r xrr
implies
2 2 2
32 2
2
1
x x
x x
x r xrr
2 2 2 2 2 3 2
3
22 3 4 5 2 3 4
1r r r r r
r
Thus
2 2 2 2 3 2
3
21 2 3 4 1 2 3 4
1r r r r r r
r
2 3
3
21 2 3 4
1
rr r r
r
3 2 3 3
2 1 2 1 1
1 1 1 1
r r r r
r r r r
3
2 if 1
pr p
p
Thus
2 2 2 2 2 31 2 3 4E X p r r r
2
2 p
p
Moments of Random Variables
Definition
Let X be a random variable (discrete or
continuous), then the kth moment of X is defined
to be:
k
k E Xm
-
if is discrete
if is continuous
k
x
k
x p x X
x f x dx X
The first moment of X , m = m1 = E(X) is the center of gravity of
the distribution of X. The higher moments give different
information regarding the distribution of X.
Definition
Let X be a random variable (discrete or
continuous), then the kth central moment of X is
defined to be:
0 k
k E Xm m
-
if is discrete
if is continuous
k
x
k
x p x X
x f x dx X
m
m
where m = m1 = E(X) = the first moment of X .
The central moments describe how the
probability distribution is distributed about the
centre of gravity, m.
0
1 E Xm m
and is denoted by the symbol var(X).
= 2nd central moment. 20
2 E Xm m
depends on the spread of the probability distribution
of X about m.
20
2 E Xm m
is called the variance of X.
20
2 E Xm m
is called the standard
deviation of X and is denoted by the symbol s.
20 2
2var X E Xm m s
The third central moment
contains information about the skewness of a
distribution.
30
3 E Xm m
The third central moment
contains information about the skewness of a
distribution.
30
3 E Xm m
Measure of skewness
0 0
3 31 3 23
0
2
m m
s m
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25 30 35
0
3 0, 0m
Positively skewed distribution
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 5 10 15 20 25 30 35
0
3 10, 0m
Negatively skewed distribution
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0 5 10 15 20 25 30 35
0
3 10, 0m
Symmetric distribution
The fourth central moment
Also contains information about the shape of a
distribution. The property of shape that is measured by
the fourth central moment is called kurtosis
40
4 E Xm m
The measure of kurtosis
0 0
4 42 24
0
2
3 3m m
s m
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0 5 10 15 20 25 30 35
0
40, moderate in size m
Mesokurtic distribution
0
0 20 40 60 80
0
40, small in size m
Platykurtic distribution
0
0 20 40 60 80
0
40, large in size m
leptokurtic distribution
Example: The uniform distribution from 0 to 1
1 0 1
0 0, 1
xf x
x x
11 1
0 0
11
1 1
kk k
k
xx f x dx x dx
k km
Finding the moments
Finding the central moments:
1
0 12
0
1kk
k x f x dx x dxm m
12
making the substitution w x
1122
1 12 2
1 11 1 12 20
1 1
k kkk
k
ww dw
k km
1
1
1if even1 1
2 12 1
0 if odd
k
k
k
kk
kk
Thus
0 0 0
2 3 42 4
1 1 1 1Hence , 0,
2 3 12 2 5 80m m m
0
2
1var
12X m
0
2
1var
12Xs m
0
31 3
0m
s
The standard deviation
The measure of skewness
0
41 24
1 803 3 1.2
1 12
m
s
The measure of kurtosis
Rules for expectation
Rules:
if is discrete
if is continuous
x
g x p x X
E g X
g x f x dx X
1. where is a constantE c c c
if then g X c E g X E c cf x dx
c f x dx c
Proof
The proof for discrete random variables is similar.
2. where , are constantsE aX b aE X b a b
if then g X aX b E aX b ax b f x dx
Proof
The proof for discrete random variables is similar.
a xf x dx b f x dx
aE X b
20
23. var X E Xm m
2 22x x f x dxm m
Proof
The proof for discrete random variables is similar.
22 2
2 1E X E X m m
2 2
var X E X x f x dxm m
2 22x f x dx xf x dx f x dxm m
2 2
2 2 12E X E Xm m m m m m
24. var varaX b a X
Proof
2
var aX baX b E aX b m
2
E aX b a bm
22E a X m
22 2 vara E X a Xm
aX b E aX b aE X b a bm m
Moment generating functions
Definition
if is discrete
if is continuous
x
g x p x X
E g X
g x f x dx X
Let X denote a random variable, Then the moment
generating function of X , mX(t) is defined by:
Recall
if is discrete
if is continuous
tx
xtX
Xtx
e p x X
m t E e
e f x dx X
Examples
1 0,1,2, ,n xx
np x p p x n
x
The moment generating function of X , mX(t) is:
1. The Binomial distribution (parameters p, n)
tX tx
X
x
m t E e e p x
0
1n
n xtx x
x
ne p p
x
0 0
1n n
x n xt x n x
x x
n ne p p a b
x x
1nn ta b e p p
0,1,2,!
x
p x e xx
The moment generating function of X , mX(t) is:
2. The Poisson distribution (parameter )
tX tx
X
x
m t E e e p x 0 !
xntx
x
e ex
0 0
using ! !
t
xt x
e u
x x
e ue e e e
x x
1tee
0
0 0
xe xf x
x
The moment generating function of X , mX(t) is:
3. The Exponential distribution (parameter )
0
tX tx tx x
Xm t E e e f x dx e e dx
0 0
t xt x e
e dxt
undefined
tt
t
2
21
2
x
f x e
The moment generating function of X , mX(t) is:
4. The Standard Normal distribution (m = 0, s = 1)
tX tx
Xm t E e e f x dx
2 22
1
2
x tx
e dx
2
21
2
xtxe e dx
2 2 2 22 2
2 2 21 1
2 2
x tx t x tx t
Xm t e dx e e dx
We will now use the fact that 2
221
1 for all 0,2
x b
ae dx a ba
We have
completed
the square
22 2
2 2 21
2
x tt t
e e dx e
This is 1
1 0
0 0
xx e xf x
x
aa
a
The moment generating function of X , mX(t) is:
4. The Gamma distribution (parameters a, )
tX tx
Xm t E e e f x dx
1
0
tx xe x e dxa
a
a
1
0
t xx e dx
aa
a
We use the fact
1
0
1 for all 0, 0a
a bxbx e dx a b
a
1
0
t x
Xm t x e dxa
a
a
1
0
t xtx e dx
tt
a aaa
a
a
Equal to 1
Properties of
Moment Generating Functions
1. mX(0) = 1
0, hence 0 1 1tX X
X Xm t E e m E e E
v) Gamma Dist'n Xm tt
a
2
2iv) Std Normal Dist'n t
Xm t e
iii) Exponential Dist'n Xm tt
1
ii) Poisson Dist'n te
Xm t e
i) Binomial Dist'n 1n
t
Xm t e p p
Note: the moment generating functions of the following
distributions satisfy the property mX(0) = 1
2 33212. 1
2! 3! !
kkXm t t t t t
k
m mmm
We use the expansion of the exponential function:
2 3
12! 3! !
ku u u u
e uk
tX
Xm t E e
2 32 31
2! 3! !
kkt t t
E tX X X Xk
2 3
2 312! 3! !
kkt t t
tE X E X E X E Xk
2 3
1 2 312! 3! !
k
k
t t tt
km m m m
0
3. 0k
k
X X kk
t
dm m t
dtm
Now
2 33211
2! 3! !
kkXm t t t t t
k
m mmm
2 1321 2 3
2! 3! !
kkXm t t t kt
k
m mmm
2 13
1 22! 1 !
kkt t tk
m mm m
1and 0Xm m
242 3
2! 2 !
kkXm t t t t
k
mmm m
2and 0Xm m
continuing we find 0k
X km m
i) Binomial Dist'n 1n
t
Xm t e p p
Property 3 is very useful in determining the moments of a
random variable X.
Examples
1
1n
t t
Xm t n e p p pe
1
0 0
10 1n
Xm n e p p pe np m m
2 1
1 1 1n n
t t t t t
Xm t np n e p p e p e e p p e
2
2
1 1 1
1 1
nt t t t
nt t t
npe e p p n e p e p p
npe e p p ne p p
2 2
21np np p np np q n p npq m
1
ii) Poisson Dist'n te
Xm t e
1 1t te e tt
Xm t e e e
1 1 2 121
t t te t e t e tt
Xm t e e e e
1 2 12 2 1
t te t e tt t
Xm t e e e e
1 2 12 3t te t e tte e e
1 3 1 2 13 23t t te t e t e t
e e e
0 1 0
1 0e
Xm e
m
0 01 0 1 02 2
2 0e e
Xm e e
m
3 0 2 0 0 3 2
3 0 3 3t
Xm e e em
To find the moments we set t = 0.
iii) Exponential Dist'n Xm tt
1
X
d tdm t
dt t dt
2 2
1 1t t
3 3
2 1 2Xm t t t
4 4
2 3 1 2 3Xm t t t
5 54
2 3 4 1 4!Xm t t t
1
!kk
Xm t k t
Thus
2
1
10Xmm m
3
2 2
20 2Xmm
1 !
0 !kk
k X k
km km
2 33211
2! 3! !
kkXm t t t t t
k
m mmm
The moments for the exponential distribution can be calculated in an
alternative way. This is note by expanding mX(t) in powers of t and
equating the coefficients of tk to the coefficients in:
2 31 11
11Xm t u u u
tt u
2 3
2 31
t t t
Equating the coefficients of tk we get:
1 ! or
!
kkk k
k
k
mm
2
2t
Xm t e
The moments for the standard normal distribution
We use the expansion of eu. 2 3
0
1! 2! 3! !
k ku
k
u u u ue u
k k
2 2 2
22
2
2 3
2 2 2
21
2! 3! !
t
kt t t
tXm t e
k
2 4 6 212 2 3
1 1 11
2 2! 2 3! 2 !
k
kt t t t
k
We now equate the coefficients tk in:
2 22211
2! ! 2 !
k kk kXm t t t t t
k k
m mmm
If k is odd: mk = 0.
2 1
2 ! 2 !
k
kk k
mFor even 2k:
2
2 !or
2 !k k
k
km
1 2 3 4 2
2! 4!Thus 0, 1, 0, 3
2 2 2!m m m m