NIPRL

Chapter 3. Discrete Probability Distributions

3.1 The Binomial Distribution3.2 The Geometric and Negative Binomial Distributions3.3 The Hypergeometric Distribution3.4 The Poisson Distribution3.5 The Multinomial Distribution

NIPRL

3.1 The Binomial Distribution3.1.1 Bernoulli Random Variables(1/2)

• To model- the outcome of a coin toss,- whether a valve is open or shut,- whether an item is defective or not,- any other process that has only two possible outcomes.

• The outcomes are labeled 0 and 1

• The random variable is defined by the parameter p, 0 p 1, which is the probability that the outcome is 1.

NIPRL

3.1.1 Bernoulli Random Variables(2/2)

• Expectations

( ) (0 ( 0)) (1 ( 1))

(0 (1 )) (1 )

E X P X P X

p p

p

2 2 2( ) (0 ( 0)) (1 ( 1))E X P X P X

p

2 2

2

( ) ( ) ( ( ))

(1 )

Var X E X E X

p p p p

NIPRL

3.1.2 Definition of the Binomial Distribution(1/5)

• Consider an experiment consisting of

– n Bernoulli trials (X1, ……, Xn)

– that are independent and– that each have a constant probability p of success.

• Then the total number of successes X, that is X=X1+…+Xn,

is a random variable that has a binomial distribution with parameters n and p, which is written

X~B(n,p)

NIPRL

3.1.2 Definition of the Binomial Distribution(2/5)

• The probability mass function of a B(n,p) random variable is

for , with

( ) (1 )x n xnP X x p p

x

0,1,...,x n

1

1

( ) ( ) ( )

( ) ( ) ( )

(1 ) (1 ) (1 )

n

n

E X E X E X p p np

Var X Var X Var X

p p p p np p

NIPRL

3.1.2 Definition of the Binomial Distribution(3/5)

• Ex) X~B(8,0.5)

3 5 88 8!( 3) 0.5 (1 0.5) 0.5 0.219

3 3!5!P X

0 8 1 7

8 8

1 ( 0) ( 1)

8 80.5 (1 0.5) 0.5 (1 0.5)

0 1

8! 8!0.5 0.5 0.004 0.031 0.035

0!8! 1!7!

P X P X P X

NIPRL

3.1.2 Definition of the Binomial Distribution(4/5)

0.004

0.273

0.219

0.109

0.0310.004

0.219

0.109

0.031

4320 1 5 6 7 8 x

Probability

P X x

x 4320 1 5 6 7 8

0.004 0.636 0.855 0.965 0.996 0.10000.3630.1440.035

• Ex) X~B(8,0.5)

NIPRL

3.1.1 Definition of the Binomial Distribution(5/5)

• Symmetric Binomial Distributions :

A B(n,0.5) distribution is a symmetric probability distribution for any value of the parameter n. The distribution is symmetric about the expected value n/2.

NIPRL

Example 24 : Air Force Scrambles(1/3)

• 16 planes.

• A probability of 0.25 that the engines of a particular plane does not start at a given attempt.

• Then, the number of planes successfully launched has a binomial distribution with parameters n = 16 and p = 0.75

• The expected number of plane launched is

16 0.75 12E X np

NIPRL

Example 24 : Air Force Scrambles(2/3)

• The variance is

• The probability that exactly 12 planes scramble successfully is

• The probability that at least 14 planes scramble successfully is

(1 ) 16 0.75 0.25 3Var X np p

12 4 12 416 16!12 0.75 0.25 0.75 0.25 0.225

12 12!4!P X

14 2 15 1 16 0

14 14 15 ( 16)

16 16 160.75 0.25 0.75 0.25 0.75 0.25

14 15 16

0.134 0.054 0.010 0.198

P X P X P X P X

NIPRL

Example 24 : Air Force Scrambles(3/3)

0.000

0.052

0.180

0.208

0.054

0.0060.0000.000

9751 3 11 13 15

x

Probability

10

862 4 12 14 16

0.000 0.0010.000 0.020

0.1100.134

0.010

0.225

P X x

x 9751 3 11 13 15

10

862 4 12 14 16

0.000 0.079 0.369 0.802 0.9900.0070.000

0.000 0.0010.000 0.027 0.189 0.936 0.1000.594

0.000

NIPRL

Proportion of successes in Bernoulli Trials

• Let

• Then, if

( , )X B n p

/Y X n

( )

(1 )( )

E Y p

p pV Y

n

NIPRL

3.2 The Geometric and Negative Binomial Distributions3.2.1 Definition of the Geometric Distribution(1/2)

• The number of trials up to and including the first success in a sequence of independent Bernoulli trials with a constant success probability p has a geometric distribution with parameter p.

• The probability mass function is

for

1(1 )xP X x p p

1, 2, 3, 4 .x

NIPRL

3.2.1 Definition of the Geometric Distribution(2/2)

• The cumulative distribution function is

• The expectations

1 (1 )xP X x p

1E X

p

2

1 pVar X

p

NIPRL

- Expectation of X:

- Variance of X:

1 12

1 1

1 1[ ] (1 ) (1 )x x

x x

E X x p p p x p pp p

2 2

2 2 1 2 1

1 1

2 12 3

1

2 2 2

( ) [ ] [ ]

[ ] (1 ) (1 )

1 1 2(1 ) 2(1 )

2 1 1( )

x x

x x

x

x

Var X E X E X

E X x p p p x p

p px p

p p p p

p pVar X

p p p

NIPRL

Example 24 : Air Force Scrambles(1/2)

• If the mechanics are unsuccessful in starting the engines, then they must wait 5 minutes before trying again.

• The distribution of the number of attempts needed to start a plane’s engine geometric distribution with p = 0.75.

• The probability that the engines start on the third attempt is

23 0.25 0.75 0.047P X

NIPRL

Example 24 : Air Force Scrambles(2/2)

• The probability that the plane is launched within 10 minutes of the first attempt to start the engines is

• The expected number of attempts required to start the engines is

1 11.33

0.75E X

p

33 1 0.25 0.984P X

NIPRL

3.2.2 Definition of the Negative Binomial Distribution(1/2)

• The number of trials up to and including the r th success in a sequence of independent Bernoulli trials with a constant success probability p has a negative binomial distribution with parameter p and r

• The probability mass function is

for

1(1 )

1x r rx

P X x p pr

, 1, 2, .x r r r

NIPRL

3.2.2 Definition of the Negative Binomial Distribution(2/2)

• The expectations

rE X

p

2

(1 )r pVar X

p

NIPRL

Example 12 : Personnel Recruitment(1/2)

• Suppose that a company wishes to hire three new workers and each applicant interviewed has a probability of 0.6 of being found acceptable.

• The distribution of the total number of applicants that the company needs to interview Negative Binomial distribution with parameter p = 0.6 and r = 3.

• The probability that exactly six applicants need to be interviewed is

3 356 0.4 0.6 0.138

2P X

NIPRL

Example 12 : Personnel Recruitment(2/2)

• If the company has a budget that allows up to six applicants to be interviewed, then the probability that the budget is sufficient is

• The expected number of interviews required is

6 3 4 5 6

0.216 0.259 0.207 0.318 0.820

P X P X P X P X P X

35

0.6

rE X

p

NIPRL

3.3 The Hypergeometric Distribution3.3.1 Definition of the Hypergeometric Distribution(1/3)

• Consider a collection of N items of which r are of a certain kind.

• If one of the items is chosen at random, the probability that it is of the special kind is clearly

• Consequently, if n items are chosen at random with replacement, then clearly the distribution of X, the number of defective items chosen, is

rp

N

~ , /X B n r N

NIPRL

3.3.1 Definition of the Hypergeometric Distribution(2/3)

• However, if n items are chosen at random without replacement, then the distribution of X is the hypergeometric distribution.

• The hypergeometric distribution has a probability mass function given by

for

r N r

x n xP X x

N

n

max{0, } min{ , }.n r N x n r

NIPRL

3.3.1 Definition of the Hypergeometric Distribution(3/3)

• The expectations

• It represents the distribution of the number of items of a certain kind in a random sample of size n drawn without replacement from a population of size N that contains r items of this kind.

11

N n r rVar X n

N N N

rE X n

N

NIPRL

• Let X and Y be independent binomial random variables such that

Let us consider the conditional probability mass function of X given

that X+Y=n.

That is, the conditional distribution of X given the value of X+Y is hypergeometric!

( , ),X B r p

{ , } { , }{ | }

{ } { }

P X x X Y n P X x Y n xP X x X Y n

P X Y n P X Y n

{ } { }{ | }

{ }

(1 ) (1 )

(1 )n

x n x n x N r n x

N n

P X x P Y n xP X x X Y n

P X Y n

n N rp p p p

x n x

Np p

n

n N r

x n x

N

n

( , )Y B N r p

NIPRL

• Expectation of X: Let Xi be the following random variable: Xi=1 if the ith selection is acceptable; 0 otherwise. Then,

Let X be the hypergeometric random variable with parameters (r,N.n). Then,

This gives

• Variance of X:

{ 1}i

rP X

N

1

n

ii

X X

1 1

[ ] [ ] { 1}n n

i ii i

nrE X E X P X

N

1 1 1

( ) ( ) ( ) 2 ( , )n n n n

i i i ji i i j i

Var X Var X Var X Cov X X

NIPRL

Since Xi is a Bernoulli random variable,

Also for i<j,

Hence,

Cf. Let Then, and As N goes to infinity,

( ) { 1}(1 { 1})i i i

r N rVar X P X P X

N N

( , ) [ ] [ ] [ ]i j i j i jCov X X E X X E X E X

[ ] { 1} { 1, 1}i j i j i jE X X P X X P X X 1

{ 1, 1} { 1} { 1| 1}1i j i j i

r rP X X P X P X X

N N

2

2

1 ( )( , )

1 ( 1)i j

r r r r N rCov X X

N N N N N

2 2

( ) ( 1) ( )( ) 1

( 1) 1

r N r n n N N r r r N nVar X n

N N N N N N

/p r N

[ ]E X np ( ) (1 )1

N nVar X np p

N

( ) (1 )Var X np p

NIPRL

Example 17 : Milk Container Contents(1/3)

• Suppose that milk is shipped to retail outlets in boxes that hold 16 milk containers. One particular box, which happens to contain six under weight containers, is opened for inspection, and five containers are chosen at random.

• The distribution of the number of underweight milk containers in the sample chosen by inspector Hypergeometric distribution with N=16, r=6, and n=5.

NIPRL

Example 17 : Milk Container Contents(2/3)

• The probability that the inspector chooses exactly two underweight containers is

• The expected number of underweight containers chosen by the inspector is

6 10 6! 10!2 3 2!4! 3!7!

2 0.41216 16!

5!11!5

P X

5 61.875

16

nrE X

N

NIPRL

Example 17 : Milk Container Contents(3/3)

0.0580.034

0.002

0.206

0.412

0.288

4320 1 5Number of underweight milk containers found by inspector

Probability

NIPRL

3.4 The Poisson Distribution3.4.1 Definition of the Poisson Distribution(1/3)

• The distribution of– The number of defects in an item– The number of radioactive particles emitted by a

substance– The number of telephone calls received by an operator

with a certain time limit

• That is, the number of “events” that occur within certain specified boundaries.

NIPRL

3.4.1 Definition of the Poisson Distribution(2/3)

• A random variable X distributed as a Poisson random variable with parameter λ, which is written

has a probability mass function

for

~X P

!

xeP X x

x

0, 1, 2, 3, .x

NIPRL

3.4.1 Definition of the Poisson Distribution(3/3)

• The Poisson distribution is often useful to model the number of times that a certain event occurs per unit of time, distance, or volume, and it has a mean and variance both equal to the parameter value λ.

• The expectations

E X Var X

NIPRL

• The Poisson random variable can be used as an approximation for a binomial random variables with parameters (n,p) when n is large and p is small:

Let X be a binomial random variable with parameters (n,p) and

Then,

That is,

For large n and small p,

Hence,

np ! !

{ } (1 ) 1( )! ! ( )! !

i n ii n in n

P X i p pn i i n i i n n

( 1) ( 1) (1 / ){ }

! (1 / )

i n

i i

n n n i nP X i

n i n

( 1) ( 1)1 , 1, 1 1

n i

i

n n n ie

n n n

{ }!

i

P X i ei

NIPRL

Example 3 : Software Errors(1/3)

• Suppose that the number of errors in a piece of software has a Poisson distribution with parameter λ=3.

• [The expected number of errors] = [The variance in the number of errors] = 3.

• The probability that a piece of software has no error is

3 0

330 0.050

0!

eP X e

NIPRL

Example 3 : Software Errors(2/3)

• The probability that there are three or more errors in a piece of software is

3 0 3 1 3 2

3

3 1 0 1 2

3 3 31

0! 1! 2!

1 3 91

1 1 2

1 0.423 0.577

P X P X P X P X

e e e

e

NIPRL

Example 3 : Software Errors(3/3)

0.101

0.050

0.168

0.0010.008

0.050

9751 3 11 13

Number of software errors

Probability

10

862 4 12

0.0220.003

0.149

0.224

0

0.224

P X x

x 9751 3 11 1310

862 4 12

0.101

0.050 0.168 0.0010.0080.050

0.022 0.0030.149

0.224

0.224

0

0.000 0.000

0.000

NIPRL

3.5 The Multinomial Distribution3.5.1 Definition of the Multinomial Distribution(1/2)

• Consider a sequence of n independent trials where each individual trial can have k outcomes that occur with constant

probability value p1,…, pk with p1+···+pk = 1. The random

variable X1,…, Xk that count the number of occurrences of

each outcome are said to have a multinomial distribution.

• The joint probability mass function is

for nonnegative integer values of the satisfying

11 1 1

1

!,...,

! !kxx

k k kk

nP X x X x p p

x x

ix 1 .kx x n

NIPRL

3.5.1 Definition of the Multinomial Distribution(2/2)

• The random variables X1,…, Xk have expectation and variances given by

but they are not independent. (why?)

(1 )i i i i iE X np Var X np p

NIPRL

Example 1 : Machine Breakdowns(1/4)

• Suppose that the machine breakdowns are attributable to electrical faults, mechanical faults, and operator misuse, and these causes occur with probabilities of 0.2, 0.5, and 0.3, respectively.

• The engineer is interested in predicting the causes of the next ten breakdowns.

• X1: the number of breakdowns due to electrical reasons.

• X2: the number of breakdowns due to mechanical reasons.

• X3: the number of breakdowns due to operator misuse.

NIPRL

Example 1 : Machine Breakdowns(2/4)

• X1+X2+X3=10

• If the breakdown causes can be assumed to be independent of one another, then the probability mass function is

• The probability that there will be three electrical breakdowns, five mechanical breakdowns, and two misuse breakdowns is

31 21 1 2 2 3 3

1 2 3

10!, , 0.2 0.5 0.3

! ! !xx xP X x X x X x

x x x

3 5 21 2 3

10!3, 5, 2 0.2 0.5 0.3 0.057

3!5!2!P X X X

NIPRL

Example 1 : Machine Breakdowns(3/4)

• The expected number of electrical breakdowns is

• The expected number of mechanical breakdowns is

• The expected number of misuse breakdowns is

1 1 10 0.2 2E X np

2 2 10 0.5 5E X np

3 3 10 0.3 3E X np

NIPRL

Example 1 : Machine Breakdowns(4/4)

• If the engineer is interested in the probability of there being no more two electrical breakdowns, then this can be

calculated by nothing that X1~B(10, 0.2), so that

1 1 1 1

0 10 1 9 2 8

2 0 1 2

10 10 100.2 0.8 0.2 0.8 0.2 0.8

0 1 2

0.107 0.268 0.302 0.677

P X P X P X P X