chapter 4 continuous random variables and probability distributions 4.1 - probability density...

Chapter 4Continuous Random Variables and Probability Distributions

4.1 - Probability Density Functions 4.2 - Cumulative Distribution Functions and

Expected Values

4.3 - The Normal Distribution

4.4 - The Exponential and Gamma Distributions 4.5 - Other Continuous Distributions

4.6 - Probability Plots

X = # “clicks” on a Geiger counter in normal background radiation.

0 T

Poisson Distribution (discrete)

For x = 0, 1, 2, …, this calculates P(x Events) in a random sample of n trials

coming from a population with rare P(Event) = .

But it may also be used to calculate P(x Events) within a random interval of

time units, for a “Poisson process” having a known “Poisson rate” α.

Recall…

X = # “clicks” on a Geiger counter in normal background radiation.

0 T

X = time between “clicks” on a Geiger counter in normal

background radiation.

Time between events is often modeled by the Exponential Distribution (continuous).

failures, deaths, births, etc.

• “Time-to-Event Analysis”• “Time-to-Failure

Analysis”• “Reliability Analysis”• “Survival Analysis”






0

ye 0

lim cy

ce

( ) 1?f x dx

0

1( )x

xf x dx e dx


parameter > 0

X = Time between events

0

1

pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp()

( )

Check pdf?

( ) 0 is clearf x

0

y

ye dy

Let ;

then

xy

dxdy

lim 1 c

ce

1

[ ] ( )E X x f x dx

0

x

x

x e dx


parameter > 0


0

1

pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp()

( )

Calculate the expected time between events

u dv uv v du Integration by Parts

xxu dv e dx

xdxdu v e

00

x x

xx e e dx

0lim 0

c x

xcc e e dx

0

Recall1 1

x

xe dx

0

[ ] ( )E X x f x dx


parameter > 0


0

1

pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp()

( )


Mean

Similarly for the variance…

2 2 2( ) ( ) ( )E X x f x dx

2 2 2 2 2( )E X x f x dx

2 2

0

1 x

xx e dx


etc... = 2

[ ] ( )E X x f x dx


parameter > 0


0

1

X ~ Exp()

( )


Mean

2 2Variance

Determine the cdf

( ) ( ) ( )x

F x P X x f t dt

00

1( )xt t

xF x e dt e

( ) 1 , 0x

F x e x

pdf1 , 0( )0, 0

x

e xf xx

[ ] ( )E X x f x dx


parameter > 0


0

1 pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp() Calculate the expected time between events

Mean

2 2Variance

Determine the cdf

( ) ( ) ( )x

F x P X x f t dt

00

1( )xt t

xF x e dt e

( ) 1 , 0x

F x e x

cdf

1 , 0( )

0, 0

x

e xF x

x

Note:

“Reliability Function” R(t)

( ) 1 ( )x

P X x F x e

“Survival Function” S(t)


parameter > 0


0

1 pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp() Example: Suppose mean time between events is known to be…

cdf

1 , 0( )

0, 0

x

e xF x

x

Mean = 2 years

Then for x 0,

2( ) ( ) 1 .x

F x P X x e

Calculate ( 3 years).P X 32(3) ( 3) 1

0.77687F P X e

Calculate the “Poisson rate” .






Ex: Suppose the mean number of instantaneous clicks/sec is = 10, then the mean time between any two successive clicks is = 1/10 sec.

Ex: Suppose the mean number of instantaneous clicks/sec is = 10, then the mean time between any two successive clicks is = 1/10 sec.

0 T

X = Time between events is often modeled by the Exponential Distribution (continuous).

The mean number of events during this time interval (0, T) is .T Therefore, the mean number of events in one unit of time is .T

However, the mean time between events was just shown to be = .

Connection?

1 second

1 ( ) .


parameter > 0


0

1 pdf1 , 0( )0, 0

x

e xf xx

X ~ Exp() Example: Suppose mean time between events is known to be…

cdf

1 , 0( )

0, 0

x

e xF x

x

Mean = 2 years

Then for x 0,

2( ) ( ) 1 .x

F x P X x e

Calculate ( 3 years).P X 32(3) ( 3) 1

0.77687F P X e

Calculate the “Poisson rate” .

1 1 event 0.5 events/yr2 years

0 T

Another property …(Event = “Failure,” etc.)

|t

|t tNo Failure

What is the probability of “No Failure” up to t + t, given “No Failure” up to t?

( | )P X t t X t ( )( )

P X t t X tP X t

1 ( )

1 ( )F t tF t

( ) ( ) 1x

F x P X x e

t t

t

e

e

t

e

independent of time t;

only depends on t

“Memory-less” property of the Exponential distribution

The conditional property of “no failure” from ANY time t to a future time t + t of fixed duration t, remains constant.

Models many systems in the “prime of their lives,” e.g., a random 30-yr old individual in the USA.

( 1) ( )n n n ( 1) ! n n

1

0( ) xx e dx

More general models exist…, e.g., The Gamma Distribution

In order to understand this, it is first necessary to understand the ”Gamma Function”

Def: For any > 0,

• Discovered by Swiss mathematician Leonhard Euler (1707-1783) in a different form.

• “Special Functions of Mathematical Physics” includes Gamma, Beta, Bessel, classical orthogonal polynomials (Jacobi, Chebyshev, Legendre, Hermite,…), etc.

• Generalization of “factorials” to all complex values of (except 0, -1, -2, -3, …).• The Exponential distribution is a special case of the Gamma distribution!Basic Properties:

(1) 1 Proof:0

(1) xe dx 0

lim cy

ce

lim 1 c

ce

1

( 1) ( ) Proof:0

( 1) xx e dx


1

x

xu x dv e dxdu x dx v e

1

0 0

x xx e x e dx

1

00 xx e dx

( )

Let = n = 1, 2, 3, …

12

1

0( ) xx e dx

(1) 0! 1 (2) 1! 1 (3) 2! 2

(4) 3! 6

(5) 4! 24

The Gamma Function

~ Gamma( , )X

( ) 1?f x dx

1

0

1( )

xx e dx

1

0

1( )

xx e dx

1 ( ) ( )

1

General Gamma Distribution

Gamma Function

11( ) for 0( )

xf x x e x

Note that if = 1, then pdf is

1

pdf parameters , 01 , 0( ) ( )

0, 0

x

x e xf xx

Standard Gamma Distribution


1( ) , 0x

f x e x

Exponential Distribution

2 2

= “shape parameter”

= “scale parameter”

1

0( ) xx e dx

~ Gamma( , )X 1

0( ) xx e dx


Gamma Function


0.5

1: ~ Exp(1)X

2 3

~ Gamma( ,1) X

1

pdf1( ) for 0( )

xf x x e x



= 1

WLOG…

1

0

1

0

cdf ( ) ( )

( )1( )

1( )

x

x y

x y

F x P X x

f y dy

y e dy

y e dy

1

0( ) xx e dx

Gamma

Function~ Gamma( ,1) X


1

pdf1( ) for 0( )

xf x x e x



1

0( ) xx e dx

Gamma

Function

1

0

1

0

cdf ( ) ( )

( )1( )

1( )

x

x y

x y

F x P X x

f y dy

y e dy

y e dy

“Incomplete

Gamma Function”

1

0

x yy e dy

1

pdf1( ) for 0( )

xf x x e x

~ Gamma( ,1) X


(No general closed form expression, but still continuous and monotonic from 0 to 1.)



~ Gamma( , )X 1

0( ) xx e dx


Gamma Function

1


0, 0

x

x e xf xx


1( ) , 0x

f x e x

Exponential Distribution

2 2

Return to…



( ) , 0xf x e x = “Poisson rate”

(= 1/ = )

Theorem: Suppose r.v.’s 1 2 3, , , , are , ~ Exp( ).nX X X X independent

“independent, identically distributed” (i.i.d.)

Then their sum 1 2 3 ~ Gamma( , ).nX X X X n e.g., failure time in machine components

~ Gamma( , )X 1

0( ) xx e dx


Gamma Function

1


0, 0

x

x e xf xx

2 2



Example: Suppose X = time between failures is known to be modeled by a Gamma distribution, with mean = 8 years, and standard deviation = 4 years. Calculate the probability of failure before 5 years.

2 28

4

42

4 1 24

1( )2 (4)

x

f x x e

3 21

(16) 3!

x

x e

3 21 , 096

x

x e x

( ) ( )F x P X x 3 20

196

txt e dt

(5) ( 5)F P X

5 3 20

196

t

t e dt

2 28

4

~ Gamma( , )X 1

0( ) xx e dx


Gamma Function

1


0, 0

x

x e xf xx

2 2



Example: Suppose X = time between failures is known to be modeled by a Gamma distribution, with mean = 8 years, and standard deviation = 4 years. Calculate the probability of failure before 5 years.

42

4 1 24

1( )2 (4)

x

f x x e

(5) ( 5)F P X

5.6

3.51.6

5.6 3

3

3.5 1 1.63.5

1( )(1.6) (3.5)

x

f x x e

Recall... ( 1) ( ) for any 0.

7 5 52 2 2

5 3 32 2 2

5 3 1 12 2 2 2

158

22

= 1

= 2

= 3

= 4 = 5 = 6

= 7

Chi-Squared Distributionwith = n 1 degrees of freedom df = 1, 2, 3,…

, 22

12 2

2

1 , 0( ) 2 ( 2)0, 0

x

x e xf xx

Special case of the Gamma distribution:

“Chi-squared Test” used in statistical analysis of categorical data.

23

F-distributionwith degrees of freedom 1 and

2 .

“F-Test” used when comparing means of two or more groups (ANOVA).

24

T-distributionwith (n – 1) degrees of freedom df = 1, 2,

3, …

“T-Test” used when analyzing means of one or two groups.

df = 1df = 2df = 5df = 10

25

T-distributionwith 1 degree of freedom

df = 1

2

1 1( ) ,1

f xx

x

“Cauchy distribution”

26


2

1 1( ) ,1

f xx

x


pdf: ( )f x dx

2

1 11

dxx

improper integral at both endpoints

0

2 20

1 1 11 1

dx dxx x

0

2 20

1 1 1lim lim1 1

b

aa bdx dx

x x

01 1

0

1 lim (tan ) lim (tan )b

aa bx x

1 11 lim ( tan ) lim (tan )a b

a b

12 2

1 1 12 2

12

12

0, 0a b

01 1

0

1 lim (tan ) lim (tan )b

aa bx x

1 11 lim ( tan ) lim (tan )a b

a b

12 2

0

2 20

1 1 11 1

dx dxx x

27


2

1 1( ) ,1

f xx

x


pdf: ( )f x dx

2

1 11

dxx

improper integral at both endpoints

0

2 20

1 1 1lim lim1 1

b

aa bdx dx

x x

( )x f x dx

2

11x dxx

0

2 20

11 1x xdx dxx x

0

2 20

1 lim lim1 1

b

aa b

x xdx dxx x

21

xyx

0

1 1 12 2

12

12

0, 0a b

0

2 20

1 lim lim1 1

b

aa b

x xdx dxx x

0

2 20

11 1x xdx dxx x

( )x f x dx

2

11x dxx

02 21 12 2 0

1 lim ln(1 ) lim ln(1 )b

aa bx x

28


2

1 1( ) ,1

f xx

x

“Cauchy distribution”improper integral at both endpoints

21xyx

0 2 21 12 2

1 lim ln(1 ) lim ln(1 )a b

a b

“indeterminate form”

|a

|b

12

12

0, 0a b

0

2 20

1 lim lim1 1

b

aa b

x xdx dxx x

0

2 20

11 1x xdx dxx x

( )x f x dx

2

11x dxx

02 21 12 2 0

1 lim ln(1 ) lim ln(1 )b

aa bx x

29


2

1 1( ) ,1

f xx

x

“Cauchy distribution”improper integral at both endpoints

21xyx

0 2 21 12 2

1 lim ln(1 ) lim ln(1 )a b

a b

“indeterminate form”

12

12

does not exist!

0, 0a b

30

Classical Continuous Probability Distributions

● Normal distribution

● Log-Normal ~ X is not normally distributed (e.g., skewed), but Y = “logarithm of X” is normally distributed

● Student’s t-distribution ~ Similar to normal distr, more flexible

● F-distribution ~ Used when comparing multiple group means

● Chi-squared distribution ~ Used extensively in categorical data analysis

● Others for specialized applications ~ Gamma, Beta, Weibull…

chapter 4 continuous random variables and probability distributions 4.1 - probability density...

Documents