chapter 5 joint probability distributions the adventure continues as we consider two or more random...

Chapter 5Joint Probability Distributions

The adventure continues as we consider two or more random variables all at the same time.

Chapter 5BDiscrete

Chapter 4 Homework – the stats

mean 87.3std dev 16.5median 91

skewness -2.12minimum 29

maximum 1001 quartile 833 quartile 97

interquartile range 14range 71

kurtosis 4.27

Kurtosis characterizes the relative peakedness or flatness of a distribution compared with the normal distribution. Positive kurtosis indicates a relatively peaked distribution. Negative kurtosis indicates a relatively flat distribution.

This week in Prob/Stat

today’s good stuffas it pertains to discrete RV

Expected Value of a Function of Two Random Variables A common measure of the relationship between two

random variables is the covariance. To describe covariance, we need to describe the expected value of a function of two RV’s:

E[h(x,y)] can be thought of as the weighted average of h(x,y) for each point in the range of (X, Y) and represents the average value of h(X, Y).

[ ( , )] ( , ) ( , ) X, Y discreteXYR

E h X Y h x y f x y

Here is an old favorite joint distribution Let X = the number of orders placed per day for a high cost item Let Y = the number of items in stock at the start of the day

fxy(x,y)

Let Z = Y - X, the daily ending inventory. Then h(x,y) = Y - X, and E[h(,x,y)] = E[Z]

= h(0,0) (.2) + h(1,0) (.1) + h(2,0) (.08) + h(0,1) (.15) + h(1,1) (.17) + h(2,1) (.09) + h(0,2) (.03) + h(1,2) (.05) + h(2,2) (.08) + h(0,3) (.01) + h(1,3) (.01) + h(2,3) (.03)

= 0 (.2) + (-1)(.1) + (-2)(.08) + (1)(.15) + (0)(.17) + (-1)(.09) + (2)(.03) + (1)(.05) + (0)(.08) + (3)(.01) + (2)(.01) + (1)(.03) = -.01

X/Y 0 1 2 3 fx(x)0 0.2 0.15 0.03 0.01 0.391 0.1 0.17 0.05 0.01 0.332 0.08 0.09 0.08 0.03 0.28

fy(y) 0.38 0.41 0.16 0.05 1

Another look at the inventory

,,

( , )x yx y

E Y X Y X f x y Y 0 1 2 3 0 1 2 3 0 1 2 3 sumX 0 0 0 0 1 1 1 1 2 2 2 2  

f(x,y) 0.2 0.15 0.03 0.01 0.1 0.17 0.05 0.01 0.08 0.09 0.08 0.03 1Y-X 0 1 2 3 -1 0 1 2 -2 -1 0 1  

(Y-X)f(x,y) 0 0.15 0.06 0.03 -0.1 0 0.05 0.02 -0.2 -0.1 0 0.03 -0.01

Recall: E[X] = x = 0 (.39) + 1 (.33) + 2 (.28) = .89

E[Y] = y = 0 (.38) + 1 (.41) + 2 (.16) + 3 (.05) = .88

E(Y – X) = E(Y) – E(X) = .88 - .89 = - .01Only worksfor linearrelationships!

Covariance Definition: The covariance between the

RV’s X & Y, denoted as cov(X,Y) or is: XY

[( )( )] [ ]

( ) ( ) ( )

( )

XY X Y X Y X Y

X Y X Y

X Y

E X Y E XY Y X

E XY E Y E X X

E XY

XY

Covariance

If the points in the joint probability distribution tend to fall along a line of positive (negative) slope, then is positive (negative).

Covariance measures the linear association between RV’s.

Covariance is not a dimensionless quantity

( )XY X YE XY

XY

Returning to our old favoriteX/Y 0 1 2 3 sum

0 0.2 0.15 0.03 0.01 0.391 0.1 0.17 0.05 0.01 0.332 0.08 0.09 0.08 0.03 0.28

sum 0.38 0.41 0.16 0.05 1

fxy(x,y)

E[XY] = h(0,0) (.2) + h(1,0) (.1) + h(2,0) (.08) + h(0,1) (.15) + h(1,1) (.17) + h(2,1) (.09) + h(0,2) (.03) + h(1,2) (.05) + h(2,2) (.08) + h(0,3) (.01) + h(1,3) (.01) + h(2,3) (.03)

= 0 (.2) + 0 (.1) + 0 (.08) + 0 (.15) + (1) (.17) + 2 (.09) + 0 (.03) + 2 (.05) + 4 (.08) + 0 (.01) + 3 (.01) + 6 (.03) = .98

Cov(XY) = E[XY] - x y = .98 – (.89) (.88) = .1968

Covariance Between X and Y

Fig 5-13 Joint distributions and the sign of the covariance between X and Y.

Correlation Definition: the correlation between RV’s X

and Y, denoted by is:

For any two random variables X and Y

Similar to covariance, correlation is also a measure of linear relationship between RV’s. If X and Y are independent RV’s, then

Correlation is dimensionless and unaffected by the units chosen for measure.

XY cov( , )

( ) ( )XY

XYX Y

X Y

V X V Y

1 1XY

0XY XY

Returning to, yes, the olde favorite

cov( , ) .1968

.2848( ) ( ) .6579 .7256

XYXY

X Y

X Y

V X V Y

X/Y 0 1 2 3 sum0 0.2 0.15 0.03 0.01 0.391 0.1 0.17 0.05 0.01 0.332 0.08 0.09 0.08 0.03 0.28

sum 0.38 0.41 0.16 0.05 1

I would say, a somewhat weak positive linear relationship.

5-3 Covariance and Correlation

Example 5-26

Figure 5-14 Joint distribution for Example 5-26.

5-3 Covariance and Correlation

Example 5-26 (continued)

Example 5-27If P(X = 1) = 0.2, P(X = 2) = 0.6, P(X = 3) = 0.2 and Y = 2X + 5, then P(Y = 7) = 0.2, P(Y = 9) = 0.6, P(Y = 11)

= 0.2. Determine the covariance and correlation between X and Y.

X

Y11

9

7

1 2 3

0.2

0.6

0.2

( , ) ( , )x y

XYR R

E X Y xyf x y

( ) ( ) ( ) ( )x y

X YR R

E X xf x E Y yf y

2

2

( ) ( )

( ) ( )

x

y

x XR

y YR

V X x f x

V Y y f y

Example 5-27 Cont’d

E(XY) = (1 x 7 x 0.2) + (2 x 9 x 0.6) + (3 x 11 x 0.2) = 18.8E(X) = (1 x 0.2) + (2 x 0.6) + (3 x 0.2) = 2.0E(Y) = (7 x 0.2) + (9 x 0.6) + (11 x 0.2) = 9.0V(X) = (1 – 2)2(0.2) + (2 – 2)2(0.6) + (3 – 2)2(0.2) =

0.4V(Y) = (7 – 9)2(0.2) + (9 – 9)2(0.6) + (11 – 9)2(0.2) =

1.6 ( ) 18.8 (2)(9) 0.8

0.81.0

( ) ( ) (0.4)(1.6)

XY X Y

XYXY

E XY

V X V Y

Example 5-27 Cont’d If we change the functional relationship of the

RV’s X and Y to be Y = - 2X + 13 and leave the marginal probabilities of X (fX(x)) unchanged, compute the covariance and correlation again.

Y

X 1 2 3

11

9

7

0.2

0.6

0.2

Example 5-29 Cont’d

For these changes, E(X), E(Y), V(X), and V(Y) remain the same. However,

E(XY) = (1 x 11 x 0.2) + (2 x 9 x 0.6) + (3 x 7 x 0.2)

= 17.2

( ) 17.2 (2)(9) 0.8

0.81

( ) ( ) (0.4)(1.6)

XY X Y

XYXY

E XY

V X V Y

Example 5-28 Finally, if we let X & Y have the following joint

distribution, we can recompute the covariance and the correlation:

X Y p 1 7 0.2 2 9 0.6 3 7 0.2

Y

X

11

9

7

1 2 3

0.2

0.6

0.2

Example 5-28 Cont’dFor this set of values, E(X) and V(X) are

unchanged from the previous versions.E(XY) = (1 x 7 x 0.2) + (2 x 9 x 0.6) + (3 x 7 x

0.2) = = 16.4E(Y) = (7 x 0.4) + (9 x 0.6) = 8.2V(Y) = (7 – 8.2)2(0.4) + (9 – 8.2)2(0.6) = 0.96

( ) 16.4 (2)(8.2) 0

00

( ) ( ) (0.4)(0.96)

XY X Y

XYXY

E XY

V X V Y

Are X and Y independent?

If X & Y are independent

X/Y 0 1 2 3 fx(x) E[X]0 0.2187 0.1458 0.2916 0.0729 0.729 0.301 0.0729 0.0486 0.0972 0.0243 0.243 V[X]2 0.0081 0.0054 0.0108 0.0027 0.027 0.273 0.0003 0.0002 0.0004 0.0001 0.001

fy(y) 0.3 0.2 0.4 0.1E[Y] = 1.3 V[Y] = 2.7E[XY] = 0.390 Cov = 0

look

5-5 Linear Combinations of Random Variables

Definition

Mean of a Linear Combination

5-5 Linear Combinations of Random Variables

Variance of a Linear Combination

Can you provide any insight on this

variance thing with covariance's?

A Demonstration

1 2 1 2 1 2

2 22 2 21 2 1 2 1 2 1 1 2 2 1 2

2 2 2 21 1 2 1 2 1 2 2 1 2 1 1 2 2

22 2 2 2 21 1 2 2 1 1 2 2

2 ; 2 ; 2, 1

2 2 ( , ) 4 4 ( , )

4 , 4 , , 4 4

4 4 4 4

Y X X E Y c c

E Y E X X x x f x x x x x x f x x

x f x x x x f x x x f x x E x E x x E x

V Y E Y E Y E x E x x E x

2 2 2 21 1 1 2 1 2 2 2

1 1 2 2

4 4 4 4

4 4 ( )

E x E x x E x

V X Cov X X V X

A most enjoyable

experience.

Linear Combinations of RV’s – A First Example

If Y = 4X1 + 3X2 and X1 and X2 are independent, then:

E(Y) = 4E(X1) + 3E(X2)

V(Y)= 16V(X1) + 9V(X2)

This squared coefficients results from the fact that the variable of interest is always squared when computing variance, so its constant must also be squared.

Something to Ponder

1 2

1 2

1 2

1 2

2 2 2

2 2 2

2

if 0 (e.g. independent)

y x x

y x x xy

y x x xy

Y X X

Shouldn’t we subtract the variances?

The Box:Let X1 = 0, 1, 2, 3, … , 10; range = 10 X2 = 0, 1, 2, 3, … , 10; range = 10Then Y = -10, -9, … -1, 0, 1, 2, … ,10; range = 20

Linear Combinations - An example

Let Xi = a discrete RV, the daily demands for product i, i = 1,…,n where Xi ~ Geo(pi) and Ri = selling price of product i

Let Y = total daily revenue and assume independence

1 1 1 1

andi

n n n ni

i i i i i xi i i i i

RY R X E Y E R X R

p

22 2

21 1 1

2

21

1

1

i

n n ni i

i i i xi i i i

ni i

yi i

R pV Y V R X R

p

R p

p

What aboutPr{Y < y} = F(y) = ?

Reproductive Discrete Distributions

1 1

If Bin( , ), then Bin ,m m

i i i ii i

X n p Y X n p

1

If Geo( ), then NegBin ,m

i ii

X p Y X r m p

1 1

If NegBin( , ), then NegBin ,m m

i i i ii i

X r p Y X r p

1 1

If Pois( ), then Poism m

i i i ii i

X Y X

Given m independent random variables, Xi, i = 1,…,n

A Reproductive Example Let Xi = a discrete random variable, the daily demand for

a reversed cylindrical aluminum spring coil Daily demand is Poisson with a mean of 2.3. Lead-time is a fixed 5 working days. There are currently 8 units in stock. A purchase order for

additional units has just been placed. What is the probability of not having a stock out before the next shipment arrives?

Let Y = the Lead-time demand

5

1

11.58

0

Then Pois 5 2.3 11.5

(11.5)Pr{ 8} .19059

!

ii

x

x

Y X

eY

x

5-5 Linear Combinations of Random Variables

Mean and Variance of an Average

Note that all Xi have the same mean and variance

Example mean and variance of an average

The number of nightly guests (rooms rented) at the Dewdrop Inn has a uniform distribution from 82 to 126.

Determine the mean and variance of the average number of guests over a 30-day period.

(82,126); [ ] 104; [ ] 168.67; 12.99ii i i xX U E X Var X

30

1

2

; 10430

168.675.622

30 302.371

ii

X

XX E X

Var X

5-6 General Functions of Random Variables

These linear relationships are child’s play, What can

you say about some nonlinear relationships

among the random variables?

5-6 General Functions of (Discrete) Random Variables

Given a discrete RV X with PMF fx(x) Let Y = h(x) and x = u(y) be the inverse

function Then fy(y) = fx[u(y)]

I could really use an example

of this.

The Example Let X = a discrete RV, the number of units tested until a

good (success) unit is found. X ~ Geo(.25). Let Y = cost of testing where Y = 20X2

1 120 20

( )20

( ) 1 .25 .75y y

y

yu y

f y p p

y = 20, 80, 180, 320, …

X fx(x) Y fy(y)1 0.25 20 0.252 0.1875 80 0.18753 0.140625 180 0.1406254 0.105469 320 0.1054695 0.079102 500 0.0791026 0.059326 720 0.0593267 0.044495 980 0.0444958 0.033371 1280 0.0333719 0.025028 1620 0.02502810 0.018771 2000 0.018771

1( ) .25(.75)xxf x

What Independence does for usIf X1 and X2 are independent:

2 1 2 1

2 1

2 2 2 2 21 2 1 2 1 2 1 2 1 2

2 2 2 22 2 1 1 2 1

( , ) ( ) ( )

( ) ( )

x x x x

x x

E X X x x f x x x x f x f x

x f x x f x E X E X

2 1 2 1

2 1

1 2 1 2 1 2 1 2 1 2

2 2 1 1 1 2

( , ) ( ) ( )

( ) ( )

x x x x

x x

E X X x x f x x x x f x f x

x f x x f x E X E X

Some pondering (i.e. deep thinking)

Now what can be said about Y = X1X2?

1 2 1 2

1 2 1 2

1 2 1 2

1 2

1 2

since ( ) ,

( )

0

X X X X

X X X X

X X X X

E X X

then E X X

if

If X1 and X2 are independent:

1 1 2 2 1 2

1 2 2 1 1 2

22

1 2 1 2 1 2

2 21 2 1 2

2 2 2 2 2 2

2 2 2 2 2 2

[ ]

from independence

x x x x x x

x x x x x x

Var X X E X X E X X

E X E X E X E X

2 2 2since E X

An Example to Ponder

A part made by the Kovery Ance Company is produced in lot sizes of 20 units. Following final assembly, each item is inspected with an independent .9 probability of passing.

Those items passing inspection must then be calibrated and aligned. Each independent attempt at alignment has a one-third chance of succeeding.

Find the mean and variance of the total number of alignment attempts that must be made for a given production lot.

Pondering that Example… Let X1 = a discrete RV, the number of units in a

production lot that passes final inspection. Let X2 = a discrete RV, the number of alignment

attempts until successful. Let Y = total number of alignment attempts

1 2

120,.9

3X Bin X Geo

1 2

1 2

1 2

1 2

120(.9) 18; 3

; assume independent

( ) 18 3 54

X X

X X

npp

Y X X

then E X X

Now Ponder the Variance

1 2

1

2 22

120,.9

3

(1 ) 18(.1) 1.8

1 1 1/ 3 2 / 36

1/ 91/ 3

X Bin X Geo

Var X np p

pVar X

p

1 2 2 1 1 2

2 2 2 2 2 21 2[ ]

18 6 3 1.8 1.8 6 124.2

11.14

x x x x x x

y

Var X X

Next Time