further probability topics

8/11/2019 Further Probability Topics

1/27

1/27

EC114 Introduction to Quantitative Economics9. Further Probability Topics

Department of EconomicsUniversity of Essex

6/8 December 2011

EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics


2/27

2/27

Outline

1 Joint and Marginal Probability Distributions

2 Expected values

3 Covariances

Reference: R. L. Thomas,Using Statistics in Economics,

McGraw-Hill, 2005, sections 6.16.3.



3/27

Joint and Marginal Probability Distributions 3/27

Often in statistics we need to compute parameter valuesrelating to two or more different populations.

Example:

Consider two cities, A and B.

Suppose we needed to know whether mean annual incomein city A was different from that in city B.Clearly we could, if necessary, take samples from each city.

But before we can draw any inference from such samples,

we need to introduce some more probability results.



4/27


Consider an experiment in which a four-sided die is rolled

twice.

The faces of the die contain the numbers 1, 2, 3, and 4.

The sample space for this experiment is shown in the

following table, and consists of 16 outcomes:

Second Roll1 2 3 4

1 1,1 1,2 1,3 1,4

First 2 2,1 2,2 2,3 2,4

Roll 3 3,1 3,2 3,3 3,4

4 4,1 4,2 4,3 4,4If the die is a fair one, then the probability of each outcome

will be 1/16.



5/27


Consider two random variables,XandY:

Xis the product of the numbers obtained on the two rolls;

Yis the absolute value of the difference between thenumbers obtained on the two rolls.

The sample space is as follows:

Roll X Y Roll X Y Roll X Y Roll X Y

1,1 1 0 2,1 2 1 3,1 3 2 4,1 4 3

1,2 2 1 2,2 4 0 3,2 6 1 4,2 8 2

1,3 3 2 2,3 6 1 3,3 9 0 4,3 12 1

1,4 4 3 2,4 8 2 3,4 12 1 4,4 16 0We can put the outcomes forXandYinto a separate table

along with their probabilities (recall that each entry above

has probability 1/16).


J i d M i l P b bili Di ib i /


6/27


Y\X 1 2 3 4 6 8 9 12 16

0 1/16 0 0 1/16 0 0 1/16 0 1/16

1 0 2/16 0 0 2/16 0 0 2/16 0

2 0 0 2/16 0 0 2/16 0 0 0

3 0 0 0 2/16 0 0 0 0 0

The values entered in the main body of this table are

known asjoint probabilities.

For example, the value in the X= 6column and the Y= 1row gives the joint probability of obtaining both X= 6andY= 1.

The probability is 2/16 because there are two outcomes,

(2,3) and (3,2), that result in X= 6andY= 1.



7/27



8/27


We denote the joint probability distribution byp(X, Y).

For example,

Pr(X= 3andY= 2) =p(3, 2),

Pr(X= 6andY= 1) =p(6, 1).

Notice that the sum of all the probabilities in the joint

distribution is equal to unity.

This will always be the case, provided we include all thepossible combinations of XandY.




9/27


For the moment, let us concentrate solely on the variable Y

(the difference between the two numbers obtained on the

two rolls of the die), that is, ignore the value of X.

The probability ofYtaking a particular value, regardless ofX, is known as amarginal probability.

We can compute marginal probabilities for Ydirectly from

the sample space.




10/27


Recall the sample space for the two-dice experiment:

Second Roll

1 2 3 4

1 1,1 1,2 1,3 1,4

First 2 2,1 2,2 2,3 2,4

Roll 3 3,1 3,2 3,3 3,4

4 4,1 4,2 4,3 4,4

For example:

The marginal probability that Y= 2is simply 4/16 since,ignoring the values for X, there are four outcomes of the

experiment that yieldY= 2(1,3; 2,4; 3,1; 4,2).Similarly, the marginal probability that Y= 1is 6/16, sincethere are six outcomes yielding Y= 1(1,2; 2,1; 2,3; 3,2;3,4; 4,3).




11/27


These marginal probabilities can be entered in the far

right-hand column of the joint distribution table and

constitute what is known as themarginal probability

distributionforY:

Y\X 1 2 3 4 6 8 9 12 16 g(Y)

0 1/16 0 0 1/16 0 0 1/16 0 1/16 4/16

1 0 2/16 0 0 2/16 0 0 2/16 0 6/162 0 0 2/16 0 0 2/16 0 0 0 4/16

3 0 0 0 2/16 0 0 0 0 0 2/16

We denote this marginal distribution by g(Y).

For example,

Pr(Y= 2) = g(2) = 4/16; Pr(Y= 1) =g(1) = 6/16.




12/27


Similarly, the marginal probabilities for Xcan be entered in

the bottom row of the table, denoted f(X

):

Y\X 1 2 3 4 6 8 9 12 16 g(Y)

0 1/16 0 0 1/16 0 0 1/16 0 1/16 4/16

1 0 2/16 0 0 2/16 0 0 2/16 0 6/16

2 0 0 2/16 0 0 2/16 0 0 0 4/16

3 0 0 0 2/16 0 0 0 0 0 2/16f(X) 1/16 2/16 2/16 3/16 2/16 2/16 1/16 2/16 1/16

For example,

Pr(X= 4) =f(4) = 3/16; Pr(X= 12) =f(12) = 2/16.

The sum of the probabilities in each marginal distribution

equals unity, as do those in the joint distribution.




13/27

g y

There is a clear relationship between the marginal and

joint distributions:

each row of joint probabilities sums to the marginalprobability in the right hand column;each column of joint probabilities sums to the marginalprobability in the bottom row.

Why is this so?A valueY= 2can be obtained either if the event(X= 3, Y= 2)occurs or if the event(X= 8, Y= 2)occurs.These two events are mutually exclusive and there are noother combinations ofXandYthat will yield Y= 2.

Hence, to find the marginal probabilityg(2)we need to addup the joint probabilities p(3, 2)and p(8, 2).But this is exactly what we do when we sum the jointprobabilities in theY= 2row to obtaing(2).




14/27

g y

Because of this relationship between joint and marginal

distributions, we do not normally have to refer back to the

sample space when computing marginal probability

distributions.

Provided we know the joint distribution forXandY, we can

find the marginal distributions just by summing rows and

columns of joint probabilities.


Expected values 15/27


15/27

Means and variances of marginal distributions can be

calculated in the same way as they were calculated for

ordinary probability distributions.

For example, using the values in the two-dice experiment,

E(X) =

Xf(X) = 6.25,

E(X2) =

X2f(X) = 56.25,

V(X) =

[X E(X)]2f(X) =E(X2) [E(X)]2 = 17.19,

E(Y) =

Yg(Y) = 1.25,

E(Y2) =

Y2g(Y) = 2.5,

V(Y) =

[YE(Y)]2g(Y) =E(Y2) [E(Y)]2 = 0.94.




16/27

But, when working with two random variables (XandY)

that are jointly distributed, we often need to computeexpectations involving both variables e.g. E(XY).

We have seen (e.g. Lecture 3) that, if Xhas probability

distributionp(X)and we want to compute E(X2), we weighteach possible value ofX2 byp(X)and add up the resulting

products:

E(X2) =

n

i=1

X2ip(Xi) =X2

1p(X1) +. . .+X2

np(Xn).

We do a similar type of thing with joint random variables,

usingp(X, Y)to weight all the possible outcomes.




17/27

So, to find E(XY), we use a double summation:

E(XY) =

XYp(X, Y).

Another way of writing this is:

E(XY) =

n

i=1

m

j=1

XiYjp(Xi, Yj),

whereX1, . . . ,Xn are the possible outcomes forX, andY1, . . . , Ym are the possible outcomes forY.

We therefore take all possible outcomes for XY, multiply

each one by the probability p(X, Y), and add them all up.




18/27

The simplest way of computing E(XY)is to write aboveeachp(X, Y)in the table the relevant value of XY.

These are entered as the numbers in the top right-hand

corner of the non-zero joint probabilities p(X, Y):

Y\X 1 2 3 4 6 8 9 12 16

0 1/160 0 0 1/160 0 0 1/160 0 1/160

1 0 2/162 0 0 2/166 0 0 2/1612 02 0 0 2/166 0 0 2/1616 0 0 0

3 0 0 0 2/1612 0 0 0 0 0

To compute E(XY)we therefore have (going row by row):

E(XY) = 22

16+6

2

16+12

2

16+6

2

16+16

2

16+12

2

16=

108

16 = 6.75.




19/27

Important: E(XY)= E(X)E(Y).

We can also compute expectations of other functions of X

andY, sayh(X, Y), in the same way:

E[h(X, Y)] =

h(X, Y)p(X, Y)

i.e. the sum of all possible values forh(X, Y)weighted bythe respective probabilitiesp(X, Y).

However, if the function is linear e.g. h(X, Y) =aX+ bY,then it is easier to use

E(aX+ bY) =aE(X) +bE(Y)

whereE(X)andE(Y)are calculated from the marginaldistributions.


Covariances 20/27


20/27

Thecovariancebetween two random variables, XandY,

measures the strength of any linear associationbetween

them.

The covariance is defined as

Cov(X, Y) =E[X E(X)][YE(Y)].

One way of computing a covariance is to use the formula

on the previous slide with h(X, Y) = [X E(X)][YE(Y)]:

Cov(X, Y) =

[X E(X)][YE(Y)]p(X, Y).

It is usually easier to obtain a covariance by using the

following relationship:

Cov(X, Y) =E(XY) E(X)E(Y).


Covariances 21/27


21/27

In the two-dice experiment, we had E(X) = 6.25andE(Y) = 1.25, and so

Cov(X, Y) =E[X 6.25][Y 1.25].

It measures the average value of the quantity

[X 6.25][Y 1.25]over very many trials of theexperiment.

Suppose there is a positive (linear) association between X

andY(i.e.Xrises as Y rises).

This implies that positive values of X 6.25(i.e. aboveaverage values ofX) will tend to coincide with positive

values ofY 1.25(i.e. above average values ofY).Similarly, negative values ofX 6.25(i.e. below averagevalues ofX) will tend to coincide with negative values of

Y 1.25(i.e. below average values ofY).


Covariances 22/27


22/27

Consequently, the product[X 6.25][Y 1.25]will tend toconsist of either two positive quantities multiplied together

or two negative quantities, and hence will tend to be

positive.

Since the covariance measures the average value of the

product over very many trials, it follows that thecovariance will be a positive quantity.

Moreover, the stronger the positive linear association

between XandY, the stronger will be the tendency for the

product[X 6.25][Y 1.25]to be positive, and the largerwill be the covariance.


Covariances 23/27


23/27

If instead there is a negative or inverse linear association

between XandY, there will be a tendency for positive

values ofX 6.25to coincide with negative values ofY 1.25.

Conversely, negative values of X 6.25will tend tocoincide with positive values for Y 1.25.

In such a situation, the product[X6

.25

][Y1

.25

]willnormally consist of one positive quantity and one negativequantity multiplied together, and hence will tend to be

negative.

Its average value, the covariance, will therefore be a

negative quantity.

Moreover, the stronger is the negative linear association

between XandY, the more negative is the covariance.


Covariances 24/27


24/27

In the two-dice example,

Cov(X, Y) = E(XY) E(X)E(Y)

= 6.75 (6.25)(1.25)

= 1

.0625

.

i.e. the covariance is negative.

Thus it appears that the larger is X, the product of the

numbers on the dice, the smaller is Y, their difference.


Covariances 25/27


25/27

A useful result is the following:

TheoremIfXandYare two random variables andaandbare constants,

then

V(aX+ bY) =a2V(X) +b2V(Y) + 2abCov(X, Y).

Two useful applications are whena= b= 1:

V(X+ Y) =V(X) +V(Y) + 2Cov(X, Y),

and whena= 1,b= 1:

V(X Y) =V(X) +V(Y) 2Cov(X, Y).


Covariances 26/27


26/27

A special case of interest is when Cov(X, Y) = 0, so that

the preceeding results reduce to:

V(aX+ bY) = a2V(X) +b2V(Y);

V(X+ Y) = V(X) +V(Y);

V(X Y) = V(X) +V(Y).

Note thatV(X Y)=V(X) V(Y)because the right handside would be negative if V(X)< V(Y)and the variance of

X Yhas to be positive!


Summary 27/27

Summary


27/27

Summary

Joint and marginal probability distributions

Expectations involving two random variables

Covariance.

Next week:

Correlation; tests about two populations.


further probability topics

Documents