Download - Further Probability Topics
-
8/11/2019 Further Probability Topics
1/27
1/27
EC114 Introduction to Quantitative Economics9. Further Probability Topics
Department of EconomicsUniversity of Essex
6/8 December 2011
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
-
8/11/2019 Further Probability Topics
2/27
2/27
Outline
1 Joint and Marginal Probability Distributions
2 Expected values
3 Covariances
Reference: R. L. Thomas,Using Statistics in Economics,
McGraw-Hill, 2005, sections 6.16.3.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
-
8/11/2019 Further Probability Topics
3/27
Joint and Marginal Probability Distributions 3/27
Often in statistics we need to compute parameter valuesrelating to two or more different populations.
Example:
Consider two cities, A and B.
Suppose we needed to know whether mean annual incomein city A was different from that in city B.Clearly we could, if necessary, take samples from each city.
But before we can draw any inference from such samples,
we need to introduce some more probability results.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
-
8/11/2019 Further Probability Topics
4/27
Joint and Marginal Probability Distributions 4/27
Consider an experiment in which a four-sided die is rolled
twice.
The faces of the die contain the numbers 1, 2, 3, and 4.
The sample space for this experiment is shown in the
following table, and consists of 16 outcomes:
Second Roll1 2 3 4
1 1,1 1,2 1,3 1,4
First 2 2,1 2,2 2,3 2,4
Roll 3 3,1 3,2 3,3 3,4
4 4,1 4,2 4,3 4,4If the die is a fair one, then the probability of each outcome
will be 1/16.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
-
8/11/2019 Further Probability Topics
5/27
Joint and Marginal Probability Distributions 5/27
Consider two random variables,XandY:
Xis the product of the numbers obtained on the two rolls;
Yis the absolute value of the difference between thenumbers obtained on the two rolls.
The sample space is as follows:
Roll X Y Roll X Y Roll X Y Roll X Y
1,1 1 0 2,1 2 1 3,1 3 2 4,1 4 3
1,2 2 1 2,2 4 0 3,2 6 1 4,2 8 2
1,3 3 2 2,3 6 1 3,3 9 0 4,3 12 1
1,4 4 3 2,4 8 2 3,4 12 1 4,4 16 0We can put the outcomes forXandYinto a separate table
along with their probabilities (recall that each entry above
has probability 1/16).
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
J i d M i l P b bili Di ib i /
-
8/11/2019 Further Probability Topics
6/27
Joint and Marginal Probability Distributions 6/27
Y\X 1 2 3 4 6 8 9 12 16
0 1/16 0 0 1/16 0 0 1/16 0 1/16
1 0 2/16 0 0 2/16 0 0 2/16 0
2 0 0 2/16 0 0 2/16 0 0 0
3 0 0 0 2/16 0 0 0 0 0
The values entered in the main body of this table are
known asjoint probabilities.
For example, the value in the X= 6column and the Y= 1row gives the joint probability of obtaining both X= 6andY= 1.
The probability is 2/16 because there are two outcomes,
(2,3) and (3,2), that result in X= 6andY= 1.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
-
8/11/2019 Further Probability Topics
7/27
Joint and Marginal Probability Distributions 8/27
-
8/11/2019 Further Probability Topics
8/27
Joint and Marginal Probability Distributions 8/27
We denote the joint probability distribution byp(X, Y).
For example,
Pr(X= 3andY= 2) =p(3, 2),
Pr(X= 6andY= 1) =p(6, 1).
Notice that the sum of all the probabilities in the joint
distribution is equal to unity.
This will always be the case, provided we include all thepossible combinations of XandY.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Joint and Marginal Probability Distributions 9/27
-
8/11/2019 Further Probability Topics
9/27
Joint and Marginal Probability Distributions 9/27
For the moment, let us concentrate solely on the variable Y
(the difference between the two numbers obtained on the
two rolls of the die), that is, ignore the value of X.
The probability ofYtaking a particular value, regardless ofX, is known as amarginal probability.
We can compute marginal probabilities for Ydirectly from
the sample space.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Joint and Marginal Probability Distributions 10/27
-
8/11/2019 Further Probability Topics
10/27
Joint and Marginal Probability Distributions 10/27
Recall the sample space for the two-dice experiment:
Second Roll
1 2 3 4
1 1,1 1,2 1,3 1,4
First 2 2,1 2,2 2,3 2,4
Roll 3 3,1 3,2 3,3 3,4
4 4,1 4,2 4,3 4,4
For example:
The marginal probability that Y= 2is simply 4/16 since,ignoring the values for X, there are four outcomes of the
experiment that yieldY= 2(1,3; 2,4; 3,1; 4,2).Similarly, the marginal probability that Y= 1is 6/16, sincethere are six outcomes yielding Y= 1(1,2; 2,1; 2,3; 3,2;3,4; 4,3).
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Joint and Marginal Probability Distributions 11/27
-
8/11/2019 Further Probability Topics
11/27
Joint and Marginal Probability Distributions 11/27
These marginal probabilities can be entered in the far
right-hand column of the joint distribution table and
constitute what is known as themarginal probability
distributionforY:
Y\X 1 2 3 4 6 8 9 12 16 g(Y)
0 1/16 0 0 1/16 0 0 1/16 0 1/16 4/16
1 0 2/16 0 0 2/16 0 0 2/16 0 6/162 0 0 2/16 0 0 2/16 0 0 0 4/16
3 0 0 0 2/16 0 0 0 0 0 2/16
We denote this marginal distribution by g(Y).
For example,
Pr(Y= 2) = g(2) = 4/16; Pr(Y= 1) =g(1) = 6/16.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Joint and Marginal Probability Distributions 12/27
-
8/11/2019 Further Probability Topics
12/27
Joint and Marginal Probability Distributions 12/27
Similarly, the marginal probabilities for Xcan be entered in
the bottom row of the table, denoted f(X
):
Y\X 1 2 3 4 6 8 9 12 16 g(Y)
0 1/16 0 0 1/16 0 0 1/16 0 1/16 4/16
1 0 2/16 0 0 2/16 0 0 2/16 0 6/16
2 0 0 2/16 0 0 2/16 0 0 0 4/16
3 0 0 0 2/16 0 0 0 0 0 2/16f(X) 1/16 2/16 2/16 3/16 2/16 2/16 1/16 2/16 1/16
For example,
Pr(X= 4) =f(4) = 3/16; Pr(X= 12) =f(12) = 2/16.
The sum of the probabilities in each marginal distribution
equals unity, as do those in the joint distribution.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Joint and Marginal Probability Distributions 13/27
-
8/11/2019 Further Probability Topics
13/27
g y
There is a clear relationship between the marginal and
joint distributions:
each row of joint probabilities sums to the marginalprobability in the right hand column;each column of joint probabilities sums to the marginalprobability in the bottom row.
Why is this so?A valueY= 2can be obtained either if the event(X= 3, Y= 2)occurs or if the event(X= 8, Y= 2)occurs.These two events are mutually exclusive and there are noother combinations ofXandYthat will yield Y= 2.
Hence, to find the marginal probabilityg(2)we need to addup the joint probabilities p(3, 2)and p(8, 2).But this is exactly what we do when we sum the jointprobabilities in theY= 2row to obtaing(2).
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Joint and Marginal Probability Distributions 14/27
-
8/11/2019 Further Probability Topics
14/27
g y
Because of this relationship between joint and marginal
distributions, we do not normally have to refer back to the
sample space when computing marginal probability
distributions.
Provided we know the joint distribution forXandY, we can
find the marginal distributions just by summing rows and
columns of joint probabilities.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Expected values 15/27
-
8/11/2019 Further Probability Topics
15/27
Means and variances of marginal distributions can be
calculated in the same way as they were calculated for
ordinary probability distributions.
For example, using the values in the two-dice experiment,
E(X) =
Xf(X) = 6.25,
E(X2) =
X2f(X) = 56.25,
V(X) =
[X E(X)]2f(X) =E(X2) [E(X)]2 = 17.19,
E(Y) =
Yg(Y) = 1.25,
E(Y2) =
Y2g(Y) = 2.5,
V(Y) =
[YE(Y)]2g(Y) =E(Y2) [E(Y)]2 = 0.94.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Expected values 16/27
-
8/11/2019 Further Probability Topics
16/27
But, when working with two random variables (XandY)
that are jointly distributed, we often need to computeexpectations involving both variables e.g. E(XY).
We have seen (e.g. Lecture 3) that, if Xhas probability
distributionp(X)and we want to compute E(X2), we weighteach possible value ofX2 byp(X)and add up the resulting
products:
E(X2) =
n
i=1
X2ip(Xi) =X2
1p(X1) +. . .+X2
np(Xn).
We do a similar type of thing with joint random variables,
usingp(X, Y)to weight all the possible outcomes.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Expected values 17/27
-
8/11/2019 Further Probability Topics
17/27
So, to find E(XY), we use a double summation:
E(XY) =
XYp(X, Y).
Another way of writing this is:
E(XY) =
n
i=1
m
j=1
XiYjp(Xi, Yj),
whereX1, . . . ,Xn are the possible outcomes forX, andY1, . . . , Ym are the possible outcomes forY.
We therefore take all possible outcomes for XY, multiply
each one by the probability p(X, Y), and add them all up.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Expected values 18/27
-
8/11/2019 Further Probability Topics
18/27
The simplest way of computing E(XY)is to write aboveeachp(X, Y)in the table the relevant value of XY.
These are entered as the numbers in the top right-hand
corner of the non-zero joint probabilities p(X, Y):
Y\X 1 2 3 4 6 8 9 12 16
0 1/160 0 0 1/160 0 0 1/160 0 1/160
1 0 2/162 0 0 2/166 0 0 2/1612 02 0 0 2/166 0 0 2/1616 0 0 0
3 0 0 0 2/1612 0 0 0 0 0
To compute E(XY)we therefore have (going row by row):
E(XY) = 22
16+6
2
16+12
2
16+6
2
16+16
2
16+12
2
16=
108
16 = 6.75.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Expected values 19/27
-
8/11/2019 Further Probability Topics
19/27
Important: E(XY)= E(X)E(Y).
We can also compute expectations of other functions of X
andY, sayh(X, Y), in the same way:
E[h(X, Y)] =
h(X, Y)p(X, Y)
i.e. the sum of all possible values forh(X, Y)weighted bythe respective probabilitiesp(X, Y).
However, if the function is linear e.g. h(X, Y) =aX+ bY,then it is easier to use
E(aX+ bY) =aE(X) +bE(Y)
whereE(X)andE(Y)are calculated from the marginaldistributions.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Covariances 20/27
-
8/11/2019 Further Probability Topics
20/27
Thecovariancebetween two random variables, XandY,
measures the strength of any linear associationbetween
them.
The covariance is defined as
Cov(X, Y) =E[X E(X)][YE(Y)].
One way of computing a covariance is to use the formula
on the previous slide with h(X, Y) = [X E(X)][YE(Y)]:
Cov(X, Y) =
[X E(X)][YE(Y)]p(X, Y).
It is usually easier to obtain a covariance by using the
following relationship:
Cov(X, Y) =E(XY) E(X)E(Y).
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Covariances 21/27
-
8/11/2019 Further Probability Topics
21/27
In the two-dice experiment, we had E(X) = 6.25andE(Y) = 1.25, and so
Cov(X, Y) =E[X 6.25][Y 1.25].
It measures the average value of the quantity
[X 6.25][Y 1.25]over very many trials of theexperiment.
Suppose there is a positive (linear) association between X
andY(i.e.Xrises as Y rises).
This implies that positive values of X 6.25(i.e. aboveaverage values ofX) will tend to coincide with positive
values ofY 1.25(i.e. above average values ofY).Similarly, negative values ofX 6.25(i.e. below averagevalues ofX) will tend to coincide with negative values of
Y 1.25(i.e. below average values ofY).
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Covariances 22/27
-
8/11/2019 Further Probability Topics
22/27
Consequently, the product[X 6.25][Y 1.25]will tend toconsist of either two positive quantities multiplied together
or two negative quantities, and hence will tend to be
positive.
Since the covariance measures the average value of the
product over very many trials, it follows that thecovariance will be a positive quantity.
Moreover, the stronger the positive linear association
between XandY, the stronger will be the tendency for the
product[X 6.25][Y 1.25]to be positive, and the largerwill be the covariance.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Covariances 23/27
-
8/11/2019 Further Probability Topics
23/27
If instead there is a negative or inverse linear association
between XandY, there will be a tendency for positive
values ofX 6.25to coincide with negative values ofY 1.25.
Conversely, negative values of X 6.25will tend tocoincide with positive values for Y 1.25.
In such a situation, the product[X6
.25
][Y1
.25
]willnormally consist of one positive quantity and one negativequantity multiplied together, and hence will tend to be
negative.
Its average value, the covariance, will therefore be a
negative quantity.
Moreover, the stronger is the negative linear association
between XandY, the more negative is the covariance.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Covariances 24/27
-
8/11/2019 Further Probability Topics
24/27
In the two-dice example,
Cov(X, Y) = E(XY) E(X)E(Y)
= 6.75 (6.25)(1.25)
= 1
.0625
.
i.e. the covariance is negative.
Thus it appears that the larger is X, the product of the
numbers on the dice, the smaller is Y, their difference.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Covariances 25/27
-
8/11/2019 Further Probability Topics
25/27
A useful result is the following:
TheoremIfXandYare two random variables andaandbare constants,
then
V(aX+ bY) =a2V(X) +b2V(Y) + 2abCov(X, Y).
Two useful applications are whena= b= 1:
V(X+ Y) =V(X) +V(Y) + 2Cov(X, Y),
and whena= 1,b= 1:
V(X Y) =V(X) +V(Y) 2Cov(X, Y).
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Covariances 26/27
-
8/11/2019 Further Probability Topics
26/27
A special case of interest is when Cov(X, Y) = 0, so that
the preceeding results reduce to:
V(aX+ bY) = a2V(X) +b2V(Y);
V(X+ Y) = V(X) +V(Y);
V(X Y) = V(X) +V(Y).
Note thatV(X Y)=V(X) V(Y)because the right handside would be negative if V(X)< V(Y)and the variance of
X Yhas to be positive!
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics
Summary 27/27
Summary
-
8/11/2019 Further Probability Topics
27/27
Summary
Joint and marginal probability distributions
Expectations involving two random variables
Covariance.
Next week:
Correlation; tests about two populations.
EC114 Introduction to Quantitative Economics 9. Fur ther Probability Topics