christopher dougherty ec220 - introduction to econometrics (review chapter) slideshow: covariance,...
Post on 31-Mar-2015
242 Views
Preview:
TRANSCRIPT
Christopher Dougherty
EC220 - Introduction to econometrics (review chapter)Slideshow: covariance, covariance and variance rules, and correlation
Original citation:
Dougherty, C. (2012) EC220 - Introduction to econometrics (review chapter). [Teaching Resource]
© 2012 The Author
This version available at: http://learningresources.lse.ac.uk/141/
Available in LSE Learning Resources Online: May 2012
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms. http://creativecommons.org/licenses/by-sa/3.0/
http://learningresources.lse.ac.uk/
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The covariance of two random variables X and Y, often written XY, is defined to be the expected value of the product of their deviations from their population means.
1
Covariance
))(( ),cov( YXXY YXEYX
000
)()()()(
)()())((
YYXX
YX
YXYX
EYEEXE
YEXEYXE
If two variables are independent, their covariance is zero. To show this, start by rewriting the covariance as the product of the expected values of its factors.
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance
))(( ),cov( YXXY YXEYX
2
000
)()()()(
)()())((
YYXX
YX
YXYX
EYEEXE
YEXEYXE
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
We are allowed to do this because (and only because) X and Y are independent (see the earlier sequence on independence.
Covariance
))(( ),cov( YXXY YXEYX
3
000
)()()()(
)()())((
YYXX
YX
YXYX
EYEEXE
YEXEYXE
))(( ),cov( YXXY YXEYX
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The expected values of both factors are zero because E(X) = X and E(Y) = Y. E(X) = X and E(Y) = Y because X and Y are constants. Thus the covariance is zero.
Covariance
4
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
There are some rules that follow in a perfectly straightforward way from the definition of covariance, and since they are going to be used frequently in later chapters it is worthwhile establishing them immediately. First, the addition rule.
Covariance rules
1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W)
5
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Next, the multiplication rule, for cases where a variable is multiplied by a constant.
Covariance rules
1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W)
2. If Y = bZ, where b is a constant,
cov(X, Y) = bcov(X, Z)
6
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Finally, a primitive rule that is often useful.
Covariance rules
1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W).
2. If Y = bZ, where b is a constant,
cov(X, Y) = bcov(X, Z)
3. If Y = b, where b is a constant,
cov(X, Y) = 0
7
),(cov),(cov
][][
),(cov
WXVX
WXVXE
WVXE
YXEYX
WXVX
WVX
YX
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The proofs of the rules are straightforward. In each case the proof starts with the definition of cov(X, Y).
Covariance rules
1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W)
Proof:
Since Y = V + W, Y = V + W
8
),(cov),(cov
][][
),(cov
WXVX
WXVXE
WVXE
YXEYX
WXVX
WVX
YX
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
We now substitute for Y and re-arrange.
Covariance rules
1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W).
Proof:
Since Y = V + W, Y = V + W
9
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
This gives us the result.
),(cov),(cov
][][
),(cov
WXVX
WXVXE
WVXE
YXEYX
WXVX
WVX
YX
Covariance rules
1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W).
Proof:
Since Y = V + W, Y = V + W
10
),(cov
))((
))((
))((),(cov
ZXb
ZXbE
bbZXE
YXEYX
ZX
ZX
YX
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Next, the multiplication rule, for cases where a variable is multiplied by a constant. The Y terms have been replaced by the corresponding bZ terms.
Covariance rules
2. If Y = bZ,
cov(X, Y) = bcov(X, Z)
Proof:
Since Y = bZ, Y = bZ
11
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
b is a common factor and can be taken out of the expression, giving us the result that we want.
),(cov
))((
))((
))((),(cov
ZXb
ZXbE
bbZXE
YXEYX
ZX
ZX
YX
Covariance rules
2. If Y = bZ,cov(X, Y) = bcov(X, Z).
Proof:
Since Y = bZ, Y = bZ
12
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The proof of the third rule is trivial.
Covariance rules
3. If Y = b,cov(X, Y) = 0.
Proof:
Since Y = b, Y = b
0
0
))((
))((),(cov
E
bbXE
YXEYX
X
YX
13
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Here is an example of the use of the covariance rules. Suppose that Y is a linear function of Z and that we wish to use this to decompose cov(X, Y). We substitute for Y (first line) and then use covariance rule 1 (second line).
Example use of covariance rules
Suppose Y = b1 + b2Z
cov(X, Y) = cov(X, [b1 + b2Z])
= cov(X, b1) + cov(X, b2Z)
= 0 + cov(X, b2Z)
= b2cov(X, Z)
14
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Next we use covariance rule 3 (third line), and finally covariance rule 2 (fourth line).
Example use of covariance rules
Suppose Y = b1 + b2Z
cov(X, Y) = cov(X, [b1 + b2Z])
= cov(X, b1) + cov(X, b2Z)
= 0 + cov(X, b2Z)
= b2cov(X, Z)
15
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Corresponding to the covariance rules, there are parallel rules for variances. First the addition rule.
Variance rules
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
16
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Next, the multiplication rule, for cases where a variable is multiplied by a constant.
Variance rules
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
17
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
A third rule to cover the special case where Y is a constant.
Variance rules
1. If Y = V + W,var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
18
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Finally, it is useful to state a fourth rule. It depends on the first three, but it is so often of practical value that it is worth keeping it in mind separately.
Variance rules
1. If Y = V + W,var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
19
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The proofs of these rules can be derived from the results for covariances, noting that the variance of Y is equivalent to the covariance of Y with itself.
Variance rules
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
).,(cov
))((
)()(var 2
XX
XXE
XEX
XX
X
20
Variance rules
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
We start by replacing one of the Y arguments by V + W.
21
Variance rules
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W).
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
We then use covariance rule 1.
22
Variance rules
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W).
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
We now substitute for the other Y argument in both terms and use covariance rule 1 a second time.
23
Variance rules
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W).
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
This gives us the result. Note that the order of the arguments does not affect a covariance expression and hence cov(W, V) is the same as cov(V, W).
24
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The proof of the variance rule 2 is even more straightforward. We start by writing var(Y) as cov(Y, Y). We then substitute for both of the iYi arguments and take the b terms outside as common factors.
Variance rules
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z).
Proof:
var(Y) = cov(Y, Y) = cov(bZ, bZ)
= b2cov(Z, Z)
= b2var(Z).
25
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The third rule is trivial. We make use of covariance rule 3. Obviously if a variable is constant, it has zero variance.
Variance rules
3. If Y = b, where b is a constant,
var(Y) = 0
Proof:
var(Y) = cov(b, b) = 0
26
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The fourth variance rule starts by using the first. The second term on the right side is zero by covariance rule 3. The third is also zero by variance rule 3.
Variance rules
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
Proof:
var(Y) = var(V) + 2cov(V, b) + var(b)
= var(V)
27
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The intuitive reason for this result is easy to understand. If you add a constant to a variable, you shift its entire distribution by that constant. The expected value of the squared deviation from the mean is unaffected.
0
0 V + b
VV
V + b
Variance rules
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
Proof:
var(Y) = var(V) + 2cov(V, b) + var(b)
= var(V)
28
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
cov(X, Y) is unsatisfactory as a measure of association between two variables X and Y because it depends on the units of measurement of X and Y.
Correlation
22YX
XYXY
29
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
A better measure of association is the population correlation coefficient because it is dimensionless. The numerator possesses the units of measurement of both X and Y.
22YX
XYXY
Correlation
30
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The variances of X and Y in the denominator possess the squared units of measurement of those variables.
22YX
XYXY
Correlation
31
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
However, once the square root has been taken into account, the units of measurement are the same as those of the numerator, and the expression as a whole is unit free.
22YX
XYXY
Correlation
32
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If X and Y are independent, XY will be equal to zero because XY will be zero.
22YX
XYXY
Correlation
33
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If there is a positive association between them, XY, and hence XY, will be positive. If there is an exact positive linear relationship, XY will assume its maximum value of 1. Similarly, if there is a negative relationship, XY will be negative, with minimum value of –1.
22YX
XYXY
Correlation
34
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If X and Y are independent, XY will be equal to zero because XY will be zero.
22YX
XYXY
Correlation
35
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If there is a positive association between them, XY, and hence XY, will be positive. If there is an exact positive linear relationship, XY will assume its maximum value of 1.
22YX
XYXY
Correlation
36
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Similarly, if there is a negative relationship, XY will be negative, with minimum value of –1.
22YX
XYXY
Correlation
37
Copyright Christopher Dougherty 2011.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section R.4 of C. Dougherty,
Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oup.com/uk/orc/bin/9780199567089/.
Individuals studying econometrics on their own and who feel that they might
benefit from participation in a formal course should consider the London School
of Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
20 Elements of Econometrics
www.londoninternational.ac.uk/lse.
11.07.25
top related