christopher dougherty ec220 - introduction to econometrics (review chapter) slideshow: covariance,...

Christopher Dougherty

EC220 - Introduction to econometrics (review chapter)Slideshow: covariance, covariance and variance rules, and correlation

Original citation:

Dougherty, C. (2012) EC220 - Introduction to econometrics (review chapter). [Teaching Resource]

© 2012 The Author

This version available at: http://learningresources.lse.ac.uk/141/

Available in LSE Learning Resources Online: May 2012

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms. http://creativecommons.org/licenses/by-sa/3.0/

http://learningresources.lse.ac.uk/

http://learningresources.lse.ac.uk/141/

http://learningresources.lse.ac.uk/141/

http://creativecommons.org/licenses/by-sa/3.0/

COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION

The covariance of two random variables X and Y, often written XY, is defined to be the expected value of the product of their deviations from their population means.

1

Covariance

))(( ),cov( YXXY YXEYX

000

)()()()(

)()())((

YYXX

YX

YXYX

EYEEXE

YEXEYXE

If two variables are independent, their covariance is zero. To show this, start by rewriting the covariance as the product of the expected values of its factors.


Covariance


2

000

)()()()(

)()())((

YYXX

YX

YXYX

EYEEXE

YEXEYXE


We are allowed to do this because (and only because) X and Y are independent (see the earlier sequence on independence.

Covariance


3

000

)()()()(

)()())((

YYXX

YX

YXYX

EYEEXE

YEXEYXE



The expected values of both factors are zero because E(X) = X and E(Y) = Y. E(X) = X and E(Y) = Y because X and Y are constants. Thus the covariance is zero.

Covariance

4


There are some rules that follow in a perfectly straightforward way from the definition of covariance, and since they are going to be used frequently in later chapters it is worthwhile establishing them immediately. First, the addition rule.

Covariance rules

1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W)

5


Next, the multiplication rule, for cases where a variable is multiplied by a constant.

Covariance rules


2. If Y = bZ, where b is a constant,

cov(X, Y) = bcov(X, Z)

6


Finally, a primitive rule that is often useful.

Covariance rules

1. If Y = V + W,cov(X, Y) = cov(X, V) + cov(X,W).



3. If Y = b, where b is a constant,

cov(X, Y) = 0

7

),(cov),(cov

][][

),(cov

WXVX

WXVXE

WVXE

YXEYX

WXVX

WVX

YX


The proofs of the rules are straightforward. In each case the proof starts with the definition of cov(X, Y).

Covariance rules


Proof:

Since Y = V + W, Y = V + W

8

),(cov),(cov

][][

),(cov

WXVX

WXVXE

WVXE

YXEYX

WXVX

WVX

YX


We now substitute for Y and re-arrange.

Covariance rules


Proof:


9


This gives us the result.

),(cov),(cov

][][

),(cov

WXVX

WXVXE

WVXE

YXEYX

WXVX

WVX

YX

Covariance rules


Proof:


10

),(cov

))((

))((

))((),(cov

ZXb

ZXbE

bbZXE

YXEYX

ZX

ZX

YX


Next, the multiplication rule, for cases where a variable is multiplied by a constant. The Y terms have been replaced by the corresponding bZ terms.

Covariance rules

2. If Y = bZ,


Proof:

Since Y = bZ, Y = bZ

11


b is a common factor and can be taken out of the expression, giving us the result that we want.

),(cov

))((

))((

))((),(cov

ZXb

ZXbE

bbZXE

YXEYX

ZX

ZX

YX

Covariance rules

2. If Y = bZ,cov(X, Y) = bcov(X, Z).

Proof:

Since Y = bZ, Y = bZ

12


The proof of the third rule is trivial.

Covariance rules

3. If Y = b,cov(X, Y) = 0.

Proof:

Since Y = b, Y = b

0

0

))((

))((),(cov

E

bbXE

YXEYX

X

YX

13


Here is an example of the use of the covariance rules. Suppose that Y is a linear function of Z and that we wish to use this to decompose cov(X, Y). We substitute for Y (first line) and then use covariance rule 1 (second line).

Example use of covariance rules

Suppose Y = b1 + b2Z

cov(X, Y) = cov(X, [b1 + b2Z])

= cov(X, b1) + cov(X, b2Z)

= 0 + cov(X, b2Z)

= b2cov(X, Z)

14


Next we use covariance rule 3 (third line), and finally covariance rule 2 (fourth line).

Example use of covariance rules

Suppose Y = b1 + b2Z

cov(X, Y) = cov(X, [b1 + b2Z])

= cov(X, b1) + cov(X, b2Z)

= 0 + cov(X, b2Z)

= b2cov(X, Z)

15


Corresponding to the covariance rules, there are parallel rules for variances. First the addition rule.

Variance rules

1. If Y = V + W,

var(Y) = var(V) + var(W) + 2cov(V, W)


var(Y) = b2var(Z)


var(Y) = 0

4. If Y = V + b, where b is a constant,

var(Y) = var(V)

16


Next, the multiplication rule, for cases where a variable is multiplied by a constant.

Variance rules

1. If Y = V + W,



var(Y) = b2var(Z)


var(Y) = 0


var(Y) = var(V)

17


A third rule to cover the special case where Y is a constant.

Variance rules

1. If Y = V + W,var(Y) = var(V) + var(W) + 2cov(V, W)


var(Y) = b2var(Z)


var(Y) = 0


var(Y) = var(V)

18


Finally, it is useful to state a fourth rule. It depends on the first three, but it is so often of practical value that it is worth keeping it in mind separately.

Variance rules

1. If Y = V + W,var(Y) = var(V) + var(W) + 2cov(V, W)


var(Y) = b2var(Z)


var(Y) = 0


var(Y) = var(V)

19


The proofs of these rules can be derived from the results for covariances, noting that the variance of Y is equivalent to the covariance of Y with itself.

Variance rules

1. If Y = V + W,


Proof:

var(Y) = cov(Y, Y) = cov([V + W], Y)

= cov(V, Y) + cov(W, Y)

= cov(V, [V + W]) + cov(W, [V + W])

= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)

= var(V) + 2cov(V, W) + var(W)

).,(cov

))((

)()(var 2

XX

XXE

XEX

XX

X

20

Variance rules

1. If Y = V + W,


Proof:



= cov(V, [V + W]) + cov(W, [V + W])




We start by replacing one of the Y arguments by V + W.

21

Variance rules

1. If Y = V + W,

var(Y) = var(V) + var(W) + 2cov(V, W).

Proof:



= cov(V, [V + W]) + cov(W, [V + W])




We then use covariance rule 1.

22

Variance rules

1. If Y = V + W,


Proof:



= cov(V, [V + W]) + cov(W, [V + W])




We now substitute for the other Y argument in both terms and use covariance rule 1 a second time.

23

Variance rules

1. If Y = V + W,


Proof:



= cov(V, [V + W]) + cov(W, [V + W])




This gives us the result. Note that the order of the arguments does not affect a covariance expression and hence cov(W, V) is the same as cov(V, W).

24


The proof of the variance rule 2 is even more straightforward. We start by writing var(Y) as cov(Y, Y). We then substitute for both of the iYi arguments and take the b terms outside as common factors.

Variance rules


var(Y) = b2var(Z).

Proof:

var(Y) = cov(Y, Y) = cov(bZ, bZ)

= b2cov(Z, Z)

= b2var(Z).

25


The third rule is trivial. We make use of covariance rule 3. Obviously if a variable is constant, it has zero variance.

Variance rules


var(Y) = 0

Proof:

var(Y) = cov(b, b) = 0

26


The fourth variance rule starts by using the first. The second term on the right side is zero by covariance rule 3. The third is also zero by variance rule 3.

Variance rules


var(Y) = var(V)

Proof:

var(Y) = var(V) + 2cov(V, b) + var(b)

= var(V)

27


The intuitive reason for this result is easy to understand. If you add a constant to a variable, you shift its entire distribution by that constant. The expected value of the squared deviation from the mean is unaffected.

0

0 V + b

VV

V + b

Variance rules


var(Y) = var(V)

Proof:

var(Y) = var(V) + 2cov(V, b) + var(b)

= var(V)

28


cov(X, Y) is unsatisfactory as a measure of association between two variables X and Y because it depends on the units of measurement of X and Y.

Correlation

22YX

XYXY

29


A better measure of association is the population correlation coefficient because it is dimensionless. The numerator possesses the units of measurement of both X and Y.

22YX

XYXY

Correlation

30


The variances of X and Y in the denominator possess the squared units of measurement of those variables.

22YX

XYXY

Correlation

31


However, once the square root has been taken into account, the units of measurement are the same as those of the numerator, and the expression as a whole is unit free.

22YX

XYXY

Correlation

32


If X and Y are independent, XY will be equal to zero because XY will be zero.

22YX

XYXY

Correlation

33


If there is a positive association between them, XY, and hence XY, will be positive. If there is an exact positive linear relationship, XY will assume its maximum value of 1. Similarly, if there is a negative relationship, XY will be negative, with minimum value of –1.

22YX

XYXY

Correlation

34


If X and Y are independent, XY will be equal to zero because XY will be zero.

22YX

XYXY

Correlation

35


If there is a positive association between them, XY, and hence XY, will be positive. If there is an exact positive linear relationship, XY will assume its maximum value of 1.

22YX

XYXY

Correlation

36


Similarly, if there is a negative relationship, XY will be negative, with minimum value of –1.

22YX

XYXY

Correlation

37

Copyright Christopher Dougherty 2011.

These slideshows may be downloaded by anyone, anywhere for personal use.

Subject to respect for copyright and, where appropriate, attribution, they may be

used as a resource for teaching an econometrics course. There is no need to

refer to the author.

The content of this slideshow comes from Section R.4 of C. Dougherty,

Introduction to Econometrics, fourth edition 2011, Oxford University Press.

Additional (free) resources for both students and instructors may be

downloaded from the OUP Online Resource Centre

http://www.oup.com/uk/orc/bin/9780199567089/.

Individuals studying econometrics on their own and who feel that they might

benefit from participation in a formal course should consider the London School

of Economics summer school course

EC212 Introduction to Econometrics

http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx

or the University of London International Programmes distance learning course

20 Elements of Econometrics

www.londoninternational.ac.uk/lse.

11.07.25

http://www.oup.com/uk/orc/bin/9780199567089/

http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx

G:\www.londoninternational.ac.uk\lse

christopher dougherty ec220 - introduction to econometrics (review chapter) slideshow: covariance,...

Documents

covariance rules

covariance slide

correlation covariance

definition of covariance

y terms

z covx

correlation b

definition of covx