applied microeconometrics #02 regression analysis by ols ... · regression analysis by ols and 2sls...

Applied Microeconometrics #02Regression Analysis by OLS and 2SLS

Shigeki Kano

Osaka Prefecture University

August 2019

Shigeki Kano (OPU) Applied Microeconometrics #02 August 2019 1 / 32

Table of Contents

1 OLS Estimation

2 IV and 2SLS Estimation

3 Practical Issues in IV/2SLS Estimation

Leading list: Cameron and Trivedi (2005, Ch.4), Wooldridge (2010, Ch.4,Ch.5).

See also Angrist and Krueger (2001) and Angrist and Pischke (2008,Ch.3, Ch.4).


Section 1

OLS Estimation


OLS estimator

Consider linear regression with K coefficients under exogeneity:

yi = x′iβ+ εi, E(εi|xi) = 0, i = 1,2, . . . ,N. (1)

Recall the exogeneity implies orthogonality condition

E(εi|xi) = 0 ⇒ E(xiεi)(K×1)

= E[xi(yi−x′iβ)] = 0. (2)

So it should be

β(K×1)

= E(xix′i)−1 E(xiyi), (3)

provided that matrix

A(K×K)

= E(xix′i) (4)

is non-singular, i.e., has inverse A−1 = E(xix′i)−1.


The sample version of population moment (2);

1N ∑xiei

(K×1)

= 0 (5)

(where ei = yi−x′iβ is the i-th residual), producing a system of Kequations,

∑xi(yi−x′iβ) = 0 ⇔ ∑xix′iβ = ∑xiyi or X ′Xβ =X ′y, (6)

known as the normal equation in classical regression analysis.

Thus we get the ordinary least squares (OLS) as a method ofmoments (MM) estimator:

β(K×1)

=(∑xix

′i)−1

∑xiyi = (X ′X)−1X ′y, (7)

where ∑xix′i =X

′X is K×K and full rank.


Asymptotic properties of OLS

Consistency of OLS: Insert eq.(1) into the OLS of eq.(7) to get

β =(∑xix

′i)−1

∑xi(x′iβ+ εi) = β+

(∑xix

′i)−1

∑xiεi

= β+

(1N ∑xix

′i

)−1 1N ∑xiεi, (8)

where, by the law of large numbers,

1N ∑xix

′i

p→A= E(xix′i). (9)

∴ OLS β is consistent for β because

βp→ β+

(1N ∑xix

′i

)−1

p→ E(xix′i)−1=A−1

1N ∑xiεi

p→ E(xiεi)=0

= β+A−1×0 = β. (10)


Asymptotic normality: Modify equation (8) to

√N(β−β) =

(1N ∑xix

′i

)−1 1√N ∑xiεi. (11)

Here summand xiεi is a K×1 vector with mean E(xiεi) = 0 andvariance

B(K×K)

= Var(xiεi) = E[(xiεi)(xiεi)′] = E(ε2

i xix′i). (12)

⇒ Due to the central limit theorem, it follows that

1√N ∑xiεi

d→ N(0,B). (13)

Thus we get (owing to the Slutsky theorem)

√N(β−β) =

(1N ∑xix

′i

)−1

p→A−1

1√N ∑xiεi

d→ N(0,B)

d→ N(0,A−1BA−1). (14)


Proposition 1

Under the exogeneity (or at least orthogonality), the OLS is consistent for

β (plim β = β or βp→ β) and asymptotically normal, where the limiting

distribution is given by

√N(β−β) d→ N(0,Ω), Ω =A−1BA−1. (15)

Proof. See above.

Taking expression (15) as an approximation, we have the asymptoticdistribution of OLS,

βa∼ N(β,V ), V = Avar(β) =

1N

Ω =1NA−1BA−1, (16)

where V is the K×K asymptotic variance matrix.

Note: Asymptotic variance V depends on sample size N but limitingvariance Ω does not.


Asymptotic variance estimation and standard errors

We want to use expression (16) for hypothesis testing and computingstandard errors of OLS β.

Using OLS residual ei = yi−x′iβ, estimate A= E(xix′i) and

B = E(ε2i xixi) by

A=1N ∑xix

′i =

1NX ′X, B =

1N ∑e2

i xix′i. (17)

Then V is estimated by the heteroskedasticity-consistent (robust)covariance matrix by White (1980):

V =1NA−1BA−1 = (X ′X)−1

∑e2i xix

′i(X

′X)−1. (18)

The j-th diagonal element of V produces the asymptotic standarderror of corresponding j-th estimate in OLS β.


Semiparametric efficiency

Chamberlain (1987) derives the lower bound for the variance matrix ofestimators defined by unconditional moments; called the semiparametricefficiency bound (SPEB):

Proposition 2

Under the orthogonality condition (unconditional moment) E(xiεi) = 0,variance matrix Ω in eq.(15) reaches the SPEB:

SPEB = Ω =A−1BA−1. (19)

Proof. Apply the general formula by Chamberlain (1987) to the OLS.

∴ The OLS is semiparametrically efficient under orthogonality.

However, if starting from exogeneity (conditional moment)E(εi|xi) = 0, the OLS is suboptimal. ⇒ We will revisit this issue.


Section 2

IV and 2SLS Estimation


Endogeneity problem

In regression model (1), partition regressors so that

xi =

[xi1xi2

]⇔ yi = x

′i1β1 +x

′i2β2 + εi (20)

and suppose xi violates exogeneity in the following sense.

E(xiεi) =

[E(xi1εi)E(xi2εi)

]=

[γ0

]6= 0. (21)

Then xi (particularly x1i) is called endogenous variables.

If xi is endogenous, the OLS is inconsistent because

plim β = β+A−1[γ0

]= β+BIAS 6= β. (22)

Here second term BIAS is an endogeneity bias.


Estimation by instrumental variables

In the presence of endogeneity, suppose we observe a new set of variablez1i with the same dimension as endogenous x1i and create new vector

zi(K×1)

=

[zi1xi2

], (23)

where

E(ziεi)(K×1)

=

[E(zi1εi)E(xi2εi)

]=

[00

]= 0. (24)

Then zi is exogenous and called the instrumental variable.

Particularly zi1 is called the excluded instruments as they do not showup on the regression as regressors.

∴ Importantly, we are not directly interested in zi1.


It follows from condition (24) and εi = yi−x′iβ that

E[zi(yi−x′iβ)] = 0 ⇔ E(zix′i)β = E(xiy′i). (25)

.

Thus, as long as K×K matrix

Azx(K×K)

= E(zix′i) (26)

is non-singular, we have

β = E(zix′i)−1 E(xiy′i). (27)


On the other hand, the sample version of condition (24);

1N ∑ziei =

1N ∑zi(yi−x′iβIV) = 0 ⇔ ∑zix

′iβ = ∑ziyi

or Z ′Xβ =Z ′y. (28)

Assume ∑zix′i =Z

′X is non-singular. Then we have instrumentalvariables (IV) estimator

βIV =(∑zix

′i)−1

∑ziyi = (Z ′X)−1Z ′y. (29)

Figure 1 describes the idea of IV: Instrument zi causes exogenousvariations on xi independent from εi so that the effect of xi on yi isidentifeid.

∴ The IV uses (a) exogenous but (usually) uninteresting zi to identifythe effect of (b) endogenous but interesting xi on yi.


xi

yi

εi

zi

Figure 1: Idea of instrumental variables


Over-identification and GIV solution

Now suppose the length of zi is longer than that of xi. ⇒ Then thecurrent moment condition,

E(ziεi)(L×1)

= E[zi(yi−x′iβ)] = 0, L > K, (30)

over-identifies β.

When L = K, the moment condition just-identifies β.

Under the over-identification, because

Azx(L×K)

= E(zixi) nor ∑zix′i

(L×K)

=Z ′X (31)

are no longer square matrices, no inverse defined on them. ⇒ So theIV estimator in eq.(29) is not defined.

... If we are just interested in consistency, discard some instrumentsto make zi have the same dimension as β; which seems inefficient.


How to us all the instruments efficiently?⇒ Let Π be a L×K matrix suchthat Π→Π as N→ ∞ and, using Π, create generalized instruments to“condense” zi:

zi∗(K×1)

= Π(K×L)

′zi

(L×1)(32)

or, in matrix, z′1∗z′2∗

...z′N∗

=

z′1z′2...z′N

Π ⇔ Z∗ =ZΠ. (33)

Taking Z∗ as just-identifying instruments, we define the generalizedinstrumental variables (GIV) estimator:

βGIV = (Z ′∗X)−1Z∗y = (Π′Z ′X)−1Π′Z ′y. (34)


Two stage least squares

Important remark: Because how to make Π is under our control, there area wide variety of GIV estimators. ⇒ Particularly, set

Π = (Z ′Z)−1Z ′X. (35)

Then Z∗ turns to the fitted value of X by regressing it on Z, i.e.,

Z∗ = X =Z(Z ′Z)−1Z ′X = PzX, (36)

where Pz is the projection matrix.

This GIV is called the two stage least squares (2SLS) estimator,

β2SLS = (X ′X)−1X ′y

= [X ′Z(Z ′Z)−1Z ′X]−1X ′Z(Z ′Z)−1Z ′y. (37)


Rewrite eq.(37) in terms of Pz (symmetric idempotent) as

β2SLS = (X ′PzX)−1X ′Pzy = (X ′PzP′zX)−1X ′Pzy

= (X ′X)−1X ′y, (38)

identical to the OLS regression of y on fitted value X. ∴ There are threeways to compute the 2SLS.

1 Equation (38): (i) Regress X on Z and get X; ⇒ (ii) regress y onX to get β2SLS, from which the name, 2SLS, originates.

2 The first line of eq.(37): (i) Regress X on Z and get X; ⇒ (ii) RunIV regression of y on X using X as instruments to get β2SLS.

3 Get β2SLS as what we can see in the second line of eq.(37) (as does aregular statistical software.)


Note: In the case of just-identification, L = K and eq.(37) is reduced to

β2SLS = (Z ′X)−1(Z ′Z)(X ′Z)−1X ′Z(Z ′Z)−1Z ′y

= (Z ′X)−1Z ′y = βIV. (39)

∴ The IV estimator is a special case of 2SLS.

Further, if E(xiεi) = 0 is true, we can use X as instruments for X.Then the IV turns to OLS.

βIV = (X ′X)−1X ′y = β. (40)


Asymptotic properties of 2SLS

Proposition 2

The 2SLS estimator is consistent and asymptotically normal:

√N(β2SLS−β)

d→ N(0,Ω2SLS),

Ω2SLS = (A′zxA−1zz Azx)

−1A′zxA−1zz BzA

−1zz Azx(A

′zxA

−1zz Azx)

−1. (41)

Proof. It is virtually indifferent from that for OLS so we skip the proof.

The asymptotic variance,

Avar(β2SLS) = V2SLS =1N

Ω2SLS, (42)

is estimated in a similar manner to the OLS case.

We will consider the efficiency of 2SLS in LN#03.


Section 3

Practical Issues in IV/2SLS Estimation


Validity and relevance of instruments

Qualifications for zi to be instruments.

Instrument validity: zi should not correlate with εi,

E(ziεi) = 0. (43)

⇒ Otherwise the IV/2SLS suffers from endogeneity bias as does OLS.

Instrument relevance: zi should correlate with xi,

Azx = E(zix′i) 6=O. (44)

⇒ More precisely, Azx should be “sufficently” non-singular and stableinverse A−1

zx is warranted.

The relevance is testable (see below) but validity is not.

See Murray (2006) for the sensible use of IV/2SLS.


Common misunderstandings: Showing the Hausman or Over-identification(OI) test results as a proof of valid instruments.

Hausman test for testing H0 : β = β2SLS. ... We assume (not test)that the instrument is valid and relevant.

OI test for testing H0 : E(ziεi) = 0 where L > K. ... We assume (nottest) that at least K instruments in over-identifying zi are valid andrelevant. ∴ The degree of freedom is L−K.

∴ Neither the Hausman nor the OI can test the validity ofinstruments.

We need convincing, non-statistical explanations to defend thevalidity of zi we employ.


Weak instruments: Symptom and diagnosis

The instruments weakly correlated to endogenous regressors (i.e.,irrelevant) are called weak instruments.

Extreme example: Random numbers generated by a PC is perfectlyindependent to errors but so is regressors. ∴ Irrelvant as IV.

Because the IV solves

Z ′XβIV =Z ′y ⇔ βIV = (Z ′X)−1Z ′y (45)

(GIV, too), roughly speaking, weak instruments cause a problem ofdividing a number by zero. ... Very similar to multicollinearity.

It seems that the use of over-identifying instruments covers theweakness. ⇒ Actually not. ... Bound et al. (1995) show that usingmany weak instruments to run the 2SLS worsens the situation.


Express the j-th endogenous regressor in reduced form

xi j = z′iπ j + v ji = z

′1iπ1 j +x

′2iπ2 j + v ji, (46)

the “first stage” of the 2SLS.

∴ Excluded instruments z1i should have a strong explanatory powereven after controlling for x2i. ⇒ Otherwise x ji is almost spanned byx2i, causing a near multicollinearity.

How to check if zi1 is weak or not? ⇒ A guideline by Staiger andStock (1997): If the F test statistic on the joint significance of

H0 : π1 j = 0 (47)

exceeds 10, probably it is safe to use zi1 as instruments.

Note: Other things being equal, the above F value tends to begreater when zi1 and xi2 are independent.


Natural experiments

In modern applications of IV/2SLS, the use of natural experiments asinstruments draws more attention.

Use the institutional and cultural features/changes affecting individualdecisions as instruments to defence its validity.

Now a natural experiment is almost synonym for IVs.

However, sometimes, particularly after controlling for many variables,such instruments turn to be weak.

∴ It is a common practice to check the relevance (strength) ofinstruments by carefully looking at the first stage, reduced formresults of equation (46).


Example 1

Angrist and Evans (1998): OLS and IV of the effect of children on femalelabor supply, focusing on the US. women who have at least two kids,

labori = β1 +β2childi + controli + εi. (48)

Child dummy childi indicates if i has more than two kids, apparentlyendogenous.

IV relevance: Couples prefer a “sex mix” of kids and, if the first twoare male/male or female/female, they tend to try to get another baby.

IV validity: The sex is random and independent from εi. ⇒ The“same sex dummy” is a valid instrument for childi.

Table 1: The OLS seems to overestimate the adverse effect ofchildren on female labor supply.


The effect of OLS IVthe 3rd birth on Coeff. SE. Coeff. SE.

Worked for pay -0.147 0.002 -0.104 0.024Weeks worked -8.25 0.09 -5.76 1.15Hours/week -6.39 0.07 -3.94 0.96

Control variables Yes Yes

Table 1: Extraction from Table 8 of Angrist and Evans (1998)


References I

Angrist, J. D. and W. N. Evans (1998). Children and their parents’ laborsupply: Evidence from exogenous variation in family size. AmericanEconomic Review 88(3), 450–77.

Angrist, J. D. and A. B. Krueger (2001). Instrumental variables and thesearch for identification: From supply and demand to naturalexperiments. Journal of Economic perspectives 15(4), 69–85.

Angrist, J. D. and J.-S. Pischke (2008). Mostly Harmless Econometrics:An Empiricist’s Companion. Princeton University Press.

Bound, J., D. A. Jaeger, and R. M. Baker (1995). Problems withinstrumental variables estimation when the correlation between theinstruments and the endogenous explanatory variable is weak. Journal ofthe American Statistical Association 90(430), 443–450.

Cameron, A. C. and P. K. Trivedi (2005). Microeconometrics: Methodsand Applications. Cambridge University Press.


References II

Chamberlain, G. (1987). Asymptotic efficiency in estimation withconditional moment restrictions. Journal of Econometrics 34(3),305–334.

Murray, M. P. (2006). Avoiding invalid instruments and coping with weakinstruments. Journal of economic Perspectives 20(4), 111–132.

Staiger, D. O. and J. H. Stock (1997). Instrumental variables regressionwith weak instruments. Econometrica 65(3), 557–586.

White, H. (1980). A heteroskedasticity-consistent covariance matrixestimator and a direct test for heteroskedasticity. Econometrica 48(4),817–838.

Wooldridge, J. M. (2010). Econometric Analysis of Cross Section andPanel Data (second ed.). MIT Press.


applied microeconometrics #02 regression analysis by ols ... · regression analysis by ols and 2sls...

Documents