econometric approaches to causal inference: difference-in-differences and instrumental variables...
Post on 01-Apr-2015
226 Views
Preview:
TRANSCRIPT
Econometric Approaches to Causal Econometric Approaches to Causal Inference:Inference:Difference-in-Differences and Difference-in-Differences and Instrumental VariablesInstrumental Variables
Graduate Methods Master ClassGraduate Methods Master Class
Department of Government, Harvard University Department of Government, Harvard University
February 25, 2005February 25, 2005
Overview: diff-in-diffs and IVOverview: diff-in-diffs and IV
Data Data Randomized experiment Randomized experiment Observational dataObservational data or natural experimentor natural experiment
Problem We cannot observe theProblem We cannot observe the OVB, selection bias, OVB, selection bias, counterfactual (what ifcounterfactual (what if simultaneous simultaneous
causalitycausality treatment group had nottreatment group had not received treatment)received treatment)
MethodMethod Difference-in-differences Difference-in-differences Instrumental Instrumental variablesvariables
Diff-in-diffs: basic ideaDiff-in-diffs: basic idea
Suppose we randomly assign treatment to some units Suppose we randomly assign treatment to some units
(or nature assigns treatment “as if” by random assignment) (or nature assigns treatment “as if” by random assignment)
To estimate the treatment effect, we could just compare the To estimate the treatment effect, we could just compare the
treated units before and after treatment treated units before and after treatment
However, we might pick up the effects of other factors that However, we might pick up the effects of other factors that
changed around the time of treatmentchanged around the time of treatment
Therefore, we use a control group to “difference out” theseTherefore, we use a control group to “difference out” these
confounding factors and isolate the treatment effectconfounding factors and isolate the treatment effect
Diff-in-diffs: without regressionDiff-in-diffs: without regression
One approach is simply to take the mean value of each One approach is simply to take the mean value of each group’s group’s
outcome before and after treatmentoutcome before and after treatment
Treatment groupTreatment group Control groupControl group
BeforeBefore T TBB C CBB
AfterAfter T TAA C CAA
and then calculate the “difference-in-differences” of the and then calculate the “difference-in-differences” of the means:means:
Treatment effect = (TTreatment effect = (TA A -- TTB B ) -) - (( CCA A -- CCB B ) )
Diff-in-diffs: with regressionDiff-in-diffs: with regression
We can get the same result in a regression framework We can get the same result in a regression framework (which (which
allows us to add regression controls, if needed):allows us to add regression controls, if needed):
yyii = = ββ00 + + ββ11 treat treatii + + ββ22 after afterii + + β β33 treat treatii*after*afterii + e + eii
wherewhere treat = 1 if in treatment group, = 0 if in control treat = 1 if in treatment group, = 0 if in control groupgroup
after = 1 if after treatment, = 0 if before after = 1 if after treatment, = 0 if before treatmenttreatment
The coefficient on the interaction term (The coefficient on the interaction term (ββ3 3 )) gives us the gives us the
difference-in-differences estimate of the treatment effectdifference-in-differences estimate of the treatment effect
Diff-in-diffs: with regressionDiff-in-diffs: with regression
To see this, plug zeros and ones into the regression To see this, plug zeros and ones into the regression equation:equation:
yyii = = ββ00 + + ββ11 treat treatii + + ββ22 after afterii + + β β33 treat treatii*after*afterii + e + eii
TreatmentTreatment Control Control GroupGroup Group Difference Group Difference
BeforeBefore ββ0 0 + + ββ11 ββ00 ββ11
AfterAfter ββ0 0 + + ββ1 1 + + ββ2 2 + + ββ33 ββ00 + + ββ22 ββ11 + + ββ33
DifferenceDifference ββ2 2 + + ββ33 ββ22 ββ33
Diff-in-diffs: exampleDiff-in-diffs: example
Card and Krueger (1994)Card and Krueger (1994)
What is the effect of increasing the minimum wage on What is the effect of increasing the minimum wage on employment at fast food restaurants?employment at fast food restaurants?
Confounding factor: national recessionConfounding factor: national recession
Treatment group = NJ Treatment group = NJ Before = Feb 92Before = Feb 92
Control group = PAControl group = PA After = Nov 92After = Nov 92
FTEFTEii = = ββ00 + + ββ11 NJ NJii + + ββ22 Nov92 Nov92ii + + ββ33 NJ NJii*Nov92*Nov92ii + e + eii
Diff-in-diffs: exampleDiff-in-diffs: example
FTEFTEii = = ββ00 + + ββ11 NJ NJii + + ββ22 Nov92 Nov92ii + + ββ33 NJ NJii*Nov92*Nov92ii + e + e
23.33 -2.89 -2.1623.33 -2.89 -2.16 2.75 2.75
FTEFTE
23.3323.33 Control group (PA)Control group (PA)
21.1721.17
20.4420.44 Treatment group (NJ)Treatment group (NJ) 21.03 21.03
TimeTime
Treatment effect of minimum wage increase = + 2.75 FTETreatment effect of minimum wage increase = + 2.75 FTE
Diff-in-diff-in-diffsDiff-in-diff-in-diffs
A difference-in-difference-in-differences (DDD) model allows us A difference-in-difference-in-differences (DDD) model allows us
to study the effect of treatment on different groupsto study the effect of treatment on different groups
If we are concerned that our estimated treatment effect might If we are concerned that our estimated treatment effect might
be spurious, a common robustness test is to introduce a be spurious, a common robustness test is to introduce a
comparison group that should not be affected by the treatmentcomparison group that should not be affected by the treatment
For example, if we want to know how welfare reform has For example, if we want to know how welfare reform has
affected labor force participation, we can use a DD modelaffected labor force participation, we can use a DD model
that takes advantage of policy variation across states, and then that takes advantage of policy variation across states, and then
use a DDD model to study how the policy has affected single use a DDD model to study how the policy has affected single
versus married womenversus married women
Diff-in-diffs: drawbacksDiff-in-diffs: drawbacks
Diff-in-diff estimation is only appropriate if treatment is Diff-in-diff estimation is only appropriate if treatment is randomrandom
- however, in the social sciences this method is usually - however, in the social sciences this method is usually applied applied
to data from natural experiments, raising questions to data from natural experiments, raising questions about about
whether treatment is truly random whether treatment is truly random
Also, diff-in-diffs typically use several years of serially-Also, diff-in-diffs typically use several years of serially-correlated correlated
data but ignore the resulting inconsistency of standard data but ignore the resulting inconsistency of standard errors errors
(see Bertrand, Duflo, and Mullainathan 2004)(see Bertrand, Duflo, and Mullainathan 2004)
IV: basic ideaIV: basic idea
Suppose we want to estimate a treatment effect using Suppose we want to estimate a treatment effect using
observational data observational data
The OLS estimator is biased and inconsistent (due to The OLS estimator is biased and inconsistent (due to correlation correlation
between regressor and error term) if there isbetween regressor and error term) if there is
- omitted variable biasomitted variable bias- selection biasselection bias- simultaneous causalitysimultaneous causality
If a direct solution (e.g. including the omitted variable) is not If a direct solution (e.g. including the omitted variable) is not
available, instrumental variables regression offers an available, instrumental variables regression offers an alternative alternative
way to obtain a consistent estimatorway to obtain a consistent estimator
IV: basic ideaIV: basic idea
Consider the following regression model:Consider the following regression model:
yyii = = ββ00 + + ββ11 X Xii + e+ eii
Variation in the endogenous regressor XVariation in the endogenous regressor X ii has two parts has two parts
- the part that is uncorrelated with the error (“good” the part that is uncorrelated with the error (“good” variation)variation)
- the part that is correlated with the error (“bad” variation)the part that is correlated with the error (“bad” variation)
The basic idea behind instrumental variables regression is The basic idea behind instrumental variables regression is to to
isolate the “good” variation and disregard the “bad” isolate the “good” variation and disregard the “bad” variationvariation
IV: conditions for a valid instrumentIV: conditions for a valid instrument
The first step is to identify a valid instrumentThe first step is to identify a valid instrument
A variable ZA variable Zii is a valid instrument for the endogenous is a valid instrument for the endogenous regressor regressor
XXii if it satisfies two conditions: if it satisfies two conditions:
1. Relevance:1. Relevance: corr (Zcorr (Zi i , X, Xii) ≠ 0) ≠ 0
2. Exogeneity:2. Exogeneity: corr (Zcorr (Zi i , e, eii) = 0) = 0
IV: two-stage least squaresIV: two-stage least squares
The most common IV method is two-stage least squares The most common IV method is two-stage least squares (2SLS)(2SLS)
Stage 1: Decompose XStage 1: Decompose Xii into the component that can be into the component that can be
predicted by Zpredicted by Zii and the problematic component and the problematic component
XXii = = 00 + + 11 Z Zii + + ii
Stage 2: Use the predicted value of XStage 2: Use the predicted value of Xii from the first-stage from the first-stage
regression to estimate its effect on Yregression to estimate its effect on Y ii
yyii = = 00 + + 11 X-hat X-hatii + + ii
Note: software packages like Stata perform the two stages in Note: software packages like Stata perform the two stages in a a
single regression, producing the correct standard errorssingle regression, producing the correct standard errors
IV: exampleIV: example
Levitt (1997): what is the effect of increasing the police forceLevitt (1997): what is the effect of increasing the police force
on the crime rate?on the crime rate?
This is a classic case of simultaneous causality (high crime This is a classic case of simultaneous causality (high crime areas areas
tend to need large police forces) resulting in an incorrectly-tend to need large police forces) resulting in an incorrectly-
signed (positive) coefficientsigned (positive) coefficient
To address this problem, Levitt uses the timing of mayoral To address this problem, Levitt uses the timing of mayoral and and
gubernatorial elections as an instrumental variablegubernatorial elections as an instrumental variable
Is this instrument valid?Is this instrument valid?
Relevance: police force increases in election yearsRelevance: police force increases in election years
Exogeneity: election cycles are pre-determinedExogeneity: election cycles are pre-determined
IV: exampleIV: example
Two-stage least squares:Two-stage least squares:
Stage 1: Decompose police hires into the component that can Stage 1: Decompose police hires into the component that can be predicted by the electoral cycle and the be predicted by the electoral cycle and the
problematic problematic componentcomponent
policepoliceii = = 00 + + 11 election electionii + + ii
Stage 2: Use the predicted value of policeStage 2: Use the predicted value of police ii from the first-stage from the first-stage regression to estimate its effect on crimeregression to estimate its effect on crime ii
crimecrimeii = = 00 + + 11 police-hat police-hatii + + ii
Finding:Finding: an increased police force reduces violent crime an increased police force reduces violent crime (but has little effect on property crime)(but has little effect on property crime)
IV: number of instrumentsIV: number of instruments
There must be at least as many instruments as There must be at least as many instruments as endogenous endogenous
regressorsregressors
Let k = number of endogenous regressorsLet k = number of endogenous regressors
m = number of instrumentsm = number of instruments
The regression coefficients areThe regression coefficients are
exactly identified if m=k exactly identified if m=k (OK)(OK)
overidentified if m>k overidentified if m>k (OK) (OK)
underidentified if m<k underidentified if m<k (not OK)(not OK)
IV: testing instrument relevanceIV: testing instrument relevance
How do we know if our instruments are valid? How do we know if our instruments are valid?
Recall our first condition for a valid instrument: Recall our first condition for a valid instrument:
1. Relevance: corr (Z1. Relevance: corr (Z i i , X, Xii) ≠ 0) ≠ 0
Stock and Watson’s rule of thumbStock and Watson’s rule of thumb: the first-stage F-statistic : the first-stage F-statistic
testing the hypothesis that the coefficients on the instruments testing the hypothesis that the coefficients on the instruments
are jointly zero should be at least 10 (for a single endogenous are jointly zero should be at least 10 (for a single endogenous
regressor)regressor)
A small F-statistic means the instruments are “weak” (they A small F-statistic means the instruments are “weak” (they
explain little of the variation in X) and the estimator is biased explain little of the variation in X) and the estimator is biased
IV: testing instrument exogeneityIV: testing instrument exogeneity
Recall our second condition for a valid instrument:Recall our second condition for a valid instrument:
2. Exogeneity: corr (Z2. Exogeneity: corr (Zi i , e, eii) = 0) = 0
If you have the same number of instruments and endogenous If you have the same number of instruments and endogenous
regressors, it is impossible to test for instrument exogeneityregressors, it is impossible to test for instrument exogeneity
But if you have more instruments than regressors:But if you have more instruments than regressors:
Overidentifying restrictions testOveridentifying restrictions test – regress the residuals from – regress the residuals from
the 2SLS regression on the instruments (and any exogenous the 2SLS regression on the instruments (and any exogenous
control variables) and test whether the coefficients on the control variables) and test whether the coefficients on the
instruments are all zeroinstruments are all zero
IV: drawbacksIV: drawbacks
It can be difficult to find an instrument that is both It can be difficult to find an instrument that is both relevant relevant
(not weak) and exogenous(not weak) and exogenous
Assessment of instrument exogeneity can be highly Assessment of instrument exogeneity can be highly subjectivesubjective
when the coefficients are exactly identifiedwhen the coefficients are exactly identified
IV can be difficult to explain to those who are unfamiliar IV can be difficult to explain to those who are unfamiliar with itwith it
SourcesSources
Stock and Watson, Stock and Watson, Introduction to EconometricsIntroduction to Econometrics
Bertrand, Duflo, and Mullainathan, “How Much Should We Trust Bertrand, Duflo, and Mullainathan, “How Much Should We Trust Differences-in-Differences Estimates?” Differences-in-Differences Estimates?” Quarterly Journal of EconomicsQuarterly Journal of Economics February 2004February 2004
Card and Krueger, "Minimum Wages and Employment: A Case Study of Card and Krueger, "Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania," the Fast Food Industry in New Jersey and Pennsylvania," American American Economic ReviewEconomic Review, September 1994, September 1994
Angrist and Krueger, “Instrumental Variables and the Search for Angrist and Krueger, “Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments,”Identification: From Supply and Demand to Natural Experiments,”Journal of Economic PerspectivesJournal of Economic Perspectives, Fall 2001, Fall 2001
Levitt, “Using Electoral Cycles in Police Hiring to Estimate the Effect ofLevitt, “Using Electoral Cycles in Police Hiring to Estimate the Effect ofPolice on Crme,” Police on Crme,” American Economic ReviewAmerican Economic Review, June 1997, June 1997
top related