synthetic control methods yiqing xu · the fundamental problem of causal inference...

17.802 – LectureSynthetic Control Methods

Yiqing Xu

MIT

[email protected]

Yiqing Xu (MIT) Synth 1 / 43

What’s Different about Panel Data?

The fundamental problem of causal inference

A statistical solution makes use of the population

e.g. T = E [Y1]− E [Y0]

A scientific solution exploits homogeneity or invariance assumptions

e.g. A rock is a rock.e.g. The long-run growth rate of the US economy is 2.5%.

Panel data allow us to construct the counterfactuals of the treatedunits in the post-treatment period using information from boththe control group and treatment group in the pre-treatment period


T0 T1 T2

Causal Inference with Panel Data

T0 T1 T2

Solution 1: Time Series Analysis

DOT forecasts of road traffic volume

T0 T1 T2

Solution 2: Matching

T0 T1 T2

Solution 3: Find Parallel Worlds

Causal Inference with Panel Data

Statistical solution: SOO, e.g., matching

Scientific solution: modelling

Panel data make both easier

Matching on lagged outcomes makes SOO more plausible

Parallel trends assumption is somewhat “testable”


Many Models, Same Approach

Fixed effects and diff-in-diffs

Yit(0) = αi + δt + εit

Lag dependent variable

Yit(0) = α + θYi ,t−1 + δt + εit

Synthetic control (non-parametric)

Yit(0) =∑j∈C

w∗j Yjt

LimitationsFE and LDV assume homogeneous treatment effectDID assumes fixed treatment timingSynth allows only one treated unit and inference is less formal


Theoretic Motivations

What if the true model is as complicated as:

Yit(0) = Xitβ + Ziθt + λtµi + εit ,

Compare with the fixed effects (or DID) model

Yit(0) = Xitβ + Ziθt + δt + αi + εit ,

λtµi are fixed effects interacted with time-varying coefficient, inwhich δt and αi are special cases

Abadie and Gardeazabal (2003), Abadie, Diamond, and Hainmueller(2010) figured out a way to solve this problem when there is only onetreated unit


Comparative Case Studies

Comparative Case Studies:

Compare the evolution of an aggregate outcome for the unit affectedby the intervention to the evolution of the same aggregate for somecontrol group (e.g. Card, 1990, Card and Krueger, 1994, Abadie andGardeazabal, 2003)

Events or interventions take place at an aggregate level (e.g., cities,states, countries).

Challenges:

Ntr is small by definition

Selection of control group is often ambiguous

Standard errors do not reflect uncertainty about the ability of thecontrol group to reproduce the counterfactual of interest

Why synthetic control can be useful?


The Synthetic Control Method: Setup

Suppose that we observe J + 1 units in periods 1, 2, . . . ,T .

Region “one” is exposed to the intervention during periodsT0 + 1, . . . ,T .

Let Y Nit be the outcome that would be observed for unit i at time t in

the absence of the intervention.

Let Y Iit be the outcome that would be observed for unit i at time t if

unit i is exposed to the intervention in periods T0 + 1 to T .

We aim to estimate the effect of the intervention on the treated unit(α1T0+1, . . . , α1T ), where α1t = Y I

1t − Y N1t = Y1t − Y N

1t for t > T0.


1 T0 T

J

1 ?

Setup

Suppose that Y Nit is given by a factor model:

Y Nit = δt + Ziθt + λtµi + εit ,

◦ δt is an unobserved (common) time-dependent factor,◦ Zi is a (1× r) vector of observed covariates,◦ θt is a (r × 1) vector of unknown parameters,◦ λt is a (1× F ) vector of unknown common factors,◦ µi is a (F × 1) vector of unknown factor loadings,◦ εit are unobserved transitory shocks.

λtµi : heterogeneous responses to multiple unobserved factors

Basic idea: reweight the control group such that the synthetic controlunit matches Zi and (some) pre-treatment Yit of the treated unit; asa result, µi is automatically matched


1 T0 T

J

1

W*

Theory

Let W = (w2, . . . ,wJ+1)′ with wj ≥ 0 for j = 2, . . . , J + 1 andw2 + · · ·+ wJ+1 = 1. Each value of W represents a potentialsynthetic control

Let Y K1i , . . . , Y KM

i be M linear functions of pre-intervention outcomes(M ≥ F )

Suppose that we can choose W ∗ such that:

J+1∑j=2

w∗j Zj = Z1,

J+1∑j=2

w∗j YK1j = Y K1

1 , · · · ,J+1∑j=2

w∗j YKMj = Y KM

1 .

Then (if T0 is large relative to the scale of εit), an approximatelyunbiased estimator of α1t is:

α1t = Y1t −J+1∑j=2

w∗j Yjt

for t ∈ {T0 + 1, . . . ,T}Yiqing Xu (MIT) Synth 20 / 43

Implementation

Let X1 = (Z1, YK11 , . . . , Y KM

1 )′ be a (k × 1) vector of pre-interventioncharacteristics.

Similarly, X0 is a (k × J) matrix which contains the same variables forthe unaffected units.

The vector W ∗ is chosen to minimize ‖X1 − X0W ‖, subject to ourweight constraints.

We consider ‖X1 − X0W ‖V =√

(X1 − X0W )′V (X1 − X0W ), whereV is some (k × k) symmetric and positive semidefinite matrix.

Various ways to choose V (subjective assessment of predictive powerof X , regression, minimize MSPE, cross-validation, etc.).


Application: California’s Proposition 99

In 1988, California first passed comprehensive tobacco control legislation:

increased cigarette tax by 25 cents/pack

earmarked tax revenues to health and anti-smoking budgets

funded anti-smoking media campaigns

spurred clean-air ordinances throughout the state

produced more than $100 million per year in anti-tobacco projects

Other states that subsequently passed control programs are excluded fromdonor pool of controls (AK, AZ, FL, HA, MA, MD, MI, NJ, NY, OR, WA,DC)


Cigarette Consumption: CA and the Rest of the U.S.

1970 1975 1980 1985 1990 1995 2000

020

4060

8010

012

014

0

year

per−

capi

ta c

igar

ette

sal

es (

in p

acks

)

Californiarest of the U.S.

Passage of Proposition 99


Cigarette Consumption: CA and synthetic CA

1970 1975 1980 1985 1990 1995 2000

020

4060

8010

012

014

0

year

per−

capi

ta c

igar

ette

sal

es (

in p

acks

)

Californiasynthetic California



Predictor Means: Actual vs. Synthetic California

California Average ofVariables Real Synthetic 38 control statesLn(GDP per capita) 10.08 9.86 9.86Percent aged 15-24 17.40 17.40 17.29Retail price 89.42 89.41 87.27Beer consumption per capita 24.28 24.20 23.75Cigarette sales per capita 1988 90.10 91.62 114.20Cigarette sales per capita 1980 120.20 120.43 136.58Cigarette sales per capita 1975 127.10 126.99 132.81

Note: All variables except lagged cigarette sales are averaged for the 1980-1988 period (beer consumption is averaged 1984-1988).


Smoking Gap Between CA and synthetic CA

1970 1975 1980 1985 1990 1995 2000

−30

−20

−10

010

2030

year

gap

in p

er−

capi

ta c

igar

ette

sal

es (

in p

acks

)



Inference

Strategy:

Whether the effect estimated by the synthetic control for the unitaffected by the intervention is large relative to the effect estimated fora unit chosen at random

Valid regardless of the number of available comparison units, timeperiods, and whether the data are individual or aggregate

Implementation:

Iteratively apply the synthetic method to each state in the “donorpool” and obtain a distribution of placebo effects

Compare the gap for California to the distribution of the placebo gaps


Smoking Gap for CA and 38 control states

(All States in Donor Pool)

1970 1975 1980 1985 1990 1995 2000

−30

−20

−10

010

2030

year

gap

in p

er−

capi

ta c

igar

ette

sal

es (

in p

acks

) Californiacontrol states



Smoking Gap for CA and 19 control states

(Pre-Prop. 99 MSPE ≤ 2 Times Pre-Prop. 99 MSPE for CA)

1970 1975 1980 1985 1990 1995 2000

−30

−20

−10

010

2030

year

gap

in p

er−

capi

ta c

igar

ette

sal

es (

in p

acks

) Californiacontrol states



Ratio Post-Prop. 99 MSPE to Pre-Prop. 99 MSPE

(All 38 States in Donor Pool)

0 20 40 60 80 100 120

01

23

45

post/pre−Proposition 99 mean squared prediction error

freq

uenc

y

California


Application: German unification

Synth in cross-country studies

Cross-country regressions are often criticized because they putside-by-side countries of very different characteristics.

The synthetic control method provides an appealing data-drivenprocedure to study the effects of events or interventions that takeplace at the country level.

Application:

The economic impact of the 1990 German unification in WestGermany.

Donor pool is restricted to 21 OECD countries.


Economic Growth Predictors Means

West Synthetic OECD SampleGermany West Germany excl. West Germany

GDP per-capita 8169.8 8163.1 8049.3Trade openness 45.8 54.4 32.6Inflation rate 3.4 4.7 7.3Industry share 34.7 34.7 34.3Schooling 55.5 55.6 43.8Investment rate 27.0 27.1 25.9

Note: GDP, inflation rate, and trade openness are averaged for the1960–1989 period. Industry share is averaged for the 1980–1989 pe-riod. Investment rate and schooling are averaged for the 1980–1985period.


West Germany and synthetic West Germany

1960 1970 1980 1990 2000

050

0010

000

1500

020

000

2500

030

000

year

per−

capi

ta G

DP

(P

PP,

200

2 U

SD

)

West Germanysynthetic West Germany

reunification


GDP Gap: West Germany and synthetic West Germany

1960 1970 1980 1990 2000

−40

00−

2000

020

0040

00

year

gap

in p

er−

capi

ta G

DP

(P

PP,

200

2 U

SD

)

reunification


Country Weights in the Synthetic West Germany

Country Weight Country Weight

Australia 0 Netherlands 0.11Austria 0.47 New Zealand 0.11Belgium 0 Norway 0Canada 0 Portugal 0Denmark 0 Spain 0France 0 Sweden 0Greece 0 Switzerland 0Ireland 0 United Kingdom 0.17Italy 0 United States 0Japan 0 0.14


Country Weights in the Synthetic West Germany

Synthetic Regression Synthetic RegressionCountry Weight Weight Country Weight WeightAustralia 0 0.1 Netherlands 0.11 0.18Austria 0.47 0.33 New Zealand 0 -0.08Belgium 0 0.1 Norway 0 -0.07Canada 0 0.09 Portugal 0 -0.14Denmark 0 0.04 Spain 0 0France 0 0.16 Switzerland 0.17 -0.06Greece 0 0.02 United Kingdom 0 -0.04Italy 0 -0.17 United States 0.14 0.21Japan 0.11 0.32

Note: Synthetic Weight: Unit weight assigned by the synthetic controlmethod. Regression Weight: Unit weight assigned by linear regression.


Placebo Reunification 1980

1960 1965 1970 1975 1980 1985

050

0010

000

1500

020

000

2500

0

year

per−

capi

ta G

DP

(P

PP,

200

2 U

SD

)


placebo reunification


Placebo Reunification 1970

1960 1965 1970 1975 1980 1985

050

0010

000

1500

020

000

2500

0

year

per−

capi

ta G

DP

(P

PP,

200

2 U

SD

)


placebo reunification


Ratio of post- and pre-reunification MSPE

Portugal

UK

Switzerland

Italy

France

Netherlands

Austria

Japan

New Zealand

Denmark

Belgium

Greece

Canada

Australia

Spain

USA

Norway

West Germany

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

5 10 15

Post−Period RMSE / Pre−Period RMSE


In Summary

Working with panel data makes identification easier

Time-invariant confounders are controlled forSelection on unobservables is allowedThe identifying assumption is not directly testable, but can besupported by data

However, diff-in-diffs type of research designs (DID/FE/LDV) are notfree from modelling assumptions

The synthetic control method relaxes the model assumptions, butconsiders only one treated unit, requires smooth outcomes, and theinference is less formal → we are working on it


synthetic control methods yiqing xu · the fundamental problem of causal inference...

Documents