evaluating public programs with close …crwalters/papers/kline_walters.pdfevaluating public...
TRANSCRIPT
EVALUATING PUBLIC PROGRAMS WITH CLOSESUBSTITUTES THE CASE OF HEAD START
Patrick Kline and
Christopher R Walters
We use data from the Head Start Impact Study (HSIS) to evaluate the cost-effectiveness of Head Start the largest early childhood education program inthe United States Head Start draws roughly a third of its participants fromcompeting preschool programs many of which receive public funds We showthat accounting for the fiscal impacts of such program substitution pushesestimates of Head Startrsquos benefit-cost ratio well above one under a widerange of assumptions on the structure of the market for preschool servicesand the dollar value of test score gains To parse the programrsquos test score im-pacts relative to home care and competing preschools we selection-correct testscores in each care environment using excluded interactions between experi-mental assignments and household characteristics We find that Head Startgenerates larger test score gains for children who would not otherwise attendpreschool and for children who are less likely to participate in the programJEL Codes I20 J24 H52 C30
I Introduction
Many government programs provide services that can be ob-tained in roughly comparable form via markets or through otherpublic organizations The presence of close program substitutes com-plicates the task of program evaluation by generating ambiguity re-garding which causal estimands are of interest Standard intent-to-treat impacts from experimental demonstrations can yield undulynegative assessments of program effectiveness if most participantswould receive similar services in the absence of an intervention(Heckman et al 2000) On the other hand experiments that artifi-cially restrict substitution alternatives may yield impacts that arenot representative of the costs and benefits of actual policy changes
We thank Danny Yagan James Heckman Nathan Hendren Magne MogstadJesse Rothstein Melissa Tartari and seminar participants at UC Berkeley theUniversity of Chicago Arizona State University Harvard University StanfordUniversity the NBER Public Economics Spring 2015 Meetings ColumbiaUniversity UC San Diego Princeton University the NBER Labor StudiesSummer Institute 2015 Meetings Uppsala University MIT VanderbiltUniversity and four anonymous referees for helpful comments Raffaele Saggioprovided outstanding research assistance We also thank Research Connectionsfor providing the data Generous funding support for this project was provided bythe Berkeley Center for Equitable Growth
The Author(s) 2016 Published by Oxford University Press on behalf of Presidentand Fellows of Harvard College All rights reserved For Permissions please emailjournalspermissionsoupcomThe Quarterly Journal of Economics (2016) 1795ndash1848 doi101093qjeqjw027Advance Access publication on July 11 2016
1795
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
This article assesses the cost-effectiveness of Head Startmdashaprominent public education program for which close public andprivate substitutes are widely available Head Start is the largestearly childhood education program in the United StatesLaunched in 1965 as part of President Lyndon Johnsonrsquos waron poverty the program has evolved from an eight-weeksummer program into a year-round program that offers educa-tion health and nutrition services to disadvantaged children andtheir families By 2013 Head Start enrolled about 900000 three-and four-year-old children at a cost of $76 billion (US DHHS2013)
Views on the effectiveness of Head Start vary widely (Ludwigand Phillips 2007 and Gibbs Ludwig and Miller 2011 providereviews) A number of observational studies find substantialshort- and long-run impacts on test scores and other outcomes(Currie and Thomas 1995 Garces Thomas and Currie 2002Ludwig and Miller 2007 Deming 2009 Carneiro and Ginja2014) By contrast a recent randomized evaluationmdashthe HeadStart Impact Study (HSIS)mdashfinds small impacts on test scoresthat fade out quickly (Puma et al 2010 Puma Bell and Heid2012) These results have generally been interpreted as evidencethat Head Start is ineffective and in need of reform (Barnett 2011Klein 2011)
Two observations suggest that such conclusions are prema-ture First research on early childhood interventions finds long-run gains in adult outcomes despite short-run fade-out of testscore impacts (Heckman et al 2010 Heckman Pinto andSavelyev 2013 Chetty et al 2011 Chetty Friedman andRockoff 2014b) Second roughly one third of the HSIS controlgroup participated in alternate forms of preschool This suggeststhat the HSIS may have shifted many students between differentsorts of preschools without altering their exposure to preschoolservices The aim of this article is to clarify how the presence ofsubstitute preschools affects the interpretation of the HSIS re-sults and the cost-effectiveness of the Head Start program
Our study begins by revisiting the experimental impacts ofthe HSIS on student test scores We replicate the fade-out patternfound in previous work but find that adjusting for experimentalnon compliance leads to imprecise estimates of the effect of HeadStart participation beyond the first year of the experiment As aresult the conclusion of complete effect fade-out is less clear thannaive intent-to-treat estimates suggest Turning to substitution
QUARTERLY JOURNAL OF ECONOMICS1796
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
patterns we find that roughly one third of Head Start lsquolsquocompliersrsquorsquo(Angrist Imbens and Rubin 1996) in the HSIS experiment wouldhave participated in other forms of preschool had they not beenlotteried into the program These alternative preschools drawheavily on public funding which mitigates the net costs to gov-ernment of shifting children from other preschools into HeadStart
These facts motivate a theoretical analysis clarifying whichparameters are (and are not) policy relevant when publicly sub-sidized program substitutes are present We work with a stylizedmodel where test score impacts are valued according to theireffects on childrenrsquos after-tax lifetime earnings We show thatwhen competing preschool programs are not rationed thepolicy-relevant causal parameter governing the benefits of HeadStart expansion is an average effect of Head Start participationrelative to the next best alternative regardless of whether thatalternative is a competing program or home care This parametercoincides with the local average treatment effect (LATE) identi-fied by a randomized experiment with imperfect compliance whenthe experiment contains a representative sample of program com-pliers Hence imperfect compliance and program substitutionoften thought to be confounding limitations of social experimentsturn out to be virtues when the substitution patterns in the ex-periment replicate those found in the broader population
We use this result to derive an estimable benefit-cost ratioassociated with Head Start expansions This ratio scales HeadStartrsquos projected impacts on the after-tax earnings of childrenby its net costs to government inclusive of fiscal externalitiesChief among these externalities is the cost savings that arisewhen Head Start draws children away from competing subsidizedpreschool programs Although such effects are typically ignoredin cost-benefit analyses of Head Start and other similar programs(eg CEA 2015) we find via a calibration exercise that such omis-sions can be quantitatively important Head Start roughly breakseven when the cost savings associated with program substitutionare ignored but yields benefits nearly twice as large as costswhen these savings are incorporated This appears to be arobust findingmdashafter accounting for fiscal externalities HeadStartrsquos benefits exceed its costs whenever short-run test scoreimpacts yield earnings gains within the range found in therecent literature
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1797
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A limitation of our baseline analysis is that it assumeschanges in program scale do not alter the mix of program compli-ers To address this issue we also consider lsquolsquostructural reformsrsquorsquo toHead Start that change the mix of compliers without affectingtest score outcomes Examples of such reforms might include in-creased transportation services marketing efforts or spendingon program features that parents value Households who respondto structural reforms may differ from experimental compliers onunobserved dimensions including their mix of counterfactualprogram choices Assessing these reforms therefore requiresknowledge of parameters not directly identified by the HSIS ex-periment Specifically we show that such reforms require identi-fication of a variant of the marginal treatment effect (MTE)concept of Heckman and Vytlacil (1999)
To assess reforms that attract new children we develop aselection model that parameterizes variation in treatment effectswith respect to counterfactual care alternatives as well as ob-served and unobserved child characteristics We prove that themodel parameters are identified and propose a two-step controlfunction estimator that exploits heterogeneity in the response toHead Start offers across sites and demographic groups to inferrelationships between unobserved factors driving preschool en-rollment and potential outcomes The estimator is shown to passa variety of specification tests and accurately reproduce patternsof treatment effect heterogeneity found in the experiment Themodel estimates indicate that Head Start has large positiveshort-run effects on the test scores of children who would haveotherwise been cared for at home and insignificant effects on chil-dren who would otherwise attend other preschoolsmdasha finding cor-roborated by Feller et al (forthcoming) who reach similarconclusions using principal stratification methods (Frangakisand Rubin 2002) Our estimates also reveal a lsquolsquoreverse Royrsquorsquo pat-tern of selection whereby children with unobserved characteris-tics that make them less likely to enroll in Head Start experiencelarger test score gains
We conclude with an assessment of prospects for increasingHead Startrsquos rate of return via outreach to new populations Ourestimates suggest that expansions of Head Start could boost theprogramrsquos rate of return provided that the proposed technologyfor increasing enrollment (eg improved transportation services)is not too costly We also use our estimated selection model toexamine the robustness of our results to rationing of competing
QUARTERLY JOURNAL OF ECONOMICS1798
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschools Rationing implies that competing subsidized pre-schools do not contract when Head Start expands which shutsdown a form of public savings On the other hand expandingHead Start generates opportunities for new children to fill va-cated seats in substitute programs Our estimates indicate thatthe effect on test scores (and therefore earnings) of moving chil-dren from home care to competing preschools is substantial lead-ing us to conclude that rationing is in fact likely to increase thefavorable estimated rates of return found in our baselineanalysis
The rest of the article is structured as follows Section II pro-vides background on Head Start Section III describes the HSISdata and basic experimental impacts Section IV presents evi-dence on substitution patterns Section V introduces a theoreticalframework for assessing public programs with close substitutesSection VI provides a cost-benefit analysis of Head Start SectionVII develops our econometric selection model and discusses iden-tification and estimation Section VIII reports estimates of themodel Section IX simulates the effects of structural program re-forms Section X concludes
II Background on Head Start
Head Start provides preschool for disadvantaged children inthe United States The program is funded by federal grantsawarded to local public or private organizations Grantees arerequired to match at least 20 of their Head Start awards fromother sources and must meet a set of program-wide performancecriteria Eligibility for Head Start is generally limited to childrenfrom households below the federal poverty line though familiesabove this threshold may be eligible if they meet other criteriasuch as participation in the Temporary Aid for Needy Families(TANF) program Up to 10 of a Head Start centerrsquos enrollmentcan also come from higher-income families The program is freeHead Start grantees are prohibited from charging families feesfor services (US DHHS 2014) It is also oversubscribed in 200285 of Head Start participants attended programs with moreapplicants than available seats (Puma et al 2010)
Head Start is not the only form of subsidized preschool avail-able to poor families Preschool participation rates for disadvan-taged children have risen over time as cities and states expanded
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1799
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
their public preschool offerings (Cascio and Schanzenbach 2013)Moreover the Child Care Development Fund program providesblock grants that finance childcare subsidies for low-income fam-ilies often in the form of child care vouchers that can be used forcenter-based preschool (US DHHS 2012) Most states also useTANF funds to finance additional child care subsidies(Schumacher Greenberg and Duffy 2001) Because Head Startservices are provided by local organizations who themselves mustraise outside funds it is unclear to what extent Head Start andother public preschool programs actually differ in their educationtechnology
A large nonexperimental literature suggests that Head Startproduced large short- and long-run benefits for early cohorts ofprogram participants Several studies estimate the effects ofHead Start by comparing program participants to their nonpar-ticipant siblings (Currie and Thomas 1995 Garces Thomas andCurrie 2002 Deming 2009) Results from this research designshow positive short-run effects on test scores and long-run effectson educational attainment earnings and crime Other studiesexploit discontinuities in Head Start program rules to infer pro-gram effects (Ludwig and Miller 2007 Carneiro and Ginja 2014)These studies show longer run improvements in health outcomesand criminal activity
In contrast to these nonexperimental estimates results froma recent randomized controlled trial reveal smaller less persis-tent effects The 1998 Head Start reauthorization bill included acongressional mandate to determine the effects of the programThis mandate resulted in the HSIS an experiment in which morethan 4000 applicants were randomly assigned via lottery toeither a treatment group with access to Head Start or a controlgroup without access in fall 2002 The experimental resultsshowed that a Head Start offer increased measures of cognitiveachievement by roughly 01 standard deviations during pre-school but these gains faded out by kindergarten Moreoverthe experiment showed little evidence of effects on noncognitiveor health outcomes (Puma et al 2010 Puma Bell and Heid2012) These results suggest both smaller short-run effects andfaster fade-out than nonexperimental estimates for earlier co-horts Scholars and policy makers have generally interpretedthe HSIS results as evidence that Head Start is ineffective andin need of reform (Barnett 2011) The experimental results havealso been cited in the popular media to motivate calls for dramatic
QUARTERLY JOURNAL OF ECONOMICS1800
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
restructuring or elimination of the program (Klein 2011 Stossel2014)1
Differences between the HSIS results and the nonexperi-mental literature could be due to changes in program effective-ness over time or to selection bias in nonexperimental siblingcomparisons Another explanation however is that these tworesearch designs identify different parameters Most nonexperi-mental analyses have focused on recovering the effect of HeadStart relative to home care In contrast the HSIS measures theeffect of Head Start relative to a mix of alternative care environ-ments including other preschools
III Data and Experimental Impacts
Before turning to an analysis of program substitution issueswe describe the HSIS data and report experimental impacts ontest scores and program compliance
IIIA Data
Our core analysis sample includes 3571 HSIS applicantswith nonmissing baseline characteristics and spring 2003 testscores Online Appendix A describes construction of this sampleThe outcome of interest is a summary index of cognitive testscores that averages Woodcock Johnson III (WJIII) test scoreswith Peabody Picture and Vocabulary Test (PPVT) scores witheach test normed to have mean 0 and variance 1 in the controlgroup by cohort and year We use WJIII and PPVT scores becausethese are among the most reliable tests in the HSIS data both areavailable in each year of the experiment allowing us to producecomparable estimates over time
1 Subsequent analyses of the HSIS data suggest caveats to this negative in-terpretation but do not overturn the finding of modest mean test score impactsaccompanied by rapid fade-out Gelber and Isen (2013) find persistent effects onparental engagement with children Bitler Domina and Hoynes (2014) find largerexperimental impacts at low quantiles of the test score distribution These quantiletreatment effects fade out by first grade though there is some evidence of persistenteffects at the bottom of the distribution for Spanish speakers Walters (2015) findsevidence of substantial heterogeneity in impacts across experimental sites and in-vestigates the relationship between this heterogeneity and observed program char-acteristics Walters finds smaller effects for Head Start centers that draw morechildren from other preschools rather than home care a finding we explore inmore detail here
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1801
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
This article assesses the cost-effectiveness of Head Startmdashaprominent public education program for which close public andprivate substitutes are widely available Head Start is the largestearly childhood education program in the United StatesLaunched in 1965 as part of President Lyndon Johnsonrsquos waron poverty the program has evolved from an eight-weeksummer program into a year-round program that offers educa-tion health and nutrition services to disadvantaged children andtheir families By 2013 Head Start enrolled about 900000 three-and four-year-old children at a cost of $76 billion (US DHHS2013)
Views on the effectiveness of Head Start vary widely (Ludwigand Phillips 2007 and Gibbs Ludwig and Miller 2011 providereviews) A number of observational studies find substantialshort- and long-run impacts on test scores and other outcomes(Currie and Thomas 1995 Garces Thomas and Currie 2002Ludwig and Miller 2007 Deming 2009 Carneiro and Ginja2014) By contrast a recent randomized evaluationmdashthe HeadStart Impact Study (HSIS)mdashfinds small impacts on test scoresthat fade out quickly (Puma et al 2010 Puma Bell and Heid2012) These results have generally been interpreted as evidencethat Head Start is ineffective and in need of reform (Barnett 2011Klein 2011)
Two observations suggest that such conclusions are prema-ture First research on early childhood interventions finds long-run gains in adult outcomes despite short-run fade-out of testscore impacts (Heckman et al 2010 Heckman Pinto andSavelyev 2013 Chetty et al 2011 Chetty Friedman andRockoff 2014b) Second roughly one third of the HSIS controlgroup participated in alternate forms of preschool This suggeststhat the HSIS may have shifted many students between differentsorts of preschools without altering their exposure to preschoolservices The aim of this article is to clarify how the presence ofsubstitute preschools affects the interpretation of the HSIS re-sults and the cost-effectiveness of the Head Start program
Our study begins by revisiting the experimental impacts ofthe HSIS on student test scores We replicate the fade-out patternfound in previous work but find that adjusting for experimentalnon compliance leads to imprecise estimates of the effect of HeadStart participation beyond the first year of the experiment As aresult the conclusion of complete effect fade-out is less clear thannaive intent-to-treat estimates suggest Turning to substitution
QUARTERLY JOURNAL OF ECONOMICS1796
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
patterns we find that roughly one third of Head Start lsquolsquocompliersrsquorsquo(Angrist Imbens and Rubin 1996) in the HSIS experiment wouldhave participated in other forms of preschool had they not beenlotteried into the program These alternative preschools drawheavily on public funding which mitigates the net costs to gov-ernment of shifting children from other preschools into HeadStart
These facts motivate a theoretical analysis clarifying whichparameters are (and are not) policy relevant when publicly sub-sidized program substitutes are present We work with a stylizedmodel where test score impacts are valued according to theireffects on childrenrsquos after-tax lifetime earnings We show thatwhen competing preschool programs are not rationed thepolicy-relevant causal parameter governing the benefits of HeadStart expansion is an average effect of Head Start participationrelative to the next best alternative regardless of whether thatalternative is a competing program or home care This parametercoincides with the local average treatment effect (LATE) identi-fied by a randomized experiment with imperfect compliance whenthe experiment contains a representative sample of program com-pliers Hence imperfect compliance and program substitutionoften thought to be confounding limitations of social experimentsturn out to be virtues when the substitution patterns in the ex-periment replicate those found in the broader population
We use this result to derive an estimable benefit-cost ratioassociated with Head Start expansions This ratio scales HeadStartrsquos projected impacts on the after-tax earnings of childrenby its net costs to government inclusive of fiscal externalitiesChief among these externalities is the cost savings that arisewhen Head Start draws children away from competing subsidizedpreschool programs Although such effects are typically ignoredin cost-benefit analyses of Head Start and other similar programs(eg CEA 2015) we find via a calibration exercise that such omis-sions can be quantitatively important Head Start roughly breakseven when the cost savings associated with program substitutionare ignored but yields benefits nearly twice as large as costswhen these savings are incorporated This appears to be arobust findingmdashafter accounting for fiscal externalities HeadStartrsquos benefits exceed its costs whenever short-run test scoreimpacts yield earnings gains within the range found in therecent literature
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1797
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A limitation of our baseline analysis is that it assumeschanges in program scale do not alter the mix of program compli-ers To address this issue we also consider lsquolsquostructural reformsrsquorsquo toHead Start that change the mix of compliers without affectingtest score outcomes Examples of such reforms might include in-creased transportation services marketing efforts or spendingon program features that parents value Households who respondto structural reforms may differ from experimental compliers onunobserved dimensions including their mix of counterfactualprogram choices Assessing these reforms therefore requiresknowledge of parameters not directly identified by the HSIS ex-periment Specifically we show that such reforms require identi-fication of a variant of the marginal treatment effect (MTE)concept of Heckman and Vytlacil (1999)
To assess reforms that attract new children we develop aselection model that parameterizes variation in treatment effectswith respect to counterfactual care alternatives as well as ob-served and unobserved child characteristics We prove that themodel parameters are identified and propose a two-step controlfunction estimator that exploits heterogeneity in the response toHead Start offers across sites and demographic groups to inferrelationships between unobserved factors driving preschool en-rollment and potential outcomes The estimator is shown to passa variety of specification tests and accurately reproduce patternsof treatment effect heterogeneity found in the experiment Themodel estimates indicate that Head Start has large positiveshort-run effects on the test scores of children who would haveotherwise been cared for at home and insignificant effects on chil-dren who would otherwise attend other preschoolsmdasha finding cor-roborated by Feller et al (forthcoming) who reach similarconclusions using principal stratification methods (Frangakisand Rubin 2002) Our estimates also reveal a lsquolsquoreverse Royrsquorsquo pat-tern of selection whereby children with unobserved characteris-tics that make them less likely to enroll in Head Start experiencelarger test score gains
We conclude with an assessment of prospects for increasingHead Startrsquos rate of return via outreach to new populations Ourestimates suggest that expansions of Head Start could boost theprogramrsquos rate of return provided that the proposed technologyfor increasing enrollment (eg improved transportation services)is not too costly We also use our estimated selection model toexamine the robustness of our results to rationing of competing
QUARTERLY JOURNAL OF ECONOMICS1798
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschools Rationing implies that competing subsidized pre-schools do not contract when Head Start expands which shutsdown a form of public savings On the other hand expandingHead Start generates opportunities for new children to fill va-cated seats in substitute programs Our estimates indicate thatthe effect on test scores (and therefore earnings) of moving chil-dren from home care to competing preschools is substantial lead-ing us to conclude that rationing is in fact likely to increase thefavorable estimated rates of return found in our baselineanalysis
The rest of the article is structured as follows Section II pro-vides background on Head Start Section III describes the HSISdata and basic experimental impacts Section IV presents evi-dence on substitution patterns Section V introduces a theoreticalframework for assessing public programs with close substitutesSection VI provides a cost-benefit analysis of Head Start SectionVII develops our econometric selection model and discusses iden-tification and estimation Section VIII reports estimates of themodel Section IX simulates the effects of structural program re-forms Section X concludes
II Background on Head Start
Head Start provides preschool for disadvantaged children inthe United States The program is funded by federal grantsawarded to local public or private organizations Grantees arerequired to match at least 20 of their Head Start awards fromother sources and must meet a set of program-wide performancecriteria Eligibility for Head Start is generally limited to childrenfrom households below the federal poverty line though familiesabove this threshold may be eligible if they meet other criteriasuch as participation in the Temporary Aid for Needy Families(TANF) program Up to 10 of a Head Start centerrsquos enrollmentcan also come from higher-income families The program is freeHead Start grantees are prohibited from charging families feesfor services (US DHHS 2014) It is also oversubscribed in 200285 of Head Start participants attended programs with moreapplicants than available seats (Puma et al 2010)
Head Start is not the only form of subsidized preschool avail-able to poor families Preschool participation rates for disadvan-taged children have risen over time as cities and states expanded
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1799
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
their public preschool offerings (Cascio and Schanzenbach 2013)Moreover the Child Care Development Fund program providesblock grants that finance childcare subsidies for low-income fam-ilies often in the form of child care vouchers that can be used forcenter-based preschool (US DHHS 2012) Most states also useTANF funds to finance additional child care subsidies(Schumacher Greenberg and Duffy 2001) Because Head Startservices are provided by local organizations who themselves mustraise outside funds it is unclear to what extent Head Start andother public preschool programs actually differ in their educationtechnology
A large nonexperimental literature suggests that Head Startproduced large short- and long-run benefits for early cohorts ofprogram participants Several studies estimate the effects ofHead Start by comparing program participants to their nonpar-ticipant siblings (Currie and Thomas 1995 Garces Thomas andCurrie 2002 Deming 2009) Results from this research designshow positive short-run effects on test scores and long-run effectson educational attainment earnings and crime Other studiesexploit discontinuities in Head Start program rules to infer pro-gram effects (Ludwig and Miller 2007 Carneiro and Ginja 2014)These studies show longer run improvements in health outcomesand criminal activity
In contrast to these nonexperimental estimates results froma recent randomized controlled trial reveal smaller less persis-tent effects The 1998 Head Start reauthorization bill included acongressional mandate to determine the effects of the programThis mandate resulted in the HSIS an experiment in which morethan 4000 applicants were randomly assigned via lottery toeither a treatment group with access to Head Start or a controlgroup without access in fall 2002 The experimental resultsshowed that a Head Start offer increased measures of cognitiveachievement by roughly 01 standard deviations during pre-school but these gains faded out by kindergarten Moreoverthe experiment showed little evidence of effects on noncognitiveor health outcomes (Puma et al 2010 Puma Bell and Heid2012) These results suggest both smaller short-run effects andfaster fade-out than nonexperimental estimates for earlier co-horts Scholars and policy makers have generally interpretedthe HSIS results as evidence that Head Start is ineffective andin need of reform (Barnett 2011) The experimental results havealso been cited in the popular media to motivate calls for dramatic
QUARTERLY JOURNAL OF ECONOMICS1800
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
restructuring or elimination of the program (Klein 2011 Stossel2014)1
Differences between the HSIS results and the nonexperi-mental literature could be due to changes in program effective-ness over time or to selection bias in nonexperimental siblingcomparisons Another explanation however is that these tworesearch designs identify different parameters Most nonexperi-mental analyses have focused on recovering the effect of HeadStart relative to home care In contrast the HSIS measures theeffect of Head Start relative to a mix of alternative care environ-ments including other preschools
III Data and Experimental Impacts
Before turning to an analysis of program substitution issueswe describe the HSIS data and report experimental impacts ontest scores and program compliance
IIIA Data
Our core analysis sample includes 3571 HSIS applicantswith nonmissing baseline characteristics and spring 2003 testscores Online Appendix A describes construction of this sampleThe outcome of interest is a summary index of cognitive testscores that averages Woodcock Johnson III (WJIII) test scoreswith Peabody Picture and Vocabulary Test (PPVT) scores witheach test normed to have mean 0 and variance 1 in the controlgroup by cohort and year We use WJIII and PPVT scores becausethese are among the most reliable tests in the HSIS data both areavailable in each year of the experiment allowing us to producecomparable estimates over time
1 Subsequent analyses of the HSIS data suggest caveats to this negative in-terpretation but do not overturn the finding of modest mean test score impactsaccompanied by rapid fade-out Gelber and Isen (2013) find persistent effects onparental engagement with children Bitler Domina and Hoynes (2014) find largerexperimental impacts at low quantiles of the test score distribution These quantiletreatment effects fade out by first grade though there is some evidence of persistenteffects at the bottom of the distribution for Spanish speakers Walters (2015) findsevidence of substantial heterogeneity in impacts across experimental sites and in-vestigates the relationship between this heterogeneity and observed program char-acteristics Walters finds smaller effects for Head Start centers that draw morechildren from other preschools rather than home care a finding we explore inmore detail here
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1801
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
patterns we find that roughly one third of Head Start lsquolsquocompliersrsquorsquo(Angrist Imbens and Rubin 1996) in the HSIS experiment wouldhave participated in other forms of preschool had they not beenlotteried into the program These alternative preschools drawheavily on public funding which mitigates the net costs to gov-ernment of shifting children from other preschools into HeadStart
These facts motivate a theoretical analysis clarifying whichparameters are (and are not) policy relevant when publicly sub-sidized program substitutes are present We work with a stylizedmodel where test score impacts are valued according to theireffects on childrenrsquos after-tax lifetime earnings We show thatwhen competing preschool programs are not rationed thepolicy-relevant causal parameter governing the benefits of HeadStart expansion is an average effect of Head Start participationrelative to the next best alternative regardless of whether thatalternative is a competing program or home care This parametercoincides with the local average treatment effect (LATE) identi-fied by a randomized experiment with imperfect compliance whenthe experiment contains a representative sample of program com-pliers Hence imperfect compliance and program substitutionoften thought to be confounding limitations of social experimentsturn out to be virtues when the substitution patterns in the ex-periment replicate those found in the broader population
We use this result to derive an estimable benefit-cost ratioassociated with Head Start expansions This ratio scales HeadStartrsquos projected impacts on the after-tax earnings of childrenby its net costs to government inclusive of fiscal externalitiesChief among these externalities is the cost savings that arisewhen Head Start draws children away from competing subsidizedpreschool programs Although such effects are typically ignoredin cost-benefit analyses of Head Start and other similar programs(eg CEA 2015) we find via a calibration exercise that such omis-sions can be quantitatively important Head Start roughly breakseven when the cost savings associated with program substitutionare ignored but yields benefits nearly twice as large as costswhen these savings are incorporated This appears to be arobust findingmdashafter accounting for fiscal externalities HeadStartrsquos benefits exceed its costs whenever short-run test scoreimpacts yield earnings gains within the range found in therecent literature
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1797
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A limitation of our baseline analysis is that it assumeschanges in program scale do not alter the mix of program compli-ers To address this issue we also consider lsquolsquostructural reformsrsquorsquo toHead Start that change the mix of compliers without affectingtest score outcomes Examples of such reforms might include in-creased transportation services marketing efforts or spendingon program features that parents value Households who respondto structural reforms may differ from experimental compliers onunobserved dimensions including their mix of counterfactualprogram choices Assessing these reforms therefore requiresknowledge of parameters not directly identified by the HSIS ex-periment Specifically we show that such reforms require identi-fication of a variant of the marginal treatment effect (MTE)concept of Heckman and Vytlacil (1999)
To assess reforms that attract new children we develop aselection model that parameterizes variation in treatment effectswith respect to counterfactual care alternatives as well as ob-served and unobserved child characteristics We prove that themodel parameters are identified and propose a two-step controlfunction estimator that exploits heterogeneity in the response toHead Start offers across sites and demographic groups to inferrelationships between unobserved factors driving preschool en-rollment and potential outcomes The estimator is shown to passa variety of specification tests and accurately reproduce patternsof treatment effect heterogeneity found in the experiment Themodel estimates indicate that Head Start has large positiveshort-run effects on the test scores of children who would haveotherwise been cared for at home and insignificant effects on chil-dren who would otherwise attend other preschoolsmdasha finding cor-roborated by Feller et al (forthcoming) who reach similarconclusions using principal stratification methods (Frangakisand Rubin 2002) Our estimates also reveal a lsquolsquoreverse Royrsquorsquo pat-tern of selection whereby children with unobserved characteris-tics that make them less likely to enroll in Head Start experiencelarger test score gains
We conclude with an assessment of prospects for increasingHead Startrsquos rate of return via outreach to new populations Ourestimates suggest that expansions of Head Start could boost theprogramrsquos rate of return provided that the proposed technologyfor increasing enrollment (eg improved transportation services)is not too costly We also use our estimated selection model toexamine the robustness of our results to rationing of competing
QUARTERLY JOURNAL OF ECONOMICS1798
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschools Rationing implies that competing subsidized pre-schools do not contract when Head Start expands which shutsdown a form of public savings On the other hand expandingHead Start generates opportunities for new children to fill va-cated seats in substitute programs Our estimates indicate thatthe effect on test scores (and therefore earnings) of moving chil-dren from home care to competing preschools is substantial lead-ing us to conclude that rationing is in fact likely to increase thefavorable estimated rates of return found in our baselineanalysis
The rest of the article is structured as follows Section II pro-vides background on Head Start Section III describes the HSISdata and basic experimental impacts Section IV presents evi-dence on substitution patterns Section V introduces a theoreticalframework for assessing public programs with close substitutesSection VI provides a cost-benefit analysis of Head Start SectionVII develops our econometric selection model and discusses iden-tification and estimation Section VIII reports estimates of themodel Section IX simulates the effects of structural program re-forms Section X concludes
II Background on Head Start
Head Start provides preschool for disadvantaged children inthe United States The program is funded by federal grantsawarded to local public or private organizations Grantees arerequired to match at least 20 of their Head Start awards fromother sources and must meet a set of program-wide performancecriteria Eligibility for Head Start is generally limited to childrenfrom households below the federal poverty line though familiesabove this threshold may be eligible if they meet other criteriasuch as participation in the Temporary Aid for Needy Families(TANF) program Up to 10 of a Head Start centerrsquos enrollmentcan also come from higher-income families The program is freeHead Start grantees are prohibited from charging families feesfor services (US DHHS 2014) It is also oversubscribed in 200285 of Head Start participants attended programs with moreapplicants than available seats (Puma et al 2010)
Head Start is not the only form of subsidized preschool avail-able to poor families Preschool participation rates for disadvan-taged children have risen over time as cities and states expanded
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1799
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
their public preschool offerings (Cascio and Schanzenbach 2013)Moreover the Child Care Development Fund program providesblock grants that finance childcare subsidies for low-income fam-ilies often in the form of child care vouchers that can be used forcenter-based preschool (US DHHS 2012) Most states also useTANF funds to finance additional child care subsidies(Schumacher Greenberg and Duffy 2001) Because Head Startservices are provided by local organizations who themselves mustraise outside funds it is unclear to what extent Head Start andother public preschool programs actually differ in their educationtechnology
A large nonexperimental literature suggests that Head Startproduced large short- and long-run benefits for early cohorts ofprogram participants Several studies estimate the effects ofHead Start by comparing program participants to their nonpar-ticipant siblings (Currie and Thomas 1995 Garces Thomas andCurrie 2002 Deming 2009) Results from this research designshow positive short-run effects on test scores and long-run effectson educational attainment earnings and crime Other studiesexploit discontinuities in Head Start program rules to infer pro-gram effects (Ludwig and Miller 2007 Carneiro and Ginja 2014)These studies show longer run improvements in health outcomesand criminal activity
In contrast to these nonexperimental estimates results froma recent randomized controlled trial reveal smaller less persis-tent effects The 1998 Head Start reauthorization bill included acongressional mandate to determine the effects of the programThis mandate resulted in the HSIS an experiment in which morethan 4000 applicants were randomly assigned via lottery toeither a treatment group with access to Head Start or a controlgroup without access in fall 2002 The experimental resultsshowed that a Head Start offer increased measures of cognitiveachievement by roughly 01 standard deviations during pre-school but these gains faded out by kindergarten Moreoverthe experiment showed little evidence of effects on noncognitiveor health outcomes (Puma et al 2010 Puma Bell and Heid2012) These results suggest both smaller short-run effects andfaster fade-out than nonexperimental estimates for earlier co-horts Scholars and policy makers have generally interpretedthe HSIS results as evidence that Head Start is ineffective andin need of reform (Barnett 2011) The experimental results havealso been cited in the popular media to motivate calls for dramatic
QUARTERLY JOURNAL OF ECONOMICS1800
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
restructuring or elimination of the program (Klein 2011 Stossel2014)1
Differences between the HSIS results and the nonexperi-mental literature could be due to changes in program effective-ness over time or to selection bias in nonexperimental siblingcomparisons Another explanation however is that these tworesearch designs identify different parameters Most nonexperi-mental analyses have focused on recovering the effect of HeadStart relative to home care In contrast the HSIS measures theeffect of Head Start relative to a mix of alternative care environ-ments including other preschools
III Data and Experimental Impacts
Before turning to an analysis of program substitution issueswe describe the HSIS data and report experimental impacts ontest scores and program compliance
IIIA Data
Our core analysis sample includes 3571 HSIS applicantswith nonmissing baseline characteristics and spring 2003 testscores Online Appendix A describes construction of this sampleThe outcome of interest is a summary index of cognitive testscores that averages Woodcock Johnson III (WJIII) test scoreswith Peabody Picture and Vocabulary Test (PPVT) scores witheach test normed to have mean 0 and variance 1 in the controlgroup by cohort and year We use WJIII and PPVT scores becausethese are among the most reliable tests in the HSIS data both areavailable in each year of the experiment allowing us to producecomparable estimates over time
1 Subsequent analyses of the HSIS data suggest caveats to this negative in-terpretation but do not overturn the finding of modest mean test score impactsaccompanied by rapid fade-out Gelber and Isen (2013) find persistent effects onparental engagement with children Bitler Domina and Hoynes (2014) find largerexperimental impacts at low quantiles of the test score distribution These quantiletreatment effects fade out by first grade though there is some evidence of persistenteffects at the bottom of the distribution for Spanish speakers Walters (2015) findsevidence of substantial heterogeneity in impacts across experimental sites and in-vestigates the relationship between this heterogeneity and observed program char-acteristics Walters finds smaller effects for Head Start centers that draw morechildren from other preschools rather than home care a finding we explore inmore detail here
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1801
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A limitation of our baseline analysis is that it assumeschanges in program scale do not alter the mix of program compli-ers To address this issue we also consider lsquolsquostructural reformsrsquorsquo toHead Start that change the mix of compliers without affectingtest score outcomes Examples of such reforms might include in-creased transportation services marketing efforts or spendingon program features that parents value Households who respondto structural reforms may differ from experimental compliers onunobserved dimensions including their mix of counterfactualprogram choices Assessing these reforms therefore requiresknowledge of parameters not directly identified by the HSIS ex-periment Specifically we show that such reforms require identi-fication of a variant of the marginal treatment effect (MTE)concept of Heckman and Vytlacil (1999)
To assess reforms that attract new children we develop aselection model that parameterizes variation in treatment effectswith respect to counterfactual care alternatives as well as ob-served and unobserved child characteristics We prove that themodel parameters are identified and propose a two-step controlfunction estimator that exploits heterogeneity in the response toHead Start offers across sites and demographic groups to inferrelationships between unobserved factors driving preschool en-rollment and potential outcomes The estimator is shown to passa variety of specification tests and accurately reproduce patternsof treatment effect heterogeneity found in the experiment Themodel estimates indicate that Head Start has large positiveshort-run effects on the test scores of children who would haveotherwise been cared for at home and insignificant effects on chil-dren who would otherwise attend other preschoolsmdasha finding cor-roborated by Feller et al (forthcoming) who reach similarconclusions using principal stratification methods (Frangakisand Rubin 2002) Our estimates also reveal a lsquolsquoreverse Royrsquorsquo pat-tern of selection whereby children with unobserved characteris-tics that make them less likely to enroll in Head Start experiencelarger test score gains
We conclude with an assessment of prospects for increasingHead Startrsquos rate of return via outreach to new populations Ourestimates suggest that expansions of Head Start could boost theprogramrsquos rate of return provided that the proposed technologyfor increasing enrollment (eg improved transportation services)is not too costly We also use our estimated selection model toexamine the robustness of our results to rationing of competing
QUARTERLY JOURNAL OF ECONOMICS1798
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschools Rationing implies that competing subsidized pre-schools do not contract when Head Start expands which shutsdown a form of public savings On the other hand expandingHead Start generates opportunities for new children to fill va-cated seats in substitute programs Our estimates indicate thatthe effect on test scores (and therefore earnings) of moving chil-dren from home care to competing preschools is substantial lead-ing us to conclude that rationing is in fact likely to increase thefavorable estimated rates of return found in our baselineanalysis
The rest of the article is structured as follows Section II pro-vides background on Head Start Section III describes the HSISdata and basic experimental impacts Section IV presents evi-dence on substitution patterns Section V introduces a theoreticalframework for assessing public programs with close substitutesSection VI provides a cost-benefit analysis of Head Start SectionVII develops our econometric selection model and discusses iden-tification and estimation Section VIII reports estimates of themodel Section IX simulates the effects of structural program re-forms Section X concludes
II Background on Head Start
Head Start provides preschool for disadvantaged children inthe United States The program is funded by federal grantsawarded to local public or private organizations Grantees arerequired to match at least 20 of their Head Start awards fromother sources and must meet a set of program-wide performancecriteria Eligibility for Head Start is generally limited to childrenfrom households below the federal poverty line though familiesabove this threshold may be eligible if they meet other criteriasuch as participation in the Temporary Aid for Needy Families(TANF) program Up to 10 of a Head Start centerrsquos enrollmentcan also come from higher-income families The program is freeHead Start grantees are prohibited from charging families feesfor services (US DHHS 2014) It is also oversubscribed in 200285 of Head Start participants attended programs with moreapplicants than available seats (Puma et al 2010)
Head Start is not the only form of subsidized preschool avail-able to poor families Preschool participation rates for disadvan-taged children have risen over time as cities and states expanded
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1799
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
their public preschool offerings (Cascio and Schanzenbach 2013)Moreover the Child Care Development Fund program providesblock grants that finance childcare subsidies for low-income fam-ilies often in the form of child care vouchers that can be used forcenter-based preschool (US DHHS 2012) Most states also useTANF funds to finance additional child care subsidies(Schumacher Greenberg and Duffy 2001) Because Head Startservices are provided by local organizations who themselves mustraise outside funds it is unclear to what extent Head Start andother public preschool programs actually differ in their educationtechnology
A large nonexperimental literature suggests that Head Startproduced large short- and long-run benefits for early cohorts ofprogram participants Several studies estimate the effects ofHead Start by comparing program participants to their nonpar-ticipant siblings (Currie and Thomas 1995 Garces Thomas andCurrie 2002 Deming 2009) Results from this research designshow positive short-run effects on test scores and long-run effectson educational attainment earnings and crime Other studiesexploit discontinuities in Head Start program rules to infer pro-gram effects (Ludwig and Miller 2007 Carneiro and Ginja 2014)These studies show longer run improvements in health outcomesand criminal activity
In contrast to these nonexperimental estimates results froma recent randomized controlled trial reveal smaller less persis-tent effects The 1998 Head Start reauthorization bill included acongressional mandate to determine the effects of the programThis mandate resulted in the HSIS an experiment in which morethan 4000 applicants were randomly assigned via lottery toeither a treatment group with access to Head Start or a controlgroup without access in fall 2002 The experimental resultsshowed that a Head Start offer increased measures of cognitiveachievement by roughly 01 standard deviations during pre-school but these gains faded out by kindergarten Moreoverthe experiment showed little evidence of effects on noncognitiveor health outcomes (Puma et al 2010 Puma Bell and Heid2012) These results suggest both smaller short-run effects andfaster fade-out than nonexperimental estimates for earlier co-horts Scholars and policy makers have generally interpretedthe HSIS results as evidence that Head Start is ineffective andin need of reform (Barnett 2011) The experimental results havealso been cited in the popular media to motivate calls for dramatic
QUARTERLY JOURNAL OF ECONOMICS1800
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
restructuring or elimination of the program (Klein 2011 Stossel2014)1
Differences between the HSIS results and the nonexperi-mental literature could be due to changes in program effective-ness over time or to selection bias in nonexperimental siblingcomparisons Another explanation however is that these tworesearch designs identify different parameters Most nonexperi-mental analyses have focused on recovering the effect of HeadStart relative to home care In contrast the HSIS measures theeffect of Head Start relative to a mix of alternative care environ-ments including other preschools
III Data and Experimental Impacts
Before turning to an analysis of program substitution issueswe describe the HSIS data and report experimental impacts ontest scores and program compliance
IIIA Data
Our core analysis sample includes 3571 HSIS applicantswith nonmissing baseline characteristics and spring 2003 testscores Online Appendix A describes construction of this sampleThe outcome of interest is a summary index of cognitive testscores that averages Woodcock Johnson III (WJIII) test scoreswith Peabody Picture and Vocabulary Test (PPVT) scores witheach test normed to have mean 0 and variance 1 in the controlgroup by cohort and year We use WJIII and PPVT scores becausethese are among the most reliable tests in the HSIS data both areavailable in each year of the experiment allowing us to producecomparable estimates over time
1 Subsequent analyses of the HSIS data suggest caveats to this negative in-terpretation but do not overturn the finding of modest mean test score impactsaccompanied by rapid fade-out Gelber and Isen (2013) find persistent effects onparental engagement with children Bitler Domina and Hoynes (2014) find largerexperimental impacts at low quantiles of the test score distribution These quantiletreatment effects fade out by first grade though there is some evidence of persistenteffects at the bottom of the distribution for Spanish speakers Walters (2015) findsevidence of substantial heterogeneity in impacts across experimental sites and in-vestigates the relationship between this heterogeneity and observed program char-acteristics Walters finds smaller effects for Head Start centers that draw morechildren from other preschools rather than home care a finding we explore inmore detail here
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1801
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschools Rationing implies that competing subsidized pre-schools do not contract when Head Start expands which shutsdown a form of public savings On the other hand expandingHead Start generates opportunities for new children to fill va-cated seats in substitute programs Our estimates indicate thatthe effect on test scores (and therefore earnings) of moving chil-dren from home care to competing preschools is substantial lead-ing us to conclude that rationing is in fact likely to increase thefavorable estimated rates of return found in our baselineanalysis
The rest of the article is structured as follows Section II pro-vides background on Head Start Section III describes the HSISdata and basic experimental impacts Section IV presents evi-dence on substitution patterns Section V introduces a theoreticalframework for assessing public programs with close substitutesSection VI provides a cost-benefit analysis of Head Start SectionVII develops our econometric selection model and discusses iden-tification and estimation Section VIII reports estimates of themodel Section IX simulates the effects of structural program re-forms Section X concludes
II Background on Head Start
Head Start provides preschool for disadvantaged children inthe United States The program is funded by federal grantsawarded to local public or private organizations Grantees arerequired to match at least 20 of their Head Start awards fromother sources and must meet a set of program-wide performancecriteria Eligibility for Head Start is generally limited to childrenfrom households below the federal poverty line though familiesabove this threshold may be eligible if they meet other criteriasuch as participation in the Temporary Aid for Needy Families(TANF) program Up to 10 of a Head Start centerrsquos enrollmentcan also come from higher-income families The program is freeHead Start grantees are prohibited from charging families feesfor services (US DHHS 2014) It is also oversubscribed in 200285 of Head Start participants attended programs with moreapplicants than available seats (Puma et al 2010)
Head Start is not the only form of subsidized preschool avail-able to poor families Preschool participation rates for disadvan-taged children have risen over time as cities and states expanded
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1799
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
their public preschool offerings (Cascio and Schanzenbach 2013)Moreover the Child Care Development Fund program providesblock grants that finance childcare subsidies for low-income fam-ilies often in the form of child care vouchers that can be used forcenter-based preschool (US DHHS 2012) Most states also useTANF funds to finance additional child care subsidies(Schumacher Greenberg and Duffy 2001) Because Head Startservices are provided by local organizations who themselves mustraise outside funds it is unclear to what extent Head Start andother public preschool programs actually differ in their educationtechnology
A large nonexperimental literature suggests that Head Startproduced large short- and long-run benefits for early cohorts ofprogram participants Several studies estimate the effects ofHead Start by comparing program participants to their nonpar-ticipant siblings (Currie and Thomas 1995 Garces Thomas andCurrie 2002 Deming 2009) Results from this research designshow positive short-run effects on test scores and long-run effectson educational attainment earnings and crime Other studiesexploit discontinuities in Head Start program rules to infer pro-gram effects (Ludwig and Miller 2007 Carneiro and Ginja 2014)These studies show longer run improvements in health outcomesand criminal activity
In contrast to these nonexperimental estimates results froma recent randomized controlled trial reveal smaller less persis-tent effects The 1998 Head Start reauthorization bill included acongressional mandate to determine the effects of the programThis mandate resulted in the HSIS an experiment in which morethan 4000 applicants were randomly assigned via lottery toeither a treatment group with access to Head Start or a controlgroup without access in fall 2002 The experimental resultsshowed that a Head Start offer increased measures of cognitiveachievement by roughly 01 standard deviations during pre-school but these gains faded out by kindergarten Moreoverthe experiment showed little evidence of effects on noncognitiveor health outcomes (Puma et al 2010 Puma Bell and Heid2012) These results suggest both smaller short-run effects andfaster fade-out than nonexperimental estimates for earlier co-horts Scholars and policy makers have generally interpretedthe HSIS results as evidence that Head Start is ineffective andin need of reform (Barnett 2011) The experimental results havealso been cited in the popular media to motivate calls for dramatic
QUARTERLY JOURNAL OF ECONOMICS1800
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
restructuring or elimination of the program (Klein 2011 Stossel2014)1
Differences between the HSIS results and the nonexperi-mental literature could be due to changes in program effective-ness over time or to selection bias in nonexperimental siblingcomparisons Another explanation however is that these tworesearch designs identify different parameters Most nonexperi-mental analyses have focused on recovering the effect of HeadStart relative to home care In contrast the HSIS measures theeffect of Head Start relative to a mix of alternative care environ-ments including other preschools
III Data and Experimental Impacts
Before turning to an analysis of program substitution issueswe describe the HSIS data and report experimental impacts ontest scores and program compliance
IIIA Data
Our core analysis sample includes 3571 HSIS applicantswith nonmissing baseline characteristics and spring 2003 testscores Online Appendix A describes construction of this sampleThe outcome of interest is a summary index of cognitive testscores that averages Woodcock Johnson III (WJIII) test scoreswith Peabody Picture and Vocabulary Test (PPVT) scores witheach test normed to have mean 0 and variance 1 in the controlgroup by cohort and year We use WJIII and PPVT scores becausethese are among the most reliable tests in the HSIS data both areavailable in each year of the experiment allowing us to producecomparable estimates over time
1 Subsequent analyses of the HSIS data suggest caveats to this negative in-terpretation but do not overturn the finding of modest mean test score impactsaccompanied by rapid fade-out Gelber and Isen (2013) find persistent effects onparental engagement with children Bitler Domina and Hoynes (2014) find largerexperimental impacts at low quantiles of the test score distribution These quantiletreatment effects fade out by first grade though there is some evidence of persistenteffects at the bottom of the distribution for Spanish speakers Walters (2015) findsevidence of substantial heterogeneity in impacts across experimental sites and in-vestigates the relationship between this heterogeneity and observed program char-acteristics Walters finds smaller effects for Head Start centers that draw morechildren from other preschools rather than home care a finding we explore inmore detail here
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1801
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
their public preschool offerings (Cascio and Schanzenbach 2013)Moreover the Child Care Development Fund program providesblock grants that finance childcare subsidies for low-income fam-ilies often in the form of child care vouchers that can be used forcenter-based preschool (US DHHS 2012) Most states also useTANF funds to finance additional child care subsidies(Schumacher Greenberg and Duffy 2001) Because Head Startservices are provided by local organizations who themselves mustraise outside funds it is unclear to what extent Head Start andother public preschool programs actually differ in their educationtechnology
A large nonexperimental literature suggests that Head Startproduced large short- and long-run benefits for early cohorts ofprogram participants Several studies estimate the effects ofHead Start by comparing program participants to their nonpar-ticipant siblings (Currie and Thomas 1995 Garces Thomas andCurrie 2002 Deming 2009) Results from this research designshow positive short-run effects on test scores and long-run effectson educational attainment earnings and crime Other studiesexploit discontinuities in Head Start program rules to infer pro-gram effects (Ludwig and Miller 2007 Carneiro and Ginja 2014)These studies show longer run improvements in health outcomesand criminal activity
In contrast to these nonexperimental estimates results froma recent randomized controlled trial reveal smaller less persis-tent effects The 1998 Head Start reauthorization bill included acongressional mandate to determine the effects of the programThis mandate resulted in the HSIS an experiment in which morethan 4000 applicants were randomly assigned via lottery toeither a treatment group with access to Head Start or a controlgroup without access in fall 2002 The experimental resultsshowed that a Head Start offer increased measures of cognitiveachievement by roughly 01 standard deviations during pre-school but these gains faded out by kindergarten Moreoverthe experiment showed little evidence of effects on noncognitiveor health outcomes (Puma et al 2010 Puma Bell and Heid2012) These results suggest both smaller short-run effects andfaster fade-out than nonexperimental estimates for earlier co-horts Scholars and policy makers have generally interpretedthe HSIS results as evidence that Head Start is ineffective andin need of reform (Barnett 2011) The experimental results havealso been cited in the popular media to motivate calls for dramatic
QUARTERLY JOURNAL OF ECONOMICS1800
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
restructuring or elimination of the program (Klein 2011 Stossel2014)1
Differences between the HSIS results and the nonexperi-mental literature could be due to changes in program effective-ness over time or to selection bias in nonexperimental siblingcomparisons Another explanation however is that these tworesearch designs identify different parameters Most nonexperi-mental analyses have focused on recovering the effect of HeadStart relative to home care In contrast the HSIS measures theeffect of Head Start relative to a mix of alternative care environ-ments including other preschools
III Data and Experimental Impacts
Before turning to an analysis of program substitution issueswe describe the HSIS data and report experimental impacts ontest scores and program compliance
IIIA Data
Our core analysis sample includes 3571 HSIS applicantswith nonmissing baseline characteristics and spring 2003 testscores Online Appendix A describes construction of this sampleThe outcome of interest is a summary index of cognitive testscores that averages Woodcock Johnson III (WJIII) test scoreswith Peabody Picture and Vocabulary Test (PPVT) scores witheach test normed to have mean 0 and variance 1 in the controlgroup by cohort and year We use WJIII and PPVT scores becausethese are among the most reliable tests in the HSIS data both areavailable in each year of the experiment allowing us to producecomparable estimates over time
1 Subsequent analyses of the HSIS data suggest caveats to this negative in-terpretation but do not overturn the finding of modest mean test score impactsaccompanied by rapid fade-out Gelber and Isen (2013) find persistent effects onparental engagement with children Bitler Domina and Hoynes (2014) find largerexperimental impacts at low quantiles of the test score distribution These quantiletreatment effects fade out by first grade though there is some evidence of persistenteffects at the bottom of the distribution for Spanish speakers Walters (2015) findsevidence of substantial heterogeneity in impacts across experimental sites and in-vestigates the relationship between this heterogeneity and observed program char-acteristics Walters finds smaller effects for Head Start centers that draw morechildren from other preschools rather than home care a finding we explore inmore detail here
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1801
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
restructuring or elimination of the program (Klein 2011 Stossel2014)1
Differences between the HSIS results and the nonexperi-mental literature could be due to changes in program effective-ness over time or to selection bias in nonexperimental siblingcomparisons Another explanation however is that these tworesearch designs identify different parameters Most nonexperi-mental analyses have focused on recovering the effect of HeadStart relative to home care In contrast the HSIS measures theeffect of Head Start relative to a mix of alternative care environ-ments including other preschools
III Data and Experimental Impacts
Before turning to an analysis of program substitution issueswe describe the HSIS data and report experimental impacts ontest scores and program compliance
IIIA Data
Our core analysis sample includes 3571 HSIS applicantswith nonmissing baseline characteristics and spring 2003 testscores Online Appendix A describes construction of this sampleThe outcome of interest is a summary index of cognitive testscores that averages Woodcock Johnson III (WJIII) test scoreswith Peabody Picture and Vocabulary Test (PPVT) scores witheach test normed to have mean 0 and variance 1 in the controlgroup by cohort and year We use WJIII and PPVT scores becausethese are among the most reliable tests in the HSIS data both areavailable in each year of the experiment allowing us to producecomparable estimates over time
1 Subsequent analyses of the HSIS data suggest caveats to this negative in-terpretation but do not overturn the finding of modest mean test score impactsaccompanied by rapid fade-out Gelber and Isen (2013) find persistent effects onparental engagement with children Bitler Domina and Hoynes (2014) find largerexperimental impacts at low quantiles of the test score distribution These quantiletreatment effects fade out by first grade though there is some evidence of persistenteffects at the bottom of the distribution for Spanish speakers Walters (2015) findsevidence of substantial heterogeneity in impacts across experimental sites and in-vestigates the relationship between this heterogeneity and observed program char-acteristics Walters finds smaller effects for Head Start centers that draw morechildren from other preschools rather than home care a finding we explore inmore detail here
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1801
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Table I provides summary statistics for our analysis sampleThe HSIS experiment included two age cohorts 55 of applicantswere randomized at age three and could attend Head Start for upto two years while the remaining 45 were randomized at age fourand could attend for up to one year The demographic informationin Table I shows that the Head Start population is disadvantagedLess than half of Head Start applicants live in two-parent house-holds and the average applicantrsquos household earns about 90 ofthe federal poverty line Column (2) of Table I compares these andother baseline characteristics for the HSIS treatment and controlgroups to check balance in randomization The results here indi-cate that randomization was successful baseline characteristicswere similar for offered and non offered applicants2
Columns (3) through (5) of Table I report summary statisticsfor children attending Head Start other preschool centers andno preschool3 Children in other preschools tend to be less disad-vantaged than children in Head Start or no preschool thoughmost differences between these groups are modest The otherpreschool group has a lower share of high school dropout mothersa higher share of mothers who attended college and higher av-erage household income than the Head Start and no preschoolgroups Children in other preschools outscore the other groups byabout 01 standard deviations on a baseline summary index ofcognitive skills The other preschool group also includes a rela-tively large share of four-year-olds likely reflecting the fact thatalternative preschool options are more widely available for four-year-olds (Cascio and Schanzenbach 2013)
IIIB Experimental Impacts
Table II reports experimental impacts on test scoresColumns (1) (4) and (7) report intent-to-treat impacts of the
2 Random assignment in the HSIS occurred at the Head Start center leveland offer probabilities differed across centers We weight all models by the inverseprobability of a childrsquos assignment calculated as the site-specific fraction of chil-dren assigned to the treatment group
3 Preschool attendance is measured from the HSIS lsquolsquofocal arrangement typersquorsquovariable which combines information from parent interviews and teachercareprovider interviews to construct a summary measure of the childcare setting SeeOnline Appendix A for details
QUARTERLY JOURNAL OF ECONOMICS1802
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
DE
SC
RIP
TIV
ES
TA
TIS
TIC
S
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
Male
04
94
05
05
00
11
05
01
05
06
04
92
(00
19)
Bla
ck03
08
02
98
00
10
03
17
03
53
02
50
(00
10)
His
pan
ic03
76
03
69
00
07
03
80
03
54
03
73
(00
10)
Tee
nm
oth
er01
59
01
74
00
15
01
59
01
69
01
76
(00
14)
Mot
her
marr
ied
04
36
04
48
00
11
04
39
04
20
04
60
(00
17)
Bot
hp
are
nts
inh
ouse
hol
d04
97
04
88
00
09
04
97
04
68
04
99
(00
17)
Mot
her
ish
igh
sch
ool
dro
pou
t03
68
03
97
00
29
03
77
03
22
04
26
(00
17)
Mot
her
att
end
edso
me
coll
ege
02
98
02
81
00
17
02
93
03
42
02
53
(00
16)
Sp
an
ish
spea
ker
02
87
02
73
00
14
02
96
02
74
02
60
(00
11)
Sp
ecia
led
uca
tion
01
36
01
08
00
28
01
34
01
45
00
91
(00
11)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1803
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EI
(CO
NT
INU
ED)
By
offe
rst
atu
sB
yp
resc
hoo
lch
oice
(1)
(2)
(3)
(4)
(5)
(6)
Vari
able
Off
ered
mea
nN
onof
fere
dm
ean
Dif
fere
nti
al
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
l
On
lych
ild
01
61
01
39
00
22
01
51
01
90
01
23
(00
12)
Inco
me
(fra
ctio
nof
FP
L)
08
96
08
96
00
00
08
92
09
83
08
51
(00
24)
Age
4co
hor
t04
48
04
51
00
03
04
26
05
67
04
13
(00
12)
Base
lin
esu
mm
ary
ind
ex00
03
00
12
00
09
00
01
01
06
00
40
(00
27)
Urb
an
08
33
08
35
00
02
08
34
08
59
08
19
(00
03)
Cen
ter
pro
vid
estr
an
spor
tati
on06
06
06
04
00
02
05
86
06
14
06
28
(00
05)
Cen
ter
qu
ali
tyin
dex
04
65
04
70
00
05
04
52
04
74
04
88
(00
05)
Joi
nt
p-v
alu
e05
06
N22
56
13
15
35
71
20
43
598
930
Not
es
All
stati
stic
sw
eigh
tby
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tS
tan
dard
erro
rsare
clu
ster
edat
the
cen
ter
level
T
he
tran
spor
tati
onan
dqu
ali
tyin
dex
vari
able
sre
fer
toa
chil
drsquos
Hea
dS
tart
cen
ter
ofra
nd
omass
ign
men
tT
he
qu
ali
tyvari
able
com
bin
esin
form
ati
onon
cen
ter
chara
cter
isti
cs(t
each
eran
dce
nte
rd
irec
tor
edu
cati
onan
dqu
ali
fica
tion
scl
ass
size
)an
dp
ract
ices
(vari
ety
ofli
tera
cyan
dm
ath
act
ivit
ies
hom
evis
itin
g
hea
lth
an
dn
utr
itio
n)
Inco
me
ism
issi
ng
for
19
ofob
serv
ati
ons
Mis
sin
gvalu
esare
excl
ud
edin
stati
stic
sfo
rin
com
eT
he
base
lin
esu
mm
ary
ind
exis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
infa
ll2002
wit
hea
chsc
ore
stan
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tT
he
join
tp
-valu
eis
from
ate
stof
the
hyp
oth
esis
that
all
coef
fici
ents
equ
al
0
QUARTERLY JOURNAL OF ECONOMICS1804
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
EX
PE
RIM
EN
TA
LIM
PA
CT
SO
NT
ES
TS
CO
RE
S
Th
ree-
yea
r-ol
dco
hor
tF
our-
yea
r-ol
dco
hor
tC
ohor
tsp
oole
d
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
Tim
ep
erio
dR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IVR
edu
ced
form
Fir
stst
age
IV
Yea
r1
01
94
06
99
02
78
01
41
06
63
02
13
01
68
06
82
02
47
(00
29)
(00
25)
(00
41)
(00
29)
(00
22)
(00
44)
(00
21)
(00
18)
(00
31)
N19
70
16
01
35
71
Yea
r2
00
87
03
56
02
45
00
15
06
70
00
22
00
46
04
97
00
93
(00
29)
(00
28)
(00
80)
(00
37)
(00
23)
(00
54)
(00
24)
(00
20)
(00
49)
N17
60
14
16
31
76
Yea
r3
00
10
03
65
00
27
00
54
06
66
00
81
00
19
05
00
00
38
(00
31)
(00
28)
(00
85)
(00
40)
(00
25)
(00
60)
(00
25)
(00
20)
(00
50)
N16
59
13
36
29
95
Yea
r4
00
38
03
44
01
10
mdashmdash
(00
34)
(00
29)
(00
98)
N15
99
Not
es
Th
ista
ble
rep
orts
exp
erim
enta
les
tim
ate
sof
the
effe
cts
ofH
ead
Sta
rton
test
scor
es
Th
eou
tcom
eis
the
aver
age
ofst
an
dard
ized
PP
VT
an
dW
JII
Isc
ores
w
ith
each
scor
est
an
dard
ized
toh
ave
mea
n0
an
dst
an
dard
dev
iati
on1
inth
eco
ntr
olgro
up
sep
ara
tely
by
ap
pli
can
tco
hor
tan
dyea
rC
olu
mn
s(1
)(4
)an
d(7
)re
por
tco
effi
cien
tsfr
omre
gre
ssio
ns
ofte
stsc
ores
onan
ind
icato
rfo
rass
ign
men
tto
Hea
dS
tart
C
olu
mn
s(2
)(5
)an
d(8
)re
por
tco
effi
cien
tsfr
omfi
rst-
stage
regre
ssio
ns
ofH
ead
Sta
rtatt
end
an
ceon
Hea
dS
tart
ass
ign
men
tT
he
att
end
an
cevari
able
isan
ind
icato
req
ual
to1
ifa
chil
datt
end
sH
ead
Sta
rtat
an
yti
me
pri
orto
the
test
C
olu
mn
s(3
)(6
)an
d(9
)re
por
tco
effi
cien
tsfr
omtw
o-st
age
least
squ
are
s(2
SL
S)
mod
els
that
inst
rum
ent
Hea
dS
tart
att
end
an
cew
ith
Hea
dS
tart
ass
ign
men
tA
llm
odel
sw
eigh
tby
the
reci
pro
cal
ofa
chil
drsquos
exp
erim
enta
lass
ign
-m
ent
an
dco
ntr
olfo
rse
x
race
S
pan
ish
lan
gu
age
teen
mot
her
m
oth
errsquos
mari
tal
statu
sp
rese
nce
ofbot
hp
are
nts
inth
eh
ome
fam
ily
size
sp
ecia
led
uca
tion
statu
sin
com
equ
art
ile
du
mm
ies
urb
an
an
da
cubic
pol
yn
omia
lin
base
lin
esc
ore
Mis
sin
gvalu
esfo
rco
vari
ate
sare
set
to0
an
dd
um
mie
sfo
rm
issi
ng
are
incl
ud
ed
Sta
nd
ard
erro
rsare
clu
ster
edby
cen
ter
ofra
nd
omass
ign
men
t
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1805
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Head Start offer separately by year and age cohort To increaseprecision we regression-adjust these treatmentcontrol differ-ences using the baseline characteristics in Table I4 Theintent-to-treat estimates mirror those previously reported inthe literature (eg Puma et al 2010) In the first year of theexperiment children offered Head Start scored higher on thesummary index For example three-year-olds offered HeadStart gained 019 standard deviations in test score outcomesrelative to those denied Head Start The corresponding effectfor four-year-olds is 014 standard deviations However thesegains diminish rapidly the pooled impact falls to a statisticallyinsignificant 002 standard deviations by year 3 Our data in-clude a fourth year of follow-up for the three-year-old cohortHere too the intent-to-treat is small and statistically insignif-icant (0038 standard deviations)
Interpretation of these intent-to-treat impacts is clouded bynoncompliance with random assignment Columns (2) (5) and(8) of Table II report first-stage effects of assignment to HeadStart on the probability of participating in Head Start and col-umns (3) (6) and (9) report instrumental variables (IV) esti-mates which scale the intent-to-treat estimates by the first-stage estimates5 These estimates can be interpreted as local av-erage treatment effects (LATEs) for lsquolsquocompliersrsquorsquomdashchildren whorespond to the Head Start offer by enrolling in Head StartAssignment to Head Start increases the probability of participa-tion by two thirds in the first year after random assignment The
4 The control vector includes sex race assignment cohort teen mothermotherrsquos education motherrsquos marital status presence of both parents an onlychild dummy a Spanish language indicator dummies for quartiles of familyincome and missing income urban status an indicator for whether the HeadStart center provides transportation an index of Head Start center quality and athird-order polynomial in baseline test scores
5 Here we define Head Start participation as enrollment at any time prior tothe test This definition includes attendance at Head Start centers outside the ex-perimental sample An experimental offer may cause some children to switch froman out-of-sample center to an experimental center if the quality of these centersdiffers the exclusion restriction required for our IV approach is violated OnlineAppendix Table AI compares characteristics of centers attended by children in thecontrol group (always takers) to those of the experimental centers to which thesechildren applied These two groups of centers are very similar suggesting thatsubstitution between Head Start centers is unlikely to bias our estimates
QUARTERLY JOURNAL OF ECONOMICS1806
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
corresponding IV estimate implies that Head Start attendanceboosts first-year test scores by 0247 standard deviations
Compliance for the three-year-old cohort falls after the firstyear as members of the control group reapply for Head Startresulting in larger standard errors for estimates in later yearsof the experiment The first stage for three-year-olds falls to 036in the second year whereas the intent-to-treat falls roughly inproportion generating a second-year IV estimate of 0245 for thiscohort Estimates in years 3 and 4 are statistically insignificantand imprecise The fourth-year estimate for the three-year-oldcohort (corresponding to first grade) is 0110 standard deviationswith a standard error of 0098 The corresponding first grade es-timate for four-year-olds is 0081 with a standard error of 0060Notably the 95 confidence intervals for first-grade impacts in-clude effects as large as 02 standard deviations for four-year-oldsand 03 standard deviations for three-year-olds These resultsshow that although the longer run estimates are insignificantthey are also imprecise due to experimental noncomplianceEvidence for fade-out is therefore less definitive than the naiveintent-to-treat estimates suggest This observation helps to rec-oncile the HSIS results with observational studies based on sib-ling comparisons which show effects that partially fade out butare still detectable in elementary school (Currie and Thomas1995 Deming 2009)6
IV Program Substitution
We turn now to documenting program substitution in theHSIS and how it influences our results It is helpful to developsome notation to describe the role of alternative care environ-ments Each Head Start applicant participates in one of three pos-sible treatments Head Start which we label h other center-based
6 One might also be interested in the effects of Head Start on noncognitiveoutcomes which appear to be important mediators of the effects of early childhoodprograms in other contexts (Chetty et al 2011 Heckman Pinto and Savelyev2013) The HSIS includes short-run parent-reported measures of behavior andteacher-reported measures of teacherndashstudent relationships and Head Start ap-pears to have no impact on these outcomes (Puma et al 2010 Walters 2015) TheHSIS noncognitive outcomes differ significantly from those analyzed in previousstudies however and it is unclear whether they capture the same skills
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1807
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
preschool programs which we label c and no preschool (ie homecare) which we label n Let Zi 2 f01g indicate whether household ihas a Head Start offer and DiethzTHORN 2 fhcng denote household irsquospotential treatment status as a function of the offer Then observedtreatment status can be written Di frac14 DiethZiTHORN
The structure of the HSIS leads to natural theoretical restric-tions on substitution patterns We expect a Head Start offer toinduce some children who would otherwise participate in c or n toenroll in Head Start By revealed preference no child shouldswitch between c and n in response to a Head Start offer andno child should be induced by an offer to leave Head Start Theserestrictions can be expressed succinctly by the followingcondition
Dieth1THORN 6frac14 Dieth0THORN ) Dieth1THORN frac14 heth1THORN
which extends the monotonicity assumption of Imbens andAngrist (1994) to a setting with multiple counterfactual treat-ments This restriction states that anyone who changes behav-ior as a result of the Head Start offer does so to attend HeadStart7
Under restriction (1) the population of Head Start applicantscan be partitioned into five groups defined by their values of Dieth1THORNand Dieth0THORN
(i) n-compliers Dieth1THORN frac14 h Dieth0THORN frac14 n(ii) c-compliers Dieth1THORN frac14 h Dieth0THORN frac14 c
(iii) n-never takers Dieth1THORN frac14 Dieth0THORN frac14 n(iv) c-never takers Dieth1THORN frac14 Dieth0THORN frac14 c(v) always takers Dieth1THORN frac14 Dieth0THORN frac14 h
The n- and c-compliers switch to Head Start from home care andcompeting preschools respectively when offered a seat The twogroups of never takers choose not to attend Head Start regard-less of the offer Always takers manage to enroll in Head Starteven when denied an offer presumably by applying to otherHead Start centers outside the HSIS sample
Using this rubric the group of children enrolled in alterna-tive preschool programs is a mixture of c-never takers and c-com-pliers denied Head Start offers Similarly the group of children in
7 See Engberg et al (2014) for discussion of related restrictions in the contextof attrition from experimental data
QUARTERLY JOURNAL OF ECONOMICS1808
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
home care includes n-never takers and n-compliers withoutoffers The two complier subgroups switch into Head Startwhen offered admission as a result the set of children enrolledin Head Start is a mixture of always takers and the two groups ofoffered compliers
IVA Substitution Patterns
Table III presents empirical evidence on substitution pat-terns by comparing program participation choices for offeredand nonoffered households In the first year of the experiment83 of households decline Head Start offers in favor of otherpreschool centers this is the share of c-never takers Similarlycolumn (3) shows that 95 of households are n-never takers Ascan be seen in column (4) 136 of households manage to attendHead Start without an offer which is the share of always takersThe Head Start offer reduces the share of children in other cen-ters from 315 to 83 and reduces the share of children inhome care from 55 to 95 This implies that 232 of house-holds are c-compliers and 455 are n-compliers
Notably in the first year of the experiment three-year-oldshave uniformly higher participation rates in Head Start andlower participation rates in competing centers which likely re-flects the fact that many state-provided programs only acceptfour-year-olds In the second year of the experiment participa-tion in Head Start drops among children in the three-year-oldcohort with a program offer suggesting that many families en-rolled in the first year decided that Head Start was a bad matchfor their child We also see that Head Start enrollment risesamong those families that did not obtain an offer in the firstround which reflects reapplication behavior
IVB Interpreting IV
How do the substitution patterns displayed in Table III affectthe interpretation of the HSIS test score impacts Let YiethdTHORNdenote child irsquos potential test score if he or she participates intreatment d 2 fhcng Observed scores are given by Yi frac14 YiethDiTHORNWe assume that Head Start offers affect test scores only throughprogram participation choices Under assumption (1) IV identi-fies a variant of the LATE of Imbens and Angrist (1994) givingthe average effect of Head Start participation for compliers
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1809
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EII
I
PR
ES
CH
OO
LC
HO
ICE
SB
YY
EA
R
CO
HO
RT
AN
DO
FF
ER
ST
AT
US
Off
ered
Not
offe
red
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Tim
ep
erio
dC
ohor
tH
ead
Sta
rtO
ther
cen
ters
No
pre
sch
ool
Hea
dS
tart
Oth
erce
nte
rsN
op
resc
hoo
lC
-com
pli
ersh
are
Yea
r1
3-y
ear-
old
s08
51
00
58
00
92
01
47
02
56
05
97
02
82
4-y
ear-
old
s07
87
01
14
00
99
01
22
03
86
04
92
04
10
Poo
led
08
22
00
83
00
95
01
36
03
15
05
50
03
38
Yea
r2
3-y
ear-
old
s06
57
02
62
00
81
04
94
03
79
01
27
07
19
Not
es
Th
ista
ble
rep
orts
share
sof
offe
red
an
dn
onof
fere
dst
ud
ents
att
end
ing
Hea
dS
tart
ot
her
cen
ter-
base
dp
resc
hoo
ls
an
dn
op
resc
hoo
lse
para
tely
by
yea
ran
dage
coh
ort
All
stati
stic
sare
wei
gh
ted
by
the
reci
pro
cal
ofth
ep
robabil
ity
ofa
chil
drsquos
exp
erim
enta
lass
ign
men
tC
olu
mn
(7)
rep
orts
esti
mate
sof
the
share
ofco
mp
lier
sd
raw
nfr
omot
her
pre
sch
ools
giv
enby
min
us
the
rati
oof
the
offe
rrsquos
effe
cton
att
end
an
ceat
oth
erp
resc
hoo
lsto
its
effe
cton
Hea
dS
tart
att
end
an
ce
QUARTERLY JOURNAL OF ECONOMICS1810
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
relative to a mix of program alternatives Specifically underequation (1) and excludability of Head Start offers
E YijZi frac14 1frac12 E YijZi frac14 0frac12
E 1 Di frac14 hf gjZi frac14 1frac12 E 1 Di frac14 hf gjZi frac14 0frac12
frac14 E YiethhTHORN YiethDieth0THORNTHORNjDieth1THORN frac14 hDieth0THORN 6frac14 hfrac12
LATEh
eth2THORN
The left-hand side of equation (2) is the population coefficientfrom a model that instruments Head Start attendance with theHead Start offer This equation implies that the IV strategyemployed in Table II yields the average effect of Head Startfor compliers relative to their own counterfactual care choicesa quantity we label LATEh
We can decompose LATEh into a weighted average oflsquolsquosubLATEsrsquorsquo measuring the effects of Head Start for compliersdrawn from specific counterfactual alternatives as follows
LATEh frac14 ScLATEch thorn eth1 ScTHORNLATEnheth3THORN
where LATEdh E YiethhTHORN YiethdTHORNjDieth1THORN frac14 hDieth0THORN frac14 dfrac12 gives theaverage treatment effect on d-compliers for d 2 fcng and the
weight Sc P Di 1eth THORNfrac14hDi 0eth THORNfrac14ceth THORN
P Di 1eth THORNfrac14hDi 0eth THORN6frac14heth THORNgives the fraction of compliers drawn
from other preschoolsColumn (7) of Table III reports estimates of Sc by year and
cohort computed as minus the ratio of the Head Start offerrsquoseffect on other preschool attendance to its effect on Head Startattendance (see Online Appendix B) In the first year of the HSISexperiment 34 of compliers would have otherwise attendedcompeting preschools IV estimates combine effects for these com-pliers with effects for compliers who would not otherwise attendpreschool
As detailed in Online Appendix D the competing preschoolsattended by c-compliers are largely publicly funded and provideservices roughly comparable to Head Start The modal alterna-tive preschool is a state-provided preschool program whereasothers receive funding from a mix of public sources (see OnlineAppendix Table AII) Moreover it is likely that even Head Startndasheligible children attending private preschool centers receivepublic funding (eg through CCDF or TANF subsidies) Nextwe consider the implications of substitution from these alterna-tive preschools for assessments of Head Startrsquos cost-effectiveness
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1811
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
V A Model of Head Start Provision
In this section we develop a model of Head Start participationwith the goal of conducting a cost-benefit analysis that acknowl-edges the presence of publicly subsidized program substitutes Ourmodel is highly stylized and focuses on obtaining an estimablelower bound on the rate of return to potential reforms of HeadStart measured in terms of lifetime earnings The analysis ignoresredistributional motives and any effects of human capital invest-ment on criminal activity (Lochner and Moretti 2004 Heckmanet al 2010) health (Deming 2009 Carneiro and Ginja 2014) orgrade repetition (Currie 2001) Adding such features would tend toraise the implied return to Head Start We also abstract from pa-rental labor supply decisions because prior analyses of the HSISfind no short-term impacts on parentsrsquo work decisions (Puma et al2010)8 Again incorporating parental labor supply responseswould likely raise the programrsquos rate of return
Our discussion emphasizes that the cost-effectiveness of HeadStart is contingent on assumptions regarding the structure of themarket for preschool services and the nature of the specific policyreforms under consideration Building on the heterogeneous ef-fects framework of the previous section we derive expressionsfor policy relevant lsquolsquosufficient statisticsrsquorsquo (Chetty 2009) in terms ofcausal effects on student outcomes Specifically we show that avariant of the LATE concept of Imbens and Angrist (1994) is policyrelevant when considering program expansions in an environmentwhere slots in competing preschools are not rationed With ration-ing a mix of LATEs becomes relevant which poses challenges toidentification with the HSIS data When considering reforms toHead Start program features that change selection into the pro-gram the policy-relevant parameter is shown to be a variant of theMTE concept of Heckman and Vytlacil (1999)
VA Setup
Consider a population of households indexed by i each witha single preschool-aged child Each household can enroll itschild in Head Start enroll in a competing preschool program
8 We replicate this analysis for our sample in Online Appendix Table AIIIwhich shows that a Head Start offer has no effect on the probability that a childrsquosmother works or on the likelihood of working full- versus part-time Recent work byLong (2015) suggests that Head Start may have small effects on full- versus part-time work for mothers of three-year-olds
QUARTERLY JOURNAL OF ECONOMICS1812
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(eg state-subsidized preschool) or care for the child at home Thegovernment rations Head Start participation via program offersZi which arrive at random via lottery with probability P Zi frac14 1eth THORN Offers are distributed in a first period In a secondperiod households make enrollment decisions Tenacious appli-cants who have not received an offer can enroll in Head Start byexerting additional effort We begin by assuming that competingprograms are not rationed and then relax this assumption later
Each household has utility over its enrollment options givenby the function Ui dzeth THORN The argument d 2 hcnf g indexes childcare environments and the argument z 2 0 1f g indexes offerstatus Head Start offers raise the value of Head Start and haveno effect on the value of other options so that
Ui h1eth THORN gt Ui h0eth THORNUi czeth THORN frac14 Ui ceth THORNUi nzeth THORN frac14 Ui neth THORN
Household irsquos enrollment choice as a function of its offer statusz is given by
Di zeth THORN frac14 arg maxd2 hcnf g
Ui dzeth THORN
It is straightforward to show that this model satisfies the mono-tonicity restriction (1) Because offers are assigned at randommarket shares for the three care environments can be writtenP Di frac14 deth THORN frac14 P Di 1eth THORN frac14 deth THORN thorn 1 eth THORNP Di 0eth THORN frac14 deth THORN
VB Benefits and Costs
Debate over the effectiveness of educational programs oftencenters on their test score impacts A standard means of valuingsuch impacts is in terms of their effects on later life earnings(Heckman et al 2010 Chetty et al 2011 Heckman Pinto andSavelyev 2013 Chetty Friedman and Rockoff 2014b)9 Let thesymbol B denote the total after-tax lifetime income of a cohort ofchildren We assume that B is linked to test scores by theequation
B frac14 B0 thorn 1 eth THORNpE Yifrac12 eth4THORN
where p gives the market price of human capital is the taxrate faced by the children of eligible households and B0 is anintercept reflecting how test scores are normed Our focus on
9 Online Appendix C considers how such valuations should be adjusted whentest score impacts yield labor supply responses
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1813
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
mean test scores neglects distributional concerns that may leadus to undervalue Head Startrsquos test score impacts (see BitlerDomina and Hoynes 2014)
The net costs to government of financing preschool are given by
C frac14 C0 thorn rsquohP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth5THORN
where the term C0 reflects fixed costs of administering the pro-gram and rsquoh gives the administrative cost of providing HeadStart services to an additional child Likewise rsquoc gives theadministrative cost to government of providing competing pre-school services (which often receive public subsidies) to anotherstudent The term pE Yifrac12 captures the revenue generated bytaxes on the adult earnings of Head Startndasheligible childrenThis formulation abstracts from the fact that program outlaysmust be determined before the children enter the labor marketand begin paying taxes a complication we adjust for in ourempirical work via discounting
VC Changing Offer Probabilities
Consider the effects of adjusting Head Start enrollment bychanging the rationing probability An increase in draws ad-ditional households into Head Start from competing programsand home care As shown in Online Appendix C the effect of achange in the offer rate on average test scores is given by
qE Yifrac12
qfrac14 LATEh|fflfflfflfflzfflfflfflffl
Effect on compliers
qP Di frac14 heth THORN
q|fflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflComplier density
eth6THORN
In words the aggregate impact on test scores of a small in-crease in the offer rate equals the average impact of HeadStart on complier test scores times the measure of compliersBy the arguments in Section IV both LATEh and q
qP Di frac14 heth THORN
frac14 P Di 1eth THORN frac14 hDi 0eth THORN 6frac14 heth THORN are identified by the HSIS experimentHence equation (6) implies that the hypothetical effects of amarket-level change in offer probabilities can be inferred froman individual-level randomized trial with a fixed offer probabil-ity This convenient result follows from the assumption thatHead Start offers are distributed at random and that doesnot directly enter the alternative specific choice utilitieswhich in turn implies that the composition of compliers (andhence LATEh) does not change with Later we explore how
QUARTERLY JOURNAL OF ECONOMICS1814
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
this expression changes when the composition of compliers re-sponds to a policy change
From equation (4) the marginal benefit of an increase in isgiven by
qB
qfrac14 1 eth THORNpLATEh
qP Di frac14 heth THORN
q
The offsetting marginal cost to government of financing such anexpansion can be written
qC
qfrac14 rsquoh|z
Provision Cost
rsquocSc|fflzfflPublic Savings
pLATEh|fflfflfflfflfflfflfflzfflfflfflfflfflfflfflAdded Revenue
0BBB
1CCCA qP Di frac14 heth THORN
qeth7THORN
This cost consists of the measure of compliers times the admin-istrative cost rsquoh of enrolling them in Head Start minus theprobability Sc that a complying household comes from a substi-tute preschool times the expected government savings rsquoc asso-ciated with reduced enrollment in substitute preschools Thequantity rsquoh rsquocSc can be viewed as a LATE of Head Start ongovernment spending for compliers Subtracted from this effectis any extra revenue the government gets from raising the pro-ductivity of the children of complying households
The ratio of marginal impacts on after-tax income and gov-ernment costs gives the marginal value of public funds (Mayshar1990 Hendren 2016) which we can write
MVPF qBqqCq
frac141 eth THORNpLATEh
rsquoh rsquocSc pLATEheth8THORN
The MVPF gives the value of an extra dollar spent on HeadStart net of fiscal externalities These fiscal externalities in-clude reduced spending on competing subsidized programs (cap-tured by the term rsquocSc) and additional tax revenue generatedby higher earnings (captured by pLATEh) As emphasized byHendren (2016) the MVPF is a metric that can easily be com-pared across programs without specifying exactly how programexpenditures are to be funded In our case if MVPF gt 1 $1 ofgovernment spending can raise the after-tax incomes of chil-dren by more than $1 which is a robust indicator that programexpansions are likely to be welfare improving
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1815
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
An important lesson of the foregoing analysis is that iden-tifying costs and benefits of changes to offer probabilities doesnot require identification of treatment effects relative to partic-ular counterfactual care states Specifically it is not necessaryto separately identify the subLATEs This result shows thatprogram substitution is not a design flaw of evaluationsRather it is a feature of the policy environment that needs tobe considered when computing the likely effects of changes topolicy parameters Here program substitution alters the usuallogic of program evaluation only by requiring identification ofthe complier share Sc which governs the degree of public sav-ings realized as a result of reducing subsidies to competingprograms
VD Rationed Substitutes
The foregoing analysis presumes that Head Start expansionsyield reductions in the enrollment of competing preschoolsHowever if competing programs are also oversubscribed the slotsvacated by c-compliers may be filled by other households This willreduce the public savings associated with Head Start expansionsbut also generate the potential for additional test score gains
With rationing in substitute preschool programs the utilityof enrollment in c can be written UiethcZicTHORN where Zic indicates anoffered slot in the competing program Household irsquos enrollmentchoice DiethZihZicTHORN depends on both the Head Start offer Zih andthe competing program offer Assume these offers are assignedindependently with probabilities h and c but that c adjusts tochanges in h to keep total enrollment in c constant In additionassume that all children induced to move into c as a result of anincrease in c come from n rather than h
We show in Online Appendix C that under these assumptionsthe marginal impact of expanding Head Start becomes
qE Yifrac12
qhfrac14 LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
where LATEnc E Yi ceth THORN Yi neth THORNjDi Zih1eth THORN frac14 cDi Zih0eth THORN frac14 nfrac12 Intuitively every c-complier now spawns a corresponding n-to-c complier who fills the vacated preschool slot
The marginal cost to government of inducing this change intest scores can be written
QUARTERLY JOURNAL OF ECONOMICS1816
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
qC
qhfrac14 rsquoh p LATEh thorn LATEnc Sceth THORN
qP Di frac14 heth THORN
qh
Relative to equation (7) rationing eliminates the public savingsfrom reduced enrollment in substitute programs but adds an-other fiscal externality in its place the tax revenue associatedwith any test score gains of shifting children from home care tocompeting preschools The resulting marginal value of publicfunds can be written
MVPFrat frac14eth1 THORNp LATEh thorn LATEnc Sceth THORN
rsquoh p LATEh thorn LATEnc Sceth THORNeth9THORN
While the impact of rationed substitutes on the marginal valueof public funds is theoretically ambiguous there is good reasonto expect MVPFrat gt MVPF in practice Specifically ignoringrationing of competing programs yields a lower bound onthe rate of return to Head Start expansions if Head Start andother forms of center-based care have roughly comparable effectson test scores and competing programs are cheaper (see OnlineAppendix C) Unfortunately effects for n-to-c compliers are notnonparametrically identified by the HSIS experiment becauseone cannot know which households that care for their childrenat home would otherwise choose to enroll them in competingpreschools We return to this issue in Section IX
VE Structural Reforms
An important assumption in the previous analyses is thatchanging lottery probabilities does not alter the mix of programcompliers Consider now the effects of altering some structuralfeature f of the Head Start program that households value buthas no impact on test scores For example Executive Order13330 issued by President George W Bush in February 2004mandated enhancements to the transportation services providedby Head Start and other federal programs (Federal Register2004) Expanding Head Start transportation services should notdirectly influence educational outcomes but may yield a composi-tional effect by drawing in households from a different mix ofcounterfactual care environments10 By shifting the composition
10 This presumes that peer effects are not an important determinant of testscore outcomes Large changes in the student composition of Head Start classroomscould potentially change the effectiveness of Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1817
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
of program participants changes in f may boost the programrsquos rateof return
To establish notation we assume that households now valueHead Start participation as
UiethhZi f THORN frac14 Ui hZieth THORN thorn f
Utilities for other preschools and home care are assumed to beunaffected by changes in f This implies that increases in fmake Head Start more attractive for all households Forsimplicity we return to our prior assumption that competingprograms are not rationed As shown in Online Appendix Cthe assumption that f has no effect on potential outcomesimplies
qE Yifrac12
qffrac14MTEh
qP Di frac14 heth THORN
qf
where
MTEh E YiethhTHORN Yi ceth THORNjUiethhZiTHORN thorn f frac14 UiethcTHORNUiethcTHORN gt UiethnTHORNfrac12 ~Sc
thorn E YiethhTHORN YiethnTHORNjUiethhZiTHORN thorn f frac14 UiethnTHORNUiethnTHORN gt UiethcTHORNfrac12 eth1 ~ScTHORN
and ~Sc gives the share of children on the margin of participat-ing in Head Start who prefer the competing program topreschool nonparticipation Following the terminology inHeckman Urzua and Vytlacil (2008) the marginal treatmenteffect MTEh is the average effect of Head Start on test scoresamong households indifferent between Head Start and the nextbest alternative This is a marginal version of the result inequation (6) where integration is now over a set of childrenwho may differ from current program compliers in their meanimpacts Like LATEh MTEh is a weighted average oflsquolsquosubMTEsrsquorsquo corresponding to whether the next best alternativeis home care or a competing preschool program The weight ~Sc
may differ from Sc if inframarginal participants are drawn fromdifferent sources than marginal ones
The test score effects of improvements to the program featuremust be balanced against the costs We suppose that changingprogram features changes the average cost rsquoh feth THORN of Head Startservices so that the net costs to government of financing pre-school are now
QUARTERLY JOURNAL OF ECONOMICS1818
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
C feth THORN frac14 C0 thorn rsquoh feth THORNP Di frac14 heth THORN thorn rsquocP Di frac14 ceth THORN pE Yifrac12 eth10THORN
where qrsquoh feth THORNqf 0 The marginal costs to government (per program
complier) of a change in the program feature can be written
qC feth THORN
qfqP Di frac14 heth THORN
qf
frac14 rsquoh|zMarginal Provision Cost
thorn
qrsquoh feth THORN
qfqln P Di frac14 heth THORN
qf|fflfflfflfflfflfflfflfflfflfflfflzfflfflfflfflfflfflfflfflfflfflfflInframarginal Provision Cost
rsquoc~Sc|fflzffl
Public Savings
pMTEh|fflfflfflfflfflfflzfflfflfflfflfflfflAdded Revenue
eth11THORN
The first term on the right-hand side of equation (11) gives theadministrative cost of enrolling another child The second termgives the increased cost of providing inframarginal familieswith the improved program feature The third term is the ex-pected savings in reduced funding to competing preschool pro-grams The final term gives the additional tax revenue raisedby the boost in the marginal enrolleersquos human capital
Letting qln rsquoethf THORN
qfqln PethDi frac14 hTHORN
qf
be the elasticity of costs with respect to
enrollment we can write the marginal value of public funds as-sociated with a change in program features as
MVPFf
qBqf
qC feth THORNqf
frac141 eth THORNpMTEh
rsquoh 1thorn eth THORN rsquoc~Sc pMTEh
eth12THORN
As in our analysis of optimal program scale equation (11) showsthat it is not necessary to separately identify the subMTEs thatcompose MTEh to determine the optimal value of f Rather it issufficient to identify the average causal effect of Head Start forchildren on the margin of participation along with the averagenet cost of an additional seat in this population
VI A Cost-Benefit Analysis of Program Expansion
We use the HSIS data to conduct a formal cost-benefit anal-ysis of changes to Head Startrsquos offer rate under the assumptionthat competing programs are not rationed (we consider the casewith rationing in Section IX) Our analysis focuses on the costs
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1819
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
and benefits associated with one year of Head Start atten-dance11 This exercise requires estimates of each term in equa-tion (7) We estimate LATEh and Sc from the HSIS and calibratethe remaining parameters using estimates from the literatureCalibrated parameters are listed in Table IV Panel A To beconservative we deliberately bias our calibrations towardunderstating Head Startrsquos benefits and overstating its costs toarrive at a lower bound rate of return Further details of thecalibration exercise are provided in Online Appendix D
Table IV Panel B reports estimates of the marginal value ofpublic funds associated with an expansion of Head Start offers(MVPF) To account for sampling uncertainty in our estimates ofLATEh and Sc we report standard errors calculated via the deltamethod Because asymptotic delta method approximations can beinaccurate when the statistic of interest is highly nonlinear(Lafontaine and White 1986) we also report bootstrap p-valuesfrom one-tailed tests of the null hypothesis that the benefit-costratio is less than 112
The results show that accounting for the public savings as-sociated with enrollment in substitute preschools has a largeeffect on the estimated social value of Head Start We conductcost-benefit analyses under three assumptions rsquoc is either 050 or 75 of rsquoh Our preferred calibration uses rsquoc frac14 075rsquohreflecting that fact that roughly 75 of competing centers arepublicly funded (see Online Appendix D) Setting rsquoc frac14 0 yields aMVPF of 110 Setting rsquoc equal to 05rsquoh and 075rsquoh raises theMVPF to 150 and 184 respectively This indicates that the fiscalexternality generated by program substitution has an importanteffect on the social value of Head Start Bootstrap tests decisivelyreject values of MVPF less than 1 when rsquoc frac14 05rsquoh or 075rsquoh
11 Children in the three-year-old cohort who enroll for two years generate ad-ditional costs As shown in Table III a Head Start offer raises the probability ofenrollment in the second year by only 016 implying that first-year offers havemodest net effects on second-year costs Enrollment for two years may also generateadditional benefits but these cannot be estimated without strong assumptions onthe Head Start dose-response function We therefore consider only first-year ben-efits and costs
12 This test is computed by a nonparametric block bootstrap of the Studentizedt-statistic that resamples Head Start sites We have found in Monte Carlo exercisesthat delta method confidence intervals for MVPF tend to overcover whereas boot-strap-t tests have approximately correct size This is in accord with theoreticalresults from Hall (1992) that show bootstrap-t methods yield a higher-order refine-ment to p-values based on the standard delta method approximation
QUARTERLY JOURNAL OF ECONOMICS1820
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elA
P
ara
met
ervalu
esp
Eff
ect
ofa
1st
d
dev
in
crea
sein
test
scor
eson
earn
ings
01
eT
able
AI
V
e US
US
aver
age
pre
sen
td
isco
un
ted
valu
eof
life
-ti
me
earn
ings
at
age
34
$4380
00
Ch
etty
etal
(2011)
wit
h3
dis
cou
nt
rate
e pa
ren
te U
SA
ver
age
earn
ings
ofH
ead
Sta
rtp
are
nts
rela
tive
toU
S
aver
age
04
6H
ead
Sta
rtP
rogra
mF
act
s
IGE
Inte
rgen
erati
onal
inco
me
elast
icit
y04
0L
eean
dS
olon
(2009)
eA
ver
age
pre
sen
td
isco
un
ted
valu
eof
life
tim
eea
rnin
gs
for
Hea
dS
tart
ap
pli
can
ts$3433
92
[1
(1
e pa
ren
te U
S)I
GE
]eU
S
01
eE
ffec
tof
a1
std
d
ev
incr
ease
inte
stsc
ores
onea
rnin
gs
ofH
ead
Sta
rtap
pli
can
ts$343
39
LA
TE
hL
ocal
aver
age
trea
tmen
tef
fect
02
47
HS
IS
Marg
inal
tax
rate
for
Hea
dS
tart
pop
ula
tion
03
5C
BO
(2012)
Sc
Sh
are
ofH
ead
Sta
rtp
opu
lati
ond
raw
nfr
omot
her
pre
sch
ools
03
4H
SIS
uh
Marg
inal
cost
ofen
roll
men
tin
Hea
dS
tart
$80
00
Hea
dS
tart
Pro
gra
mF
act
su
cM
arg
inal
cost
ofen
roll
men
tin
oth
erp
resc
hoo
ls$0
Naiv
eass
um
pti
on
uc
=0
$40
00
Pes
sim
isti
cass
um
pti
on
uc
=05
uh
$60
00
Pre
ferr
edass
um
pti
on
uc
=07
5u
h
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1821
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIV
(CO
NT
INU
ED)
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
Pan
elB
M
arg
inal
valu
eof
pu
bli
cfu
nd
sN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
lati
onn
etof
taxes
$55
13
(1)
pL
AT
Eh
MF
CM
arg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uh
ucS
cp
LA
TE
h
naiv
eass
um
pti
on$36
71
Pes
sim
isti
cass
um
pti
on$29
91
Pre
ferr
edass
um
pti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0(0
22)
NM
BM
FC
(std
er
r)
naiv
eass
um
pti
onp
-valu
e=
1B
reak
even
p= efrac14
00
9eth00
1THORN
15
0(0
34)
Pes
sim
isti
cass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
8eth00
1THORN
18
4(0
47)
Pre
ferr
edass
um
pti
onp
-valu
e=
00
Bre
ak
even
p=efrac14
00
7eth00
1THORN
Not
es
Th
ista
ble
rep
orts
resu
lts
ofco
st-b
enefi
tca
lcu
lati
ons
for
Hea
dS
tart
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
Sta
nd
ard
erro
rsfo
rM
VP
Fra
tios
are
calc
ula
ted
usi
ng
the
del
tam
eth
od
P-v
alu
esare
from
one-
tail
edte
sts
ofth
en
ull
hyp
oth
eses
that
the
MV
PF
isle
ssth
an
1
Th
ese
test
sare
per
form
edvia
non
para
met
ric
blo
ckboo
tstr
ap
ofth
et-
stati
stic
cl
ust
ered
at
the
Hea
dS
tart
cen
ter
level
B
reak
even
sgiv
ep
erce
nta
ge
effe
cts
ofa
stan
dard
dev
iati
onof
test
scor
eson
earn
ings
that
set
MV
PF
equ
al
to1
QUARTERLY JOURNAL OF ECONOMICS1822
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Notably our preferred estimate of 184 is well above the esti-mated rates of return of comparable expenditure programs sum-marized in Hendren (2016) and comparable to the marginalvalue of public funds associated with increases in the top mar-ginal tax rate (between 133 and 20)
To assess the sensitivity of our results to alternative assump-tions regarding the relationship between test score effects andearnings Table IV also reports breakeven relationships betweentest scores and earnings that set MVPF equal to 1 for each valueof rsquoc When rsquoc frac14 0 the breakeven earnings effect is 9 per testscore standard deviation only slightly below our calibrated valueof 10 This indicates that when substitution is ignored HeadStart is close to breaking even and small changes in assumptionswill yield values of MVPF below 1 Increasing rsquoc to 05rsquoh or 075rsquoh
reduces the breakeven earnings effect to 8 or 7 respectivelyThe latter figure is well below comparable estimates in the recentliterature such as estimates from Chetty et alrsquos (2011) study ofthe Tennessee STAR class size experiment (13 see AppendixTable AIV) Therefore after accounting for fiscal externalitiesHead Startrsquos costs are estimated to exceed its benefits only if itstest score impacts translate into earnings gains at a lower ratethan similar interventions for which earnings data are available
VII Beyond LATE
Thus far we have evaluated the return to a marginal expan-sion of Head Start under the assumption that the mix of compli-ers can be held constant However it is likely that major reformsto Head Start would entail changes to program features such asaccessibility that could in turn change the mix of program com-pliers To evaluate such reforms it is necessary to predict howselection into Head Start is likely to change and how this affectsthe programrsquos rate of return
VIIA IV Estimates of SubLATEs
A first way in which selection into Head Start could change isif the mix of compliers drawn from home care and competingpreschools were altered while holding the composition of thosetwo groups constant To predict the effects of such a change onthe programrsquos rate of return we need to estimate the subLATEs inequation (3)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1823
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
One approach to identifying subLATEs is to conduct two-stage least squares (2SLS) estimation treating Head Start enroll-ment and enrollment in other preschools as separate endogenousvariables A common strategy for generating instruments in suchsettings is to interact an experimentally assigned program offerwith observed covariates or site indicators (eg Kling Liebmanand Katz 2007 Abdulkadiroglu Angrist and Pathak 2014) Suchapproaches can secure identification in a constant effects frame-work but as we demonstrate in Online Appendix E will typicallyfail to identify interpretable parameters if the subLATEs them-selves vary across the interacting groups (see Hull 2015 andKirkeboen Leuven and Mogstad 2016 for related results)
Table V reports 2SLS estimates of the separate effects ofHead Start and competing preschools using as instruments theHead Start offer indicator and its interaction with eight student-and site-level covariates likely to capture heterogeneity in com-pliance patterns13 These instruments strongly predict HeadStart enrollment but induce relatively weak independent varia-tion in enrollment in other preschools with a partial first-stage F-statistic of only 18 The 2SLS estimates indicate that Head Startand other centers yield large and roughly equivalent effects ontest scores of approximately 04 standard deviations This findingis roughly in line with the view that preschool effects are homo-geneous and that program substitution simply attenuates IV es-timates of the effect of Head Start relative to home careCautioning against this interpretation is the 2SLS overidentifica-tion test which strongly rejects the constant effects model indi-cating the presence of substantial effect heterogeneity acrosscovariate groups
A separate source of variation comes from experimental sitesthe HSIS was implemented at hundreds of individual Head Startcenters and previous studies have shown substantial variation intreatment effects across these centers (Bloom and Weiland 2015
13 Previous analyses of the HSIS have shown important effect heterogeneitywith respect to baseline scores and first language (Bitler Domina and Hoynes2014 Bloom and Weiland 2015) so we include these in the list of student-levelinteractions We also allow interactions with variables measuring whether achildrsquos center of random assignment offers transportation to Head Start whetherthe center of random assignment is above the median of the Head Start qualitymeasure the education level of the childrsquos mother whether the child is age fourwhether the child is black and an indicator for family income above the povertyline
QUARTERLY JOURNAL OF ECONOMICS1824
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Walters 2015) Using site interactions as instruments againyields much more independent variation in Head Start enroll-ment than in competing preschools14 However the estimatedimpact of Head Start is smaller and competing centers are esti-mated to yield no gains relative to home care While these site-based estimates are nominally more precise than those obtainedfrom the covariate interactions with 183 instruments the
TABLE V
TWO-STAGE LEAST SQUARES ESTIMATES WITH INTERACTION INSTRUMENTS
One endogenousvariable Two endogenous
variables
Head Start Head Start Other centersInstruments (1) (2) (3)
Offer 0247(1 instrument) (0031)
Offer covariates 0241 0384 0419(9 instruments) (0030) (0127) (0359)First-stage F 2762 177 18Overid p-value 007 006
Offer sites 0210 0213 0008(183 instruments) (0026) (0039) (0095)First-stage F 2151 900 27Overid p-value 002 002
Offer site groups 0229 0265 0110(6 instruments) (0029) (0056) (0146)First-stage F 10152 3391 326Overid p-value 077 050
Offer covariates and 0229 0302 0225offer site groups
(14 instruments)(0029) (0054) (0134)
First-stage F 3402 1212 133Overid p-value 012 010
Notes This table reports two-stage least squares estimates of the effects of Head Start and otherpreschool centers in spring 2003 The model in the first row instruments Head Start attendance with theHead Start offer Models in the second row instrument Head Start and other preschool attendance withinteractions of the offer and transportation above-median quality race Spanish language motherrsquos ed-ucation an indicator for income above the federal poverty line and baseline score The third row uses theHead Start offer interacted with 183 experimental site indicators as instruments The fourth row usesinteractions of the offer and indicators for groups of experimental sites obtained from a multinomial probitmodel with unobserved group fixed effects as described in Online Appendix G The fifth row uses bothcovariate and site group interactions All models control for main effects of the interacting variables andbaseline covariates First-stage F-statistics are Angrist and Pischke (2009) partial Frsquos Standard errors areclustered at the center level
14 To avoid extreme imbalance in site size we grouped the 356 sites in our datainto 183 sites with 10 or more observations See Online Appendix G for details
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1825
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
asymptotic standard errors may provide a poor guide to thedegree of uncertainty in the parameter estimates (BoundJaeger and Baker 1995) We explore this issue in OnlineAppendix Table AV which reports limited information maxi-mum likelihood and jackknife IV estimates of the same modelThese approaches yield much larger standard errors and verydifferent point estimates suggesting that weak instrumentbiases are at play here
To deal with these statistical problems we use a choice modelwith discrete unobserved heterogeneity (described in more detaillater) to aggregate Head Start sites together into six groups withsimilar substitution patterns Using the site group interactions asinstruments yields significant independent variation in bothHead Start and competing preschool enrollment and producesresults more in line with those obtained from the covariate inter-actions Pooling the site group and covariate interaction instru-ments together yields the most precise estimates which indicatethat both preschool types increase scores relative to home careand that Head Start is slightly more effective than competingpreschools However the overidentification test continues toreject the constant effects model suggesting that these estimatesare still likely to provide a misleading guide to the underlyingsubLATEs Another important limitation of the interacted 2SLSapproach is that it conditions on realized selection patterns andtherefore cannot be used to predict the effects of reforms thatchange the underlying composition of n- and c-compliers Wenow turn to developing an econometric selection model thatallows us to address both of these limitations
VIIB Selection Model
Our selection model parameterizes the preferences and po-tential outcomes introduced in the model of Section V to motivatea two-step control function estimator Like the interacted 2SLSapproach the proposed estimator exploits interactions of theHead Start offer with covariates and site groups to separatelyidentify the causal effects of care alternatives Unlike the inter-acted 2SLS approach the control function estimator allows theinteracting groups to have different subLATEs that vary para-metrically with the probability of enrolling in Head Start andcompeting preschools
QUARTERLY JOURNAL OF ECONOMICS1826
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Normalizing the value of preschool nonparticipation to zerowe assume households have utilities over program alternativesgiven by
Ui hZieth THORN frac14 h XiZieth THORN thorn vih
Ui ceth THORN frac14 c Xieth THORN thorn vic
Ui neth THORN frac14 0
eth13THORN
where Xi denotes the vector of baseline household and experi-mental site characteristics listed in Table I and Zi again de-notes the Head Start offer dummy The stochastic componentsof utility vih viceth THORN reflect unobserved differences in householddemand for Head Start and competing preschools relative tohome care In addition to pure preference heterogeneity theseterms may capture unobserved constraints such as whetherfamily members are available to help with child care We sup-pose these components obey a multinomial probit specification
vih viceth THORNjXiZi N 01 Xieth THORN
Xieth THORN 1
which allows for violations of the independence from irrelevantalternatives (IIA) condition that underlies multinomial logit se-lection models such as that of Dubin and McFadden (1984)
As in the Heckman (1979) selection framework we modelendogeneity in participation decisions by allowing linear depen-dence of mean potential outcomes on the unobservables that in-fluence choices Specifically for each program alternatived 2 hcnf g we assume
E Yi deth THORNjXiZi vih vicfrac12 frac14 d Xieth THORN thorn dhvih thorn dcviceth14THORN
The dh dc
coefficients in equation (14) describe the nature of
selection on unobservables This specification can accommodate avariety of selection schemes For example if dh frac14 h gt 0 thenconditional on observables selection into Head Start is governedby potential outcome levelsmdashthose most likely to participate inHead Start have higher test scores in all care environments Butif hh gt 0 and nh frac14 hh then households engage in Roy (1951)-style selection into Head Start based on test score gainsmdashthosemost likely to participate in Head Start receive larger test scorebenefits when they switch from home care to Head Start
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1827
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
By iterated expectations equation (14) implies the condi-tional expectation of realized outcomes can be written
E YijXiZiDi frac14 dfrac12 frac14 d Xieth THORN thorn dhh XiZideth THORN
thorn dcc XiZideth THORNeth15THORN
where h XiZiDieth THORN E vihjXiZiDifrac12 and c XiZiDieth THORN
E vicjXiZiDifrac12 are generalizations of the standard inverseMills ratio terms used in the two-step Heckman (1979) selectioncorrection (see Online Appendix F for details) These termsdepend on Xi and Zi only through the conditional probabilitiesof enrolling in Head Start and other preschools
VIIC Identification
To demonstrate identification of the selection coefficientsdh dc
it is useful to eliminate the main effect of the covariates
by differencing equation (15) across values of the program offer Zi
as follows
E YijXiZi frac14 1Di frac14 dfrac12 E YijXiZi frac14 0Di frac14 dfrac12
frac14 dh h Xi1deth THORN h Xi0deth THORNfrac12 thorn dc c Xi1deth THORN c Xi0deth THORNfrac12 eth16THORN
This difference measures how selected test score outcomes in aparticular care alternative respond to a Head Start offerResponses in selected outcomes are driven entirely by composi-tional changesmdashthat is from compliers switching betweenalternatives
With two values of the covariates Xi equation (16) can beevaluated twice yielding two equations in the two unknown se-lection coefficients Online Appendix F details the conditionsunder which this system can be solved and provides expressionsfor the selection coefficients in terms of population momentsAdditive separability of the potential outcomes in observablesand unobservables is essential for identification If the selectioncoefficients in equation (16) were allowed to depend on Xi therewould be two unknowns for every value of the covariates andidentification would fail Heuristically then our key assumptionis that selection on unobservables works lsquolsquothe same wayrsquorsquo for everyvalue of the covariates which allows us to exploit variation inselected outcome responses across subgroups to infer the param-eters governing the selection process
QUARTERLY JOURNAL OF ECONOMICS1828
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
To understand this restriction suppose (as turns out to bethe case) that college-educated mothers are more likely to enrolltheir children in competing preschools when denied access toHead Start Our model allows Head Start and other preschoolsto have different average treatment effects on the children ofmore and less educated mothers However it rules out the possi-bility that children with college-educated mothers sort into HeadStart on the basis of potential test score gains while children ofless educated mothers exhibit no sorting on these gains As inBrinch Mogstad and Wiswall (forthcoming) this restriction istestable when Xi takes more than two values because it implieswe should obtain similar estimates of the selection coefficientsbased on variation in different subsets of the covariates We pro-vide evidence along these lines by contrasting estimates that ex-ploit site variation with estimates based on household covariates
VIID Estimation
To make estimation tractable we approximate h XZeth THORN
and cethXTHORN with flexible linear functions The nonseparabilityof hethXZTHORN is captured by linear interactions between Z andthe eight covariates used in our earlier 2SLS analysis We alsoallow interactions with the 183 experimental site indicatorsbut to avoid incidental parameters problems we constrain thecoefficients on those dummies to belong to one of K discrete cat-egories Results from Bonhomme and Manresa (2015) and Saggio(2012) suggest that this lsquolsquogrouped fixed effectsrsquorsquo approach shouldyield good finite sample performance even when some sites haveas few as 10 observations As described in Online Appendix Gwe choose the number of site groups K using the Bayesian in-formation criterion (BIC) Finally all of the interacting variables(both site groups and covariates) are allowed to influence thecorrelation parameter Xeth THORN We assume that arctanhethXTHORN frac14 1
2 ln1thorn Xeth THORN1 Xeth THORN
is linear in these variables a standard transformation
that ensures the correlation is between 1 and 1 (Cox 2008)The model is fit in two steps First we estimate the param-
eters of the probit model via simulated maximum likelihood eval-uating choice probabilities with the Geweke-Hajivassiliou-Keane(GHK) simulator (Geweke 1989 Keane 1994 Hajivassiliou andMcFadden 1998) Models including site groups are estimatedwith an algorithm that alternates between maximizing the like-lihood and reassigning groups described in detail in Online
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1829
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Appendix G Second we use the parameters of the choice model toform control function estimates h XiZiDieth THORN c XiZiDieth THORN
which are then used in a second step regression of the form
Yifrac14n0thornX 0nxthornnhh XiZiDieth THORNthornncc XiZiDieth THORN
thorn1 Difrac14cf gc0n0eth THORNthornXi
0 cxnxeth THORNthorn chnh
h XiZiceth THORN
thorn ccnc
c XiZiceth THORN
thorn1 Difrac14hf gh0n0eth THORNthornXi
0 hxnxeth THORNthorn hhnh
h XiZiheth THORN
thorn hcnc
c XiZiheth THORN
24
35thornei
(17)
The covariate vector Xi is normed to have unconditional meanzero so the intercepts d0 can be interpreted as average potentialoutcomes Hence the differences h0 n0 and h0 c0 captureaverage treatment effects of Head Start and other preschools rel-ative to no preschool To avoid overfitting we restrict variablesother than the site types and eight key covariates to havecommon coefficients across care alternatives15 Inference on thesecond-step parameters is conducted via the nonparametricblock bootstrap clustered by experimental site
VIII Model Estimates
VIIIA Model Parameters
Table VI reports estimates of the full choice model obtainedfrom exploiting both covariates and site heterogeneity The BICselects a specification with six site groups for the full model (seeOnline Appendix Table AVI) with group shares that vary be-tween 12 and 21 of the sample These assignments make upthe site groups used in the earlier 2SLS analysis of Table V
Columns (1) and (2) of Table VI show the coefficients govern-ing the mean utility of enrollment in Head Start We easily rejectthe null hypothesis that the program offer interaction effects inthe Head Start utility equation are homogeneous Panel A col-umn (2) indicates that the effects of an offer are greater at high-quality centers and lower among nonpoor children that would
15 This restriction cannot be statistically rejected and has minimal effects onthe point estimates
QUARTERLY JOURNAL OF ECONOMICS1830
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VI
MULTINOMIAL PROBIT ESTIMATES
Head Start utility
(1) (2) (3) (4)Maineffect
Offerinteraction
Other centerutility Arctanh
Panel A CovariatesCenter provides 0022 0111 0054 0096transportation (0114) (0142) (0087) (0178)Above-median 0233 0425 0115 0007center quality (0091) (0102) (0082) (0153)Black 0095 0282 0206 0185
(0108) (0127) (0100) (0166)Spanish speaker 0049 0273 0213 0262
(0136) (0122) (0169) (0224)Motherrsquos education 0106 0021 0105 0219
(0056) (0060) (0064) (0110)Income above FPL 0216 0305 0173 0097
(0128) (0121) (0126) (0192)Baseline score 0080 0025 0292 0026
(0094) (0108) (0069) (0094)Age 4 0164 0277 0518 0010
(0142) (0166) (0104) (0170)p-values no heterogeneity 015 000 000 666
Panel B Experimental site groupsGroup 1 0644 2095 0424 0435(share = 0215) (0136) (0153) (0085) (0128)Group 2 4847 6760 0577 0496(share = 0183) (0076) (0158) (0045) (0172)Group 3 2148 2912 0768 0530(share = 0183) (0312) (0340) (0081) (0159)Group 4 0488 0541 0139 0483(share = 0151) (0130) (0150) (0226) (0322)Group 5 1243 2849 1643 0772(share = 0145) (0108) (0171) (0164) (0359)Group 6 0072 1191 0110 2988(share = 0124) (0127) (0183) (0106) (0925)p-values no heterogeneity 000 000 000 000
Notes This table reports simulated maximum likelihood estimates of a multinomial probit model ofpreschool choice The model includes fixed effects for six unobserved groups of experimental sites esti-mated as described in Online Appendix G The Head Start and other center utilities also include the maineffects of sex test language teen mother motherrsquos marital status presence of both parents family sizespecial education family income categories and second- and third-order terms in baseline test scores Thelikelihood is evaluated using the GHK simulator and likelihood contributions are weighted by the recip-rocal of the probability of experimental assignments P-values for site heterogeneity are from tests that allgroup-specific constants are equal P-values for covariate heterogeneity are from tests that all covariatecoefficients in a column are 0 Standard errors are clustered at the Head Start center level
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1831
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
typically be ineligible for Head Start enrollment16 Panel B col-umn (2) reveals the presence of significant heterogeneity acrosssite groups in the response to a program offer which likely re-flects unobserved market features such as the presence or ab-sence of state provided preschool
Column (4) of Table VI reports the parameters governing thecorrelation in unobserved tastes for Head Start and competingprograms The correlation is positive for four of six site groupsindicating that most households view preschool alternatives asmore similar to each other than to home care This establishesthat the IIA condition underlying logit-based choice models isempirically violated Although there is some evidence of hetero-geneity in the correlation based on motherrsquos education we cannotreject the joint null hypothesis that the correlation is constantacross covariate groups However we easily reject that the cor-relation is constant across site groups
The many sources of heterogeneity captured by the choicemodel yield substantial variation in predicted enrollment sharesfor Head Start and competing preschools Online Appendix FigureAI shows that these predictions match variation in choice proba-bilities across subgroups Moreover diagnostics indicate this vari-ation is adequate to secure separate identification of the secondstage control function coefficients From equation (16) the modelis underidentified if for any alternative d the control functiondifference h Xi1deth THORN h Xi0deth THORN is linearly dependent on the cor-responding difference c Xi1deth THORN c Xi0deth THORN Online AppendixFigure AII shows that the deviations from linear dependence arevisually apparent and strongly statistically significant
Table VII reports second-step estimates of the parameters inequation (17) Column (1) omits all controls and simply reportsdifferences in mean test scores across care alternatives (the omit-ted category is home care) Head Start students score 02 stan-dard deviations higher than students in home care and thecorresponding difference for students in competing preschools is026 standard deviations Column (2) adds controls for baseline
16 The quality variable aggregates information on center characteristics (tea-cher and center director education and qualifications class size) and practices (va-riety of literacy and math activities home visiting health and nutrition) measuredin interviews with center directors teachers and parents of children enrolled in thepreschool center
QUARTERLY JOURNAL OF ECONOMICS1832
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TABLE VII
SELECTION-CORRECTED ESTIMATES OF PRESCHOOL EFFECTS
Least squares Control function
(1) (2) (3) (4) (5)No controls Covariates Covariates Site groups Full model
Head Start 0202 0218 0483 0380 0470(0037) (0022) (0117) (0121) (0101)
Other preschools 0262 0151 0183 0065 0109(0052) (0035) (0269) (0991) (0253)
lh 0015 0004 0019(0053) (0063) (0053)
Head Start lh 0167 0137 0158(0080) (0126) (0091)
Otherpreschools lh
0030 0047 0000(0109) (0366) (0115)
lc 0333 0174 0293(0203) (0187) (0115)
Head Start lc 0224 0065 0131(0306) (0453) (0172)
Otherpreschools lc
0488 0440 0486(0248) (0926) (0197)
p-valuesNo selection 016 510 046No selection
on gains133 560 084
Additiveseparability
261 452 349
Notes This table reports selection-corrected estimates of the effects of Head Start and other preschoolcenters in spring 2003 Each column shows coefficients from regressions of test scores on an intercept aHead Start indicator an other preschool indicator and controls Column (1) shows estimates with nocontrols Column (2) adds controls for sex race home language test language motherrsquos education teenmother motherrsquos marital status presence of both parents family size special education income catego-ries experimental site characteristics (transportation above-median quality and urban status) and athird-order polynomial in baseline test score This column interacts the preschool variables with trans-portation above-median quality race Spanish language motherrsquos education an indicator for incomeabove the federal poverty line and the main effect of baseline score Covariates are demeaned in theestimation sample so that main effects can be interpreted as estimates of average treatment effectsColumn (3) adds control function terms constructed from a multinomial probit model using the covariatesfrom column (2) and the Head Start offer The interacting variables from column (2) are allowed tointeract with the Head Start offer and enter the preschool taste correlation equation in column (3)Column (4) omits observed covariates and includes indicators for experimental site groups constructedusing the algorithm described in Online Appendix G The multinomial probit model is saturated in thesesite group indicators and the second-step regression interacts site groups with preschool alternativesColumn (5) combines the variables used in columns (3) and (4) Standard errors are bootstrapped andclustered at the center level The bottom row shows p-values from a score test of the hypothesis thatinteractions between the control functions and covariates are 0 in each preschool alternative (see OnlineAppendix F for details)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1833
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
characteristics Because the controls include a third-order poly-nomial in baseline test scores this column can be thought of asreporting lsquolsquovalue-addedrsquorsquo estimates of the sort that have receivedrenewed attention in the education literature (Kane Rockoff andStaiger 2008 Rothstein 2010 Chetty Friedman and Rockoff2014a) Surprisingly adding these controls does little to the esti-mated effect of Head Start relative to home care but improvesprecision By contrast the estimated impact of competing pre-schools relative to home care falls significantly once controls areadded
Columns (3)ndash(5) add control functions adjusting for selectionon unobservables based on choice models with covariates sitegroups or both Unlike the specifications in previous columnsthese control function terms exploit experimental variation inoffer assignment Adjusting for selection on unobservables dra-matically raises the estimated average impact of Head Start rel-ative to home care However the estimates are fairly impreciseImprecision in estimates of average treatment effects is to be ex-pected given that these quantities are only identified via para-metric restrictions that allow us to infer the counterfactualoutcomes of always takers and never takers Below we consideraverage treatment effects on compliers which are estimatedmore precisely
Although some of the control function coefficient estimatesare also imprecise we reject the hypotheses of no selection onlevels (kd frac14 0 8ethkdTHORNTHORN and no selection on gains (dk frac14 jk ford 6frac14 j k 2 fhcg) in our most precise specification The selectioncoefficient estimates exhibit some interesting patterns One reg-ularity is that estimates of hh nh are negative in all specifica-tions (though insignificant in the model using site groups only) Inother words children who are more likely to attend Head Startreceive smaller achievement benefits when shifted from homecare to Head Start This lsquolsquoreverse Royrsquorsquo pattern of negative selec-tion on test score gains suggests large benefits for children withunobservables making them less likely to attend the program17
Other preschool programs by contrast seem to exhibit positiveselection on gains the estimated difference cc nc is always
17 Walters (2014) finds a related pattern of negative selection in the context ofcharter schools though in his setting the fall-back potential outcome (as opposed tothe charter school outcome) appears to respond positively to unobserved character-istics driving program participation
QUARTERLY JOURNAL OF ECONOMICS1834
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
positive and is significant in the full model A possible interpre-tation of these patterns is that Head Start is viewed by parents asa preschool of last resort leading to enrollment by the familiesmost desperate to get help with child care Such householdscannot be selective about whether the local Head Start center isa good match for their child which results in lower test scoregains By contrast households considering enrollment in substi-tute preschools may have greater resources that afford them theluxury of being more selective about whether such programs are agood match for their child
Estimates of the control function coefficients are very similarin columns (3) and (4) though the estimates are less precise whenonly site group interactions are used This indicates that the im-plied nature of selection is the same regardless of whether iden-tification is based on site or covariate interactions lendingcredibility to our assumption that selection works the same wayacross subgroups Also supporting this assumption are the re-sults of score tests for the additive separability of control func-tions and covariates reported in the bottom row of Table VIIThese tests are conducted by regressing residuals from the two-step models on interactions of the control functions with covari-ates and site groups along with the main effects from equation(15) In all specifications we fail to reject additive separability atconventional levels (see Online Appendix F for some additionalgoodness-of-fit tests) Although these tests do not have the powerto detect all forms of nonseparability the correspondence be-tween estimates based on covariate and site variation suggeststhat our key identifying assumption is reasonable
VIIIB Treatment Effects
Table VIII reports average treatment effects on compliers foreach of our selection-corrected models The first row uses themodel parameters to compute the pooled LATEh which is non-parametrically identified by the experiment The model estimatesline up closely with the nonparametric estimate obtained via IVOnline Appendix Figure AIII shows that this close correspon-dence between model and nonparametric LATEh holds evenacross covariate groups with very different average treatmenteffects The remaining rows of Table VIII report estimates of av-erage effects for compliers relative to specific care alternatives
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1835
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
(ie subLATEs)18 Estimates of the subLATE for n-compliersLATEnh are stable across specifications and indicate that theimpact of moving from home care to Head Start is largemdashon theorder of 037 standard deviations By contrast estimates ofLATEch though more variable across specifications never differsignificantly from zero
Our estimates of LATEnh are somewhat smaller than theaverage treatment effects of Head Start relative to home caredisplayed in Table VII This is a consequence of the reverse Roypattern captured by the control function coefficients familieswilling to switch from home care to Head Start in response toan offer have stronger than average tastes for Head Start imply-ing smaller than average gains We can reject that predicted
TABLE VIII
TREATMENT EFFECTS FOR SUBPOPULATIONS
Control function
(1) (2) (3) (4)Parameter IV Covariates Sites Full model
LATEh 0247 0261 0190 0214(0031) (0032) (0076) (0042)
LATEnh 0386 0341 0370(0143) (0219) (0088)
LATEch 0023 0122 0093(0251) (0469) (0154)
Lowest predictedquintile
LATEh 0095 0114 0027(0061) (0112) (0067)
LATEh with fixed Sc 0125 0125 0130(0060) (0434) (0119)
Highest predictedquintile
LATEh 0402 0249 0472(0042) (0173) (0079)
LATEh with fixed Sc 0364 0289 0350(0056) (1049) (0126)
Notes This table reports estimates of treatment effects for subpopulations Column (1) reports an IVestimate of the effect of Head Start Columns (2)ndash(4) show estimates of treatment effects computed fromthe control function models displayed in Table VII The bottom rows show effects in the lowest and highestquintiles of model-predicted LATE Rows with fixed c-complier shares weight subLATEs using the full-sample estimate of this share (034) Standard errors are bootstrapped and clustered at the center level
18 We compute the subLATEs by integrating over the relevant regions of Xivih and vic as described in Online Appendix F
QUARTERLY JOURNAL OF ECONOMICS1836
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
effects of moving from home care to Head Start are equal for n-compliers and n-never takers implying that this pattern is sta-tistically significant (p = 038) Likewise LATEhc is slightly neg-ative whereas the average treatment effect of Head Start relativeto other preschools is positive (047 011) In other wordsswitching from c to h reduces test scores for c-compliers butwould improve the score of an average student This reflects acombination of above average tastes for competing preschoolsamong c-compliers and positive selection on gains into other pre-schools Note that the control function coefficients in Table VIIcapture selection conditional on covariates and sites while thetreatment effects in Table VIII average over the distribution ofobservables for each subgroup The subLATE estimates showthat the selection patterns discussed here still hold when varia-tion in effects across covariate and site groups is taken intoaccount
Another interesting point of comparison is to the 2SLS esti-mates of Table V The 2SLS approach found a somewhat smallerLATEnh than our two-step estimator It also found that HeadStart preschools were slightly more effective at raising testscores than were competing programs (LATEch gt 0) whereasour full control function estimates suggest the oppositeImportantly the control function estimates corroborate thefailed overidentification tests of Table V by detecting substantialheterogeneity in the underlying subLATEs This can be seen inthe last four rows of Table VIII which report estimates for the topand bottom quintiles of the model-predicted distribution ofLATEh (see Online Appendix F for details) Fixing each grouprsquosSc at the population average brings estimates for the top andbottom quintiles closer together but a large gap remains due tosubLATE heterogeneity
Finally it is worth comparing our findings with those ofFeller et al (forthcoming) who use the principal stratificationframework of Frangakis and Rubin (2002) to estimate effects onn- and c-compliers in the HSIS They also find large effects forcompliers drawn from home and negligible effects for compliersdrawn from other preschools though their point estimate ofLATEnh is somewhat smaller than ours (021 versus 037) Thisdifference reflects a combination of different test score outcomes(Feller et al look only at PPVT scores) and different modelingassumptions Since neither estimation approach nests theother it is reassuring that we find qualitatively similar results
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1837
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
TA
BL
EIX
BE
NE
FIT
SA
ND
CO
ST
SO
FH
EA
DS
TA
RT
WH
EN
CO
MP
ET
ING
PR
ES
CH
OO
LS
AR
ER
AT
ION
ED
(1)
(2)
(3)
(4)
Para
met
erD
escr
ipti
onV
alu
eS
ourc
e
LA
TE
hH
ead
Sta
rtlo
cal
aver
age
trea
tmen
tef
fect
02
47
HS
ISL
AT
En
cE
ffec
tof
oth
erce
nte
rsfo
rm
arg
inal
chil
dre
n0
Naiv
eass
um
pti
on
no
effe
ctof
com
pet
ing
pre
sch
ools
03
70
Hom
ogen
eity
ass
um
pti
on
LA
TE
nc
equ
als
LA
TE
nh
02
94
Mod
el-b
ase
dp
red
icti
onN
MB
Marg
inal
ben
efit
toH
ead
Sta
rtp
opu
la-
tion
net
ofta
xes
$55
13
(1)
p(L
AT
Eh
+L
AT
En
cS
c)
naiv
eass
um
pti
on$83
21
Hom
ogen
eity
ass
um
pti
on$77
44
Mod
el-b
ase
dp
red
icti
onM
FC
Marg
inal
fisc
al
cost
ofH
ead
Sta
rten
roll
men
t$50
31
uhp
(LA
TE
h+
LA
TE
nc
Sc)
n
aiv
eass
um
pti
on$35
19
Hom
ogen
eity
ass
um
pti
on$38
30
Mod
el-b
ase
dp
red
icti
onM
VP
FM
arg
inal
valu
eof
pu
bli
cfu
nd
s11
0N
aiv
eass
um
pti
on23
6H
omog
enei
tyass
um
pti
on20
2M
odel
-base
dp
red
icti
on
Not
es
Th
ista
ble
rep
orts
resu
lts
ofa
rate
ofre
turn
calc
ula
tion
for
Hea
dS
tart
ass
um
ing
that
com
pet
ing
pre
sch
ools
are
rati
oned
an
dth
at
marg
inal
stu
den
tsof
fere
dse
ats
inth
ese
pro
gra
ms
as
are
sult
ofH
ead
Sta
rtex
pan
sion
wou
ldot
her
wis
ere
ceiv
eh
ome
care
P
ara
met
ervalu
esare
obta
ined
from
the
sou
rces
list
edin
colu
mn
(4)
QUARTERLY JOURNAL OF ECONOMICS1838
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
IX Policy Counterfactuals
We now use our model estimates to consider policy counter-factuals that are not nonparametrically identified by the HSISexperiment
IXA Rationed Substitutes
In the cost-benefit analysis of Section VI we assumed thatseats at competing preschools were not rationed Although thisassumption is reasonable in states with universal preschool man-dates other areas may have preschool programs that face rela-tively fixed budgets and offer any vacated seats to new childrenIn this case increases in Head Start enrollment will create op-portunities for new children to attend substitute preschoolsrather than generating cost savings in these programs Ourmodel-based estimates allow us to assess the sensitivity of ourcost-benefit results to the possibility of rationing in competingprograms
From equation (9) the marginal value of public funds underrationing depends on LATEncmdashthe average treatment effect ofcompeting preschools on lsquolsquon- to c-compliersrsquorsquo who would movefrom home care to a competing preschool program in responseto an offered seat We compute the MVPFrat under three alter-native assumptions regarding this parameter First we considerthe case where LATEnc = 0 Next we consider the case where theaverage test score effect of competing preschools for marginalstudents equals the corresponding effect for Head Start compliersdrawn from home care (ie LATEnc = LATEnh) Finally we useour model to construct an estimate for LATEnc Specifically wecompute average treatment effects of competing preschools rela-tive to home care for students who would be induced to movealong this margin by an increase in UiethcTHORN equal to the utilityvalue of the Head Start offer coefficient19 This calculation as-sumes that the utility value households place on an offered seat
19 Ideally we would compute LATEnc for students who do not receive offers tocompeting programs but would attend these programs if offered Because we do notobserve offers to substitute preschools it is not possible to distinguish betweennonoffered children and children who decline offers Our estimate of LATEnc there-fore captures a mix of effects for compliers who would respond to offers and childrenwho currently decline offers but would be induced to attend competing programs ifthese programs became more attractive
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1839
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
at a competing program is comparable to the value of a HeadStart offer
Table IX shows the results of this analysis Setting LATEnc =0 yields an MVPFrat of 110 This replicates the naive analysiswith rsquoc frac14 0 in the nonrationed analysis Both of these casesignore costs and benefits due to substitution from competing pro-grams Assuming that LATEnc = LATEnh produces a benefit-costratio of 236 Finally our preferred model estimates from SectionVIII predict that LATEnc = 0294 which produces a ratio of 202These results suggest that under plausible assumptions aboutthe effects of competing programs relative to home care account-ing for the benefits generated by vacated seats in these programsyields estimated social returns larger than those displayed inTable IV Panel B
IXB Structural Reforms
We next predict the social benefits of a reform that expandsHead Start by making it more attractive rather than by extendingoffers to additional households This reform is modeled as an im-provement in the structural program feature f as described inSection V Examples of such reforms might include increases intransportation services outreach efforts or spending on otherservices that make Head Start attractive to parents Increasesin f are assumed to draw additional households into Head Startbut to have no effect on potential outcomes which rules out peereffects generated by changes in student composition We use theestimates from our preferred model to compute marginal treat-ment effects and marginal values of public funds for such reformstreating changes in f as shifts in the mean Head Start utility hethXZTHORN
Figure I Panel A displays predicted effects of structural re-forms on test scores Since the program feature has no intrinsicscale the horizontal axis is scaled in terms of the Head Startattendance rate with a vertical line indicating the current rate(f = 0) The right axis measures ~Scmdashthe share of marginal stu-dents drawn from other preschools The left axis measures testscore effects The figure plots average treatment effects for sub-groups of marginal students drawn from home care and otherpreschools along with MTEh a weighted average of alterna-tive-specific effects
QUARTERLY JOURNAL OF ECONOMICS1840
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
A
B
FIGURE I
Effects of Structural Reforms
This figure plots predicted test score effects and marginal values of publicfunds for various values of the program feature f which shifts the utility of HeadStart attendance Horizontal axes shows the Head Start attendance rate at each fand a vertical line indicates the HSIS attendance rate (f = 0) Panel A showsmarginal treatment effects and competing preschool compliance shares The leftaxis measures test score effects MTEh is the average effect for marginal studentsand MTEnh and MTEch are effects for subgroups of marginal students drawn fromhome care and other preschools The right axis measures the share of marginalstudents drawn from other preschools The shaded region shows a 90 symmetricbootstrap confidence interval for MTEh Panel B shows predicted marginal valuesof public funds for structural reforms using the same parameter calibrations asTable IV P-values come from bootstrap tests of the hypothesis that the marginalvalue of public funds is less than or equal to 1 at f = 0
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1841
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Figure I shows that Head Startrsquos effects on marginal homecompliers increase modestly with enrollment and then level outin the neighborhood of the current program scale (f = 0) Thispattern is driven by reverse Roy selection for children drawnfrom home care increases in f attract children with weakertastes for Head Start leading to increases in effects for complierswho would otherwise stay home Predicted effects for childrendrawn from other preschools are slightly negative for all valuesof f At the current program scale the model predicts that theshare of marginal students drawn from other preschools is largerfor structural reforms than for an increase in the offer rate (044versus 035) This implies that marginal compliers are more likelyto be drawn from other preschools than are inframarginal com-pliers As a result the value of MTEh is comparable to the exper-imental LATEh despite very large effects on marginal childrendrawn from home care (roughly 05 standard deviations)
To investigate the consequences of this pattern for the socialreturn to Head Start Panel B plots MVPFf the marginal value ofpublic funds for structural reforms This figure relies on the sameparameter calibrations as Table IV Calculations of MVPFf mustaccount for the fact that changes in structural program featuresmay increase the direct costs of the program This effect is cap-tured in equation (12) by the term which gives the elasticity ofthe per child cost of Head Start with respect to the scale of theprogram Without specifying the program feature being manipu-lated there is no natural value for We start with the extremecase where = 0 which allows us to characterize costs and ben-efits associated with reforms that draw in children on the marginwithout changing the per capita cost of the program We thenconsider how the cost-benefit calculus changes when gt 0
As in our basic cost-benefit analysis the results in Figure IPanel B show that accounting for the public savings associatedwith program substitution has an important effect on the mar-ginal value of public funds The short-dashed curve plots MVPFf
setting rsquoc frac14 0 This calibration suggests a marginal value ofpublic funds slightly above 1 at the current program scale similarto the naive calibration in Table IV The solid curve accounts forpublic savings by setting rsquoc equal to our preferred value of 075rsquohThis generates an upward shift and steepens the MVPFf sched-ule indicating that both marginal and average social returns in-crease with program scale The implied marginal value of publicfunds at the current program scale (f = 0) is above 2 This is larger
QUARTERLY JOURNAL OF ECONOMICS1842
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
than the MVPF of 184 reported in Table IV which indicates thesocial returns to marginal expansions that shift the compositionof compliers are greater than those for expansions that simplyraise the offer rate
The final scenario in Panel B shows MVPFf when rsquoc frac14 075rsquoh
and frac14 0520 This scenario implies sharply rising marginal costsof Head Start provision an increase in f that doubles enrollmentraises per capita costs by 50 In this simulation the marginalvalue of public funds is roughly equal to 1 when f = 0 and fallsbelow 1 for higher values Hence if is at least 05 a $1 increasein Head Start spending generated by structural reform will resultin less than $1 transferred to Head Start applicants This exerciseillustrates the quantitative importance of determining provisioncosts when evaluating specific policy changes such as improve-ments to transportation services or marketing
Our analysis of structural reforms suggests increasing re-turns to the expansion of Head Start in the neighborhood of thecurrent program scalemdashexpansions will draw in households withweaker tastes for preschool with above average potential gainsThese findings imply that structural reforms targeting childrenwho are currently unlikely to attend Head Start and children thatare likely to be drawn from nonparticipation will generate largereffects than reforms that simply create more seats Our resultsalso echo other recent studies finding increasing returns to earlychildhood investments though the mechanism generating in-creasing returns in these studies is typically dynamic comple-mentarity in human capital investments rather than selectionand effect heterogeneity (see eg Cunha Heckman andSchennach 2010)
X Conclusion
Our analysis suggests that Head Start in its current incar-nation passes a strict cost-benefit test predicated only on proj-ected effects on adult earnings It is reasonable to expect that thisconclusion would be strengthened by incorporating the value ofany impacts on crime (as in Lochner and Moretti 2004 and
20 For this case marginal costs are obtained by solving the differential equa-tion rsquo
0
hethf THORN frac14 rsquoethf THORNqln P Difrac14heth THORN
qf
with the initial condition rsquoheth0THORN frac14 8000 This yields the
solution rsquohethf THORN frac14 8000exp ln P Di frac14 heth THORN ln P0eth THORNeth THORN where P0 is the initial HeadStart attendance rate
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1843
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman et al 2010) or other externalities such as civic engage-ment (Milligan Moretti and Oreopoulos 2004) or by incorporat-ing the value to parents of subsidized care (as in Aaberge et al2010) We find evidence that Head Start generates especiallylarge benefits for children who would not otherwise attend pre-school and for children with weak unobserved tastes for the pro-gram This suggests that the programrsquos rate of return can beboosted by reforms that target new populations though this ne-cessitates the existence of a cost-effective technology for attract-ing these children
The finding that returns are on average greater for nonpar-ticipants is potentially informative for the debate over calls foruniversal preschool which might reach high-return householdsHowever it is important to note that if competing state-level pre-school programs become ubiquitous the rationale for expansionsto federal preschool programs could be undermined To see thisconsider how the marginal value of expanding Head Startchanges as the compliance share Sc approaches 1 so thatnearly all denied Head Start applicants would otherwise enrollin competing programs If Head Start and competing programshave equivalent effects on test scores then equation (8) indicatesthat we should decide between federal and state-level provisionbased entirely on cost criteria Since state programs are oftencheaper (CEA 2014) and are expanding rapidly the case for fed-eral preschool may actually be weaker now than at the time of theHSIS
It is important to note some other limitations to our analysisFirst our cost-benefit calculations rely on literature estimates ofthe link between test score effects and earnings gains These cal-culations are necessarily speculative as the only way to be sure ofHead Startrsquos long-run effects is to directly measure long-run out-comes for HSIS participants Second we have ignored the possi-bility that substantial changes to program features or scale couldin equilibrium change the education production technology Forexample implementing recent proposals for universal preschoolcould generate a shortage of qualified teachers (Rothstein 2015)Finally we have ignored the possibility that administrative pro-gram costs might change with program scale choosing instead toequate average with marginal provision costs
Despite these caveats our analysis has shown that account-ing for program substitution in the HSIS experiment is crucial foran assessment of the Head Start programrsquos costs and benefits
QUARTERLY JOURNAL OF ECONOMICS1844
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Similar issues arise in the evaluation of job training programs(Heckman et al 2000) health insurance (Finkelstein et al2012) and housing subsidies (Kling Liebman and Katz 2007Jacob and Ludwig 2012) The tools developed here are potentiallyapplicable to a wide variety of evaluation settings where data onenrollment in competing programs are available
Supplementary Material
An Online Appendix for this article can be found at QJEonline (qjeoxfordjournalsorg)
University of California Berkeley and National
Bureau of Economic Research
University of California Berkeley and National
Bureau of Economic Research
References
Aaberge Rolf Manudeep Bhuller Audun Langoslashrgen and Magne Mogstad lsquolsquoTheDistributional Impact of Public Services When Needs Differrsquorsquo Journal ofPublic Economics 94 (2010) 549ndash562
Abdulkadiroglu Atila Joshua Angrist and Parag Pathak lsquolsquoThe Elite IllusionAchievement Effects at Boston and New York Exam SchoolsrsquorsquoEconometrica 82 (2014) 137ndash196
Angrist Joshua Guido Imbens and Donald Rubin lsquolsquoIdentification of CausalEffects Using Instrumental Variablesrsquorsquo Journal of the American StatisticalAssociation 91 (1996) 444ndash455
Angrist Joshua and Jorn-Steffen Pischke Mostly Harmless Econometrics(Princeton NJ Princeton University Press 2009)
Barnett W Steven lsquolsquoEffectiveness of Early Educational Interventionrsquorsquo Science333 (2011) 975ndash978
Bitler Marianne Thurston Domina and Hilary Hoynes lsquolsquoExperimental Evidenceon Distributional Effects of Head Startrsquorsquo NBER Working Paper 20434 2014
Bloom Howard and Christina Weiland lsquolsquoQuantifying Variation in Head StartEffects on Young Childrenrsquos Cognitive and Socio-Emotional Skills UsingData from the National Head Start Impact Studyrsquorsquo MDRC Report 2015
Bonhomme Stephane and Elena Manresa lsquolsquoGrouped Patterns of Heterogeneityin Panel Datarsquorsquo Econometrica 83 (2015) 1147ndash1184
Bound John David Jaeger and Regina Baker lsquolsquoProblems with InstrumentalVariables Estimation When the Correlation Between the Instruments andthe Endogenous Explanatory Variable Is Weakrsquorsquo Journal of the AmericanStatistical Association 90 (1995) 443ndash450
Brinch Christian Magne Mogstad and Matthew Wiswall lsquolsquoBeyond LATE with aDiscrete Instrumentrsquorsquo Journal of Political Economy forthcoming
Carneiro Pedro and Rita Ginja lsquolsquoLong-Term Impacts of Compensatory Preschoolon Health and Behavior Evidence from Head Startrsquorsquo American EconomicJournal Economic Policy 6 (2014) 135ndash173
Cascio Elizabeth and Diane Whitmore Schanzenbach lsquolsquoThe Impacts ofExpanding Access to High-Quality Preschool Educationrsquorsquo Brookings Paperson Economic Activity Fall (2013) 127ndash192
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1845
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Chetty Raj lsquolsquoSufficient Statistics for Welfare Analysis A Bridge betweenStructural and Reduced-Form Methodsrsquorsquo Annual Review of Economics 1(2009) 451ndash488
Chetty Raj John Friedman Nathaniel Hilger Emmanuel Saez DianeWhitmore Schanzenbach and Danny Yagan lsquolsquoHow Does YourKindergarten Classroom Affect Your Earnings Evidence from ProjectSTARrsquorsquo Quarterly Journal of Economics 126 (2011) 1593ndash1660
Chetty Raj John Friedman and Jonah Rockoff lsquolsquoMeasuring the Impacts ofTeachers I Measuring Bias in Teacher Value-added Estimatesrsquorsquo AmericanEconomic Review 104 (2014a) 2593ndash2632
mdashmdashmdash lsquolsquoMeasuring the Impacts of Teachers II Teacher Value-Added and StudentOutcomes in Adulthoodrsquorsquo American Economic Review 104 (2014b) 2633ndash2679
Congressional Budget Office (CBO) lsquolsquoEffective Marginal Tax Rates for Low- andModerate-Income Workersrsquorsquo 2012 available at httpswwwcbogovsitesde-faultfiles11-15-2012-MarginalTaxRatespdf
Council of Economic Advisers (CEA) lsquolsquoThe Economics of Early ChildhoodInvestmentsrsquorsquo 2014 available at httpswwwwhitehousegovsitesdefaultfilesdocsearly_childhood_report1pdf
Cox Nicholas lsquolsquoSpeaking Stata Correlation with Confidence or Fisherrsquos ZRevisitedrsquorsquo Stata Journal 8 (2008) 413ndash439
Cunha Flavio James Heckman and Susanne Schennach lsquolsquoEstimating theTechnology of Cognitive and Noncognitive Skill Formationrsquorsquo Econometrica78 (2010) 883ndash931
Currie Janet lsquolsquoEarly Childhood Education Programsrsquorsquo Journal of EconomicPerspectives 15 (2001) 213ndash238
Currie Janet and Duncan Thomas lsquolsquoDoes Head Start Make a DifferencersquorsquoAmerican Economic Review 85 (1995) 341ndash364
Deming David lsquolsquoEarly Childhood Intervention and Life-Cycle Skill DevelopmentEvidence from Head Startrsquorsquo American Economic Journal AppliedEconomics 1 (2009) 111ndash134
Dubin Jeffrey and Daniel McFadden lsquolsquoAn Econometric Analysis of ResidentialElectric Appliance Holdingsrsquorsquo Econometrica 52 (1984) 345ndash362
Engberg John Dennis Epple Jason Imbrogno Holger Sieg and Ron ZimmerlsquolsquoEvaluating Education Programs That Have Lotteried Admission andSelective Attritionrsquorsquo Journal of Labor Economics 32 (2014) 27ndash63
Federal Register lsquolsquoExecutive Order 13330 of February 24 2004rsquorsquo Federal Register69 (38) (2004) 9185ndash9187
Feller Avi Todd Grindal Luke Miratrix and Lindsay Page lsquolsquoCompared to WhatVariation in the Impact of Early Childhood Education by Alternative Care-Type Settingsrsquorsquo Annals of Applied Statistics forthcoming
Finkelstein Amy Sarah Taubman Bill Wright Mira BernsteinJonathan Gruber Joseph Newhouse Heidi Allen Katherine Baicker andthe Oregon Health Study Group lsquolsquoThe Oregon Health InsuranceExperiment Evidence from the First Yearrsquorsquo Quarterly Journal ofEconomics 127 (2012) 1057ndash1106
Frangakis Constantine and Donald Rubin lsquolsquoPrincipal Stratification in CausalInferencersquorsquo Biometrics 58 (2002) 21ndash29
Garces Eliana Duncan Thomas and Janet Currie lsquolsquoLonger-Term Effects of HeadStartrsquorsquo American Economic Review 92 (2002) 999ndash1012
Gelber Alexander and Adam Isen lsquolsquoChildrenrsquos Schooling and ParentsrsquoInvestment in Children Evidence from the Head Start Impact StudyrsquorsquoJournal of Public Economics 101 (2013) 25ndash38
Geweke John lsquolsquoBayesian Inference in Econometric Models Using Monte CarloIntegrationrsquorsquo Econometrica 57 (1989) 1317ndash1339
Gibbs Chloe Jens Ludwig and Douglas Miller lsquolsquoDoes Head Start Do Any LastingGoodrsquorsquo NBER Working Paper 17452 2011
Hajivassiliou Vassilis and Daniel McFadden lsquolsquoThe Method of Simulated Scoresfor the Estimation of LDV Modelsrsquorsquo Econometrica 66 (1998) 863ndash898
Hall Peter The Bootstrap and Edgeworth Expansion (New York Springer 1992)Heckman James lsquolsquoSample Selection Bias as a Specification Errorrsquorsquo Econometrica
47 (1979) 153ndash161
QUARTERLY JOURNAL OF ECONOMICS1846
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Heckman James Neil Hohmann Jeffrey Smith and Michael Khoo lsquolsquoSubstitutionand Dropout Bias in Social Experiments A Study of an Influential SocialExperimentrsquorsquo Quarterly Journal of Economics 115 (2000) 651ndash694
Heckman James Seong Hyeok Moon Rodrigo Pinto Peter Savelyev andAdam Yavitz lsquolsquoThe Rate of Return to the HighScope Perry PreschoolProgramrsquorsquo Journal of Public Economics 94 (2010) 114ndash128
Heckman James Rodrigo Pinto and Peter Savelyev lsquolsquoUnderstanding theMechanisms through Which an Influential Early Childhood ProgramBoosted Adult Outcomesrsquorsquo American Economic Review 103 (2013) 2052ndash2086
Heckman James Sergio Urzua and Edward Vytlacil lsquolsquoInstrumental Variables inModels with Multiple Outcomes The General Unordered Casersquorsquo AnnalesdrsquoEconomie et de Statistique 9192 (2008) 151ndash174
Heckman James and Edward Vytlacil lsquolsquoLocal Instrumental Variables andLatent Variable Models for Identifying and Bounding Treatment EffectsrsquorsquoProceedings of the National Academy of Sciences 96 (1999) 4730ndash4734
Hendren Nathaniel lsquolsquoThe Policy Elasticityrsquorsquo Tax Policy and the Economy 30(2016)
Hull Peter lsquolsquoIsoLATEing Identifying Counterfactual-Specific Treatment Effectsby Stratified Comparisonrsquorsquo working paper 2015
Imbens Guido and Joshua Angrist lsquolsquoIdentification and Estimation of LocalAverage Treatment Effectsrsquorsquo Econometrica 62 (1994) 467ndash475
Jacob Brian and Jens Ludwig lsquolsquoThe Effects of Housing Assistance on LaborSupply Evidence from a Voucher Lotteryrsquorsquo American Economic Review102 (2012) 272ndash304
Kane Thomas Jonah Rockoff and Douglas Staiger lsquolsquoWhat Does Certification TellUs about Teacher Effectiveness Evidence from New York Cityrsquorsquo Economicsof Education Review 27 (2008) 615ndash631
Keane Michael lsquolsquoA Computationally Practical Simulation Estimator for PanelDatarsquorsquo Econometrica 62 (1994) 95ndash116
Kirkeboen Lars Edwin Leuven and Magne Mogstad lsquolsquoField of Study Earningsand Self-Selectionrsquorsquo Quarterly Journal of Economics 131 (2016) doi101093qjeqjw019 (this issue)
Klein Joe lsquolsquoTime to Ax Public Programs that Donrsquot Yield Resultsrsquorsquo Time July 72011
Kling Jeffrey Jeffrey Liebman and Lawrence Katz lsquolsquoExperimental Analysis ofNeighborhood Effectsrsquorsquo Econometrica 75 (2007) 83ndash119
Lafontaine Francine and Kenneth White lsquolsquoObtaining Any Wald Statistic YouWantrsquorsquo Economics Letters 21 (1986) 35ndash40
Lee Chul-In and Gary Solon lsquolsquoTrends in Intergenerational Income MobilityrsquorsquoReview of Economics and Statistics 91 (2009) 766ndash772
Lochner Lance and Enrico Moretti lsquolsquoThe Effect of Education on Crime Evidencefrom Prison Inmates Arrests and Self-Reportsrsquorsquo American Economic Review94 (2004) 155ndash189
Long Cuiping lsquolsquoExperimental Evidence of the Effect of Head Start on MaternalHuman Capital Investmentrsquorsquo working paper 2015
Ludwig Jens and Douglas Miller lsquolsquoDoes Head Start Improve Childrenrsquos LifeChances Evidence from a Regression Discontinuity Designrsquorsquo QuarterlyJournal of Economics 122 (2007) 159ndash208
Ludwig Jens and Deborah Phillips lsquolsquoThe Benefits and Costs of Head StartrsquorsquoNBER Working Paper 12973 2007
Mayshar Joram lsquolsquoOn Measures of Excess Burden and Their ApplicationrsquorsquoJournal of Public Economics 43 (1990) 263ndash289
Milligan Kevin Enrico Moretti and Philip Oreopoulos lsquolsquoDoes Education ImproveCitizenship Evidence from the United States and the United KingdomrsquorsquoJournal of Public Economics 88 (2004) 1667ndash1695
Puma Michael Stephen Bell Ronna Cook and Camilla Heid Head Start ImpactStudy Final Report (Washington DC US Department of Health andServices Administration for Children and Families 2010)
Puma Michael Stephen Bell and Camilla Heid Third Grade Follow-Up to theHead Start Impact Study (Washington DC US Department of Health andHuman Services Administration for Children and Families 2012)
PUBLIC PROGRAMS WITH CLOSE SUBSTITUTES 1847
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from
Rothstein Jesse lsquolsquoTeacher Quality in Educational Production Tracking Decayand Student Achievementrsquorsquo Quarterly Journal of Economics 125 (2010)175ndash214
mdashmdashmdash lsquolsquoTeacher Quality Policy When Supply Mattersrsquorsquo American EconomicReview 105 (2015) 100ndash130
Roy A D lsquolsquoSome Thoughts on the Distribution of Earningsrsquorsquo Oxford EconomicsPapers 3 (1951) 135ndash146
Saggio Raffaele lsquolsquoDiscrete Unobserved Heterogeneity in Discrete Choice PanelData Modelsrsquorsquo masterrsquos thesis Center for Monetary and Financial Studies2012
Schumacher Rachel Mark Greenberg and Janellen Duffy The Impact of TANFFunding on State Child Care Subsidy Programs (Washington DC Center forLaw and Social Policy 2001)
Stossel John lsquolsquoHead Start Has Little Effect by Grade Schoolrsquorsquo Fox BusinessTelevision March 7 2014
US Department of Health and Human Services (DHHS) Administration forChildren and Families lsquolsquoChild Care and Development Fund Fact Sheetrsquorsquo2012 available at httpwwwacfhhsgovsitesdefaultfilesoccccdf_fact-sheetpdf
mdashmdashmdash lsquolsquoHead Start Program Facts Fiscal Year 2013rsquorsquo 2013 available at httpeclkcohsacfhhsgovhslcmrfactsheetsdocshs-program-fact-sheet-2011-finalpdf
mdashmdashmdash lsquolsquoHead Start Servicesrsquorsquo 2014 available at httpwwwacfhhsgovpro-gramsohsabouthead-start
Walters Christopher lsquolsquoThe Demand for Effective Charter Schoolsrsquorsquo NBERWorking Paper 20640 2014
mdashmdashmdash lsquolsquoInputs in the Production of Early Childhood Human Capital Evidencefrom Head Startrsquorsquo American Economic Journal Applied Economics 7 (2015)76ndash102
QUARTERLY JOURNAL OF ECONOMICS1848
at University of C
alifornia Berkeley on O
ctober 28 2016httpqjeoxfordjournalsorg
Dow
nloaded from