introduction to bootstrap methods - university of...
TRANSCRIPT
Introduction to Bootstrap Methods
Miguel Sarzosa
Department of Economics
University of Maryland
Econ626: Empirical Microeconomics, 2012
1 Recreating the Universe
2 Bootstrap Estimates
3 The Jackknife
4 Applications
5 Now we go to Stata!
Why is Bootstrap Important?
Only in specific instances we are able to infer population parametersfrom a data set
I Data sets are often samples of the population
Most of the estimators we use for inference rely on asymptotic results(CLT). A setting that is impossible to have in the data unless we usean approximation like bootstrap.Bootstrap is used for two things
1 Computation of SE, CI (practical)2 Asymptotic behavior of estimators (theory)
How does it work?
Bootstrap views the sample you have in your data set as thepopulation of interest.We obtain estimates of characteristics of the distribution of a givenestimator µ by drawing B sub-samples of size N with replacement.Then we get B estimates of µ (i.e., µ
b
) from which we can obtainmoments like the mean and the variance
µ ⇠
B
�1
B
Âb=1
µb
,(B �1)�1
B
Âb=1
µ
b
�B
�1
B
Âb=1
µb
!!
Algorithm
1 Take subsample of size N
2 Calculate the desired statistic on the sample3 Repeat 1. and 2. B times, where B is a large number
How Many Repetitions?
In most of the cases, the more the better. However, bootstrapping can becomputationally intensive. Andrews and Buchinsky (2000) came up withthe rule that you should take B = 384w repetitions, where w with thestatistic inquired
SE: w = (2+ g4
)4. Where g4
depends on the excess kurtosis ) fattertails mean higher B
I For no excess kurtosis B = 192I For g
4
= 8, B = 960
Two sided CI: Depends on the critical valueI For a = 0.05, B = 348I For a = 0.01, B = 685
One sided CI: Depends on the critical valueI For a = 0.05, B = 634I For a = 0.01, B = 989
SE Estimation
One of the main uses of bootstrap is to calculate the correct SE. SEestimation through bootstrap is very useful for example in 2-stepestimations (e.g., IV) whose SE are di�cult to compute. The variance ofan estimate q calculated using bootstrap is given by
S
2
q = (B �1)�1
B
Âb=1
⇣q ⇤
b
� ¯q⌘
2
where¯q = B
�1
B
Âb=1
q ⇤b
(1)
Sq is consistent, therefore it can be used to obtain CI and hypothesistesting
Bias Estimation
From (1) it is easy to see that the bias of the estimate q is given by
Biasq =⇣¯q � q
⌘
This allows to correct for the bias in an estimate. The bias-correctedestimate is given by
qcorr
= q �Biasq = q �⇣¯q � q
⌘= 2q � ¯q
The JackknifeUses N subsamples of size N �1 (drops one observation at a time). Thenthe jackknife estimate is given by
¯q = N
�1 Âi
q�i
Then the bias isBiasq = (N �1)
⇣¯q � q
⌘
and the corrected estimate
qcorr
= q �Biasq = N q � (N �1) ¯q
The jackknife SE is given by
Sq =
"N �1
N
B
Âb=1
⇣q�i
� ¯q⌘
2
#1/2
Heteroskedastic Errors
In the presence of heteroskedasticity, we use HEW SE, but they performvery poorly in small samples. Bootstrap can be a better choice
Paired Bootstrap: Obtain samples of (yi
,xi
) and estimate (i.e.,regress). Assuming that each draw is i.i.d we are able to do validinference because we are still allowing for Var [u
i
|xi
] to vary with x
i
.In Stata vce(bootstrap)
Wild Bootstrap: Obtain samples of (y⇤i
,xi
) where
y
⇤i
= x
i
b + u
⇤i
and
u
⇤i
=
8<
:
1�p
5
2
u
i
with probability 1+p
5
2
p5⇣
1� 1�p
5
2
⌘u
i
with probability 1� 1+p
5
2
p5
Panel Data and Clustered Data
Note that in the Paired Bootstrap we assumed the (yi
,xi
) draws werei.i.d. In the case we are not able to claim that because theobservations are not independently distributed (i.e., panel or clustereddata) we use panel bootstrap.Suppose a panel has two dimensions i and t. In the panel bootstrap,we resample over i and not over t. That is when we bootstrap wechoose the is that will appear in the subsample and obtain all the t
observations of those is chosenA key assumption is that the data are independent over i
The same procedure is done when the data is clustered. We resampleover the clusters, and then get all observations belonging to thatcluster. We need a huge number of clusters.
I In Stata: vce(bootstrap, cluster(varlist ))
Implementation Issues
Remember that the assumption of independence across observationsor clusters of observations is crucialIn some cases, the second moment might not exist, evenasymptotically, so the bootstrap results will be missleadingThe basic bootstrap assumes the estimator is a smooth estimator,p
N -consistent and asymptotically normal
Implementation in Stata
Stata users can perfirm bootstrapped estimations using two ways:1 There are some commands that incorporate the bootstrap option by
typing vce(bootstrap) or vce(bootstrap, cluster(varlist ))
2 Stata users can also incorporate bootstrap estimations in a largenumber of commands including:
1 Those without the vce(bootstrap) option,2 Non-estimation commands, (e.g., summarize)3 User-written commands
using the bootstrap command
The bootstrap CommandThe syntax of bootstrap is di�erent from the main style of Statacommands. It requires to first specify what is the estimate that is going tobe bootstrapped, then the bootstrap options and then the command thatis going to be boostrapped.
bootstrap exp_list [, options eform_option ] : command
whereexp_list specifies the estimates that will be bootstrapped (e.g.,_b, _b[x1] or _se)among the most importatnt options we have
Ireps(#)
Iseed(#)
Icluster(varlist )
Istrata(varlist )
Isize(#)
Example
bootstrap _b, reps(100) seed(10101) cluster(clusvar): proIVpro
if desocupa==0 & year==2000
Now we go to Stata!