remaining weeks
DESCRIPTION
Remaining Weeks. Next week: Diff-n-Diff Nov. 17: Power calculations. Nov. 24: summary, in class presentations. Dec. 1: Guests, more presentations. Motivation: Causality. AP Headline Today: Teen pregnancies tied to tastes for sexy TV shows. Real-World Complications. Attrition - PowerPoint PPT PresentationTRANSCRIPT
Remaining Weeks
• Next week: Diff-n-Diff
• Nov. 17: Power calculations.
• Nov. 24: summary, in class presentations.
• Dec. 1: Guests, more presentations.
Motivation: Causality.
AP Headline Today:
Teen pregnancies tied to tastes for sexy TV shows
Real-World Complications
Attrition
Data Quality
Cars Stuck in the Mud, Employees Robbed
What type of day are you having?
Practical ProblemsLanguage
Culture
Being around the same four Westerners 24/7 without going crazy.
Solutions:
Having had a real job?
Management skills
Actual Organizations
• CEGA (Our Sponsor)
http://cega.berkeley.edu• Poverty Action Lab (J-PAL)
http://www.povertyactionlab.org
• Innovations for Poverty Actionhttp://www.poverty-action.org
• Blum Center for Developing Economieshttp://blumcenter.berkeley.edu
CEGA-related Faculty
• Alain de Janvry
• Paul J. Gertler
• David I. Levine
• Edward Miguel
• Nancy Padian
• Elisabeth Sadoulethttp://cega.berkeley.edu/template.php?page=people
Larger NGO-types
• The World Bank
• Center for Global Development
• International Food Policy Research Institute
many, many more
Human Subjects
UC Berkeley Committee for the Protection of Human Subjects
http://cphs.berkeley.edu
In-country organization as well, for example:
Kenya Medical Research Institute
http://www.kemri.org
Attrition
Randomized trials often require that we get data from the subjects twice--once before the experiment and once after.
What if we can’t find them afterwards?
Worksheet
How might you expect people we couldn’t find to differ from those we could easily find?
What could cause people to go missing?
Attrition
Create Lower/Upper Bound for our estimates by assuming the worst about the people we couldn’t find.
(Ummm, I can’t remember this reference. Sorry.)
In our case, we’ll just say it’s important to find as many people as possible to get good data.
Attrition in KLPS
Kenyan Life Panel Survey
2003-2005 follow-up to Deworming (1998-2000)
7500 of the original 30,000 were randomly selected to be surveyed.
Attrition in KLPS
First, go their old school and ask around.
Second, try and go find their house.
Third, travel far and wide.
Attrition in KLPS
Using two-part regular and intensive tracking just like in “Moving to Opportunity.”
After finding as large a portion as you can, select random sub-sample of everyone remaining.
ERR=MRR+SRR*(1-MRR)
Attrition in KLPS
End Results:
84% successfully contacted
83% successfully surveyed
Attrition in KLPS
4 different types of being “found,” by treatment and gender
QuickTime™ and a decompressor
are needed to see this picture.
Where’d we find them?
--19% Outside Busia--14% Outside Neighboring Areas--25% Overall (Non-Snapshot)
QuickTime™ and a decompressor
are needed to see this picture.
So, We Got 84%, Are We Cool?
• Is treatment correlated with attrition?
So, We Got 84%, Are We Cool?
• Is treatment correlated with attrition?Probably Not. We found 83.9% to 85.0% in all treatment groups.
QuickTime™ and a decompressor
are needed to see this picture.
Was it worth it?
• We spent a lot of money to find the emigrants. QuickTime™ and a
decompressorare needed to see this picture.
Did we need to bother?
• Migrants are 1.7 cm shorter than non-migrants, and an additional year of treatment increased migrant height by .4 cm and only .1 cm for the full sample.
QuickTime™ and a decompressor
are needed to see this picture.
The Nuts & Bolts of Building the Dataset
• Written on hard-copy of survey.
• Sub-sample checked for mistakes.
• Data-entry place double enters.
• We check for correlation of two entries.
• We re-enter 5% sample and check against their work, accept if error rate below threshold.
• That’s the “raw” data
The Nuts & Bolts of Building the Dataset
• Depressed grad students spend whole summers in windowless Unix lab on the 6th floor of the 2nd ugliest building on campus writing cleaning files, which checks for blanks and skip-pattern violations.
• Send the list of flagged entries to location of hard copies
• Hard-copies checked against soft-copy. Soft-copy corrected, mistake flag lowered.
• Feel free to use the data.
Data Quality
Fine, we correctly recorded what the respondent said, but should we really trust what they said?
That is, if you were 16 and had a miscarriage a year ago, would you really want to tell an older man that’s a stranger about it?
GenderQuickTime™ and a decompressor
are needed to see this picture.
TribeQuickTime™ and a
decompressorare needed to see this picture.
Do Kids Know What They’re Talking About?
• Disregard the respondent/enumerator relationship. Do the kids really know what they’re talking about?
• Depends on the question.
What’s Reliable?
• We sample 5% to be resurveyed, successully resurveyed about 4%. 3 months later on average.
Baseline: If we ask “what tribe are you?” It stays the same 95% of the time.
Pretty Decent
QuickTime™ and a decompressor
are needed to see this picture.
Pretty Decent
QuickTime™ and a decompressor
are needed to see this picture.
I Can’t Throw This Very Far.
QuickTime™ and a decompressor
are needed to see this picture.
I Can’t Throw This Very Far.
QuickTime™ and a decompressor
are needed to see this picture.
Fraction Matching
• Sub-Tribe 95%
• Age in 1998 76%
• Grade in 2002 86%
• Ever left local area 91%
• Mom/Dad Education 51-53%
What Determines Remembering?
• Tables 22 and 23 show what characteristics are correlated with giving the same answer about Mom/Dad’s education in both survey and re-survey.
Conclusion
• Field work is great; go do some.• Try and find everyone.• Especially if you’re more/less likely to find
them thanks to your intervention.• Do your Field Officers effect the answers
given?• Does the respondent really know the right
answer in the first place?