can$official$records$correct errors$in$turnoutself6reports?$
TRANSCRIPT
![Page 1: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/1.jpg)
Can Official Records Correct Errors in Turnout Self-‐reports?
Ma:hew K. Berent, Stanford University
Jon A. Krosnick, Stanford University
Arthur Lupia, University of Michigan
![Page 2: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/2.jpg)
AssumpJons
• IF: EffecJve sampling.
• PLUS: EffecJve survey design.
• THEN: Accurate data.
• TO: Understand the populaJon.
![Page 3: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/3.jpg)
Theory vs. PracJce
![Page 4: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/4.jpg)
Apparent Self-‐report Errors
EsJmates from sample data do not match known populaJon values.
![Page 5: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/5.jpg)
Reasons for Discrepancies
QuesJon: Did you vote? • Non-‐response.
• The quesJon is not answered. • Survey effects.
• ParJcipaJon affects behavior. • MisreporJng.
• Answers are factually incorrect.
![Page 6: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/6.jpg)
Types of MisreporJng
QuesJon: Did you vote? • MisinterpretaJon.
• Interpreted: Is poliJcs important to you? • Misremembering.
• Recalled: I wanted to vote so I must have. • IntenJonal lying.
• Reported: (I didn’t, but saying that would not look good so I’ll say) “Yes”.
![Page 7: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/7.jpg)
When Voters Lie
“It’s a given that people fib in surveys.”
The Wall Street Journal, August 2, 2008
![Page 8: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/8.jpg)
Apparent problem: Respondents lie.
Proposed soluJon: Use official records.
What to Do
![Page 9: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/9.jpg)
Using Official Records
• Obtain official records of turnout histories.
• Match each respondent to his or her official record.
• Determine the “correct” turnout for each respondent.
![Page 10: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/10.jpg)
Two Ways Thought to IdenJfy Lying
A respondent reported one thing
BUT
• An official record shows another
OR
• No official record is found
Fail-to-match
![Page 11: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/11.jpg)
A Case Study 2008-‐2009 ANES Panel Study
Total Sample
• N=4,206 respondents recruited via RDD sampling.
• US ciJzen • VoJng age • Landline households • Completed monthly online surveys
![Page 12: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/12.jpg)
Turnout
Which one of the following best describes what you did in this elecJon?
__ Definitely did not vote __ Definitely voted in person at a polling place on elec2on day __ Definitely voted in person at a polling place before elec2on day __ Definitely voted by mailing a ballot to elec2ons officials before elec2on day __ Definitely voted in some other way __ Not completely sure whether you voted or not
The QuesJon
![Page 13: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/13.jpg)
ANES Self-‐report Turnout Rates
![Page 14: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/14.jpg)
ANES Self-‐report Turnout Rates
![Page 15: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/15.jpg)
ANES Self-‐report Turnout Rates
![Page 16: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/16.jpg)
ValidaJng Turnout Self-‐reports
The ValidaJon Task • Match government records to survey respondents.
• Determine “correct” turnout for each respondent.
Two problems • 200,000,000+ people eligible to vote in the 2008.
• Some government records are difficult to obtain.
![Page 17: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/17.jpg)
Commercial Vendors
• Several are available for records matching.
• Maintain naJonal databases of government records.
• Use records from mulJple sources to increase data accuracy.
• Developed technology for records matching.
![Page 18: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/18.jpg)
Our Requirements
• Full disclosure about the vendor’s records. • When they were obtained.
• From whom were they obtained.
• How the records are cleaned.
• Full disclosure about the matching process.
• Details about their matching algorithm.
• Their criteria for determining a match.
![Page 19: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/19.jpg)
What We Learned
• Vendors use names, addresses, and dates of birth to match records.
• Some vendors are more transparent than others.
• Some vendors can match records at different levels of confidence.
• No vendor was willing to provide complete details about their matching algorithm.
![Page 20: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/20.jpg)
In-‐house Records Matching
• Obtain government registraJon and turnout records from a sample of states.
• Use a publicly available computer applicaJon to match respondents to government records.
• LinkPlus – Developed by the CDC
![Page 21: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/21.jpg)
2008-‐2009 ANES Panel Study Our Subsample (N=761)
Respondents with a residenJal address in: • California • Florida • New York • North Carolina • Ohio • Pennsylvania
Weighted to reflect within-‐state landline populaJons:
• Age • Sex • Race • EducaJon • Income • Marital status
![Page 22: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/22.jpg)
Matching Respondents to Records
Three methods of matching
![Page 23: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/23.jpg)
Matching Respondents to Records
Last name First name Address DOB
Strict Iden2cal Iden2cal Iden2cal Iden2cal
Three methods of matching
![Page 24: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/24.jpg)
Matching Respondents to Records
Last name First name Address DOB
Strict Iden2cal Iden2cal Iden2cal Iden2cal
Moderate Same or similar
Same or similar
Same or similar
Iden2cal
Three methods of matching
![Page 25: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/25.jpg)
Matching Respondents to Records
Last name First name Address DOB
Strict Iden2cal Iden2cal Iden2cal Iden2cal
Moderate Same or similar
Same or similar
Same or similar
Iden2cal
Least Same or similar
Same or similar
Same or similar
Same or similar
Same or similar
Same or similar
Iden2cal
Iden2cal Iden2cal Iden2cal
Three methods of matching
![Page 26: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/26.jpg)
Strict Moderate Least All target states
43% 59% 69%
Turnout Rates According to Government Records
![Page 27: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/27.jpg)
Strict Moderate Least Popula2on All target states
43% 59% 69% 62%
“Correct” Turnout Turnout Rates According to Government Records
![Page 28: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/28.jpg)
Strict Moderate Least Popula2on All target states
43% 59% 69% 62%
Using Government Records
If lying causes turnout over-‐repor2ng
• Moderate matching is the most valid
• Least matching is somewhat valid
• Strict matching is not valid
• Self-‐reports cannot be trusted
Turnout Rates According to Government Records
![Page 29: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/29.jpg)
Problem Solved?
Problem: Respondents Lie
SoluJon: Use government records
Proof: Sample turnout=Popula2on turnout
DerivaJve: Sample registra2on=Popula2on registra2on
![Page 30: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/30.jpg)
“Correct” registraJon
Popula2on All target states 83%
• PopulaJon registraJon rate based on staJsJcs published by states.
• RegistraJon rate among voJng age US ciJzens in landline households.
• Adjusted for deadwood.
RegistraJon Rates According to Government StaJsJcs
![Page 31: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/31.jpg)
Popula2on Strict All target states 83% 49%
• Strict matching underesJmates populaJon registraJon rate.
“Correct” registraJon RegistraJon Rates According to Government StaJsJcs
![Page 32: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/32.jpg)
Popula2on Strict Moderate All target states 83% 49% 66%
• Moderate matching underesJmates populaJon registraJon by 17%.
• Moderate matching appears invalid.
“Correct” registraJon RegistraJon Rates According to Government StaJsJcs
![Page 33: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/33.jpg)
Popula2on Strict Moderate Least All target states 83% 49% 66% 78%
• Even Least matching underesJmates populaJon registraJon.
“Correct” registraJon RegistraJon Rates According to Government StaJsJcs
![Page 34: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/34.jpg)
Compared to the PopulaJon
Strict Moderate Least Turnout Way Too Low Closest Too High RegistraJon Way Too Low Way Too Low Closest
![Page 35: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/35.jpg)
What is going on?
![Page 36: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/36.jpg)
CompeJng Biases
• Downward bias – Using government records incorrectly idenJfies some respondents as having not turned out.
• Upward bias – People who parJcipate in surveys are more likely to vote.
![Page 37: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/37.jpg)
Evidence of an upward bias?
![Page 38: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/38.jpg)
Turnout among the Registered
Among voJng-‐age US ciJzens in landline households WHO ARE REGISTERED:
Turnout=75%
Based on staJsJcs published by states, adjusted for deadwood
![Page 39: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/39.jpg)
Turnout among the Registered
Among voJng-‐age US ciJzens in landline households WHO ARE REGISTERED:
Turnout=75%
Strict Moderate Least Turnout 88%
Strict matching suggests higher turnout among respondents
![Page 40: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/40.jpg)
Turnout among the Registered
Among voJng-‐age US ciJzens in landline households WHO ARE REGISTERED:
Turnout=75%
Strict Moderate Least Turnout 88% 90%
Moderate matching suggests higher turnout among respondents
![Page 41: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/41.jpg)
Turnout among the Registered
Among voJng-‐age US ciJzens in landline households WHO ARE REGISTERED:
Turnout=75%
Strict Moderate Least Turnout 88% 90% 88%
Least matching suggests higher turnout among respondents
![Page 42: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/42.jpg)
Why the downward bias?
![Page 43: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/43.jpg)
Self-‐Reported Voters
Truthful Record shows no turnout
Fail-‐to-‐match
Strict 50% 3% 47% Moderate 68% 4% 27% Least 80% 5% 15%
Self-report confirmed by government records
![Page 44: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/44.jpg)
Self-‐Reported Voters
Truthful Record shows no turnout
Fail-‐to-‐match
Strict 50% 3% 47% Moderate 68% 4% 27% Least 80% 5% 15%
Self-report disconfirmed by government records
![Page 45: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/45.jpg)
Self-‐Reported Voters
Truthful Record shows no turnout
Fail-‐to-‐match
Strict 50% 3% 47% Moderate 68% 4% 27% Least 80% 5% 15%
Self-report cannot be confirmed by government records
![Page 46: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/46.jpg)
Problem: Respondents lie.
Problem: Sample turnout > PopulaJon turnout.
SoluJon: Use government records.
Using government records create an illusion of accuracy.
SoluJon: Use self-‐reports.
Turnout self-‐reports appear extremely accurate.
Sample Turnout ≠ PopulaJon Turnout
![Page 47: Can$Official$Records$Correct Errors$in$TurnoutSelf6reports?$](https://reader034.vdocument.in/reader034/viewer/2022042702/6265e3b4471bca34b336d177/html5/thumbnails/47.jpg)
Thank You!