what to do about biases in survey researchanalysis, 15, 1 (winter, 2007): 46–66. gary king;...
TRANSCRIPT
![Page 1: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/1.jpg)
What to do about Biases in Survey Research
Gary King
http://GKing.Harvard.Edu
September 6, 2007
Gary King () What to do about Biases in Survey Researchhttp://GKing.Harvard.Edu September 6, 2007 1
/ 45
![Page 2: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/2.jpg)
Readings
Gary King and Jonathan Wand. “Comparing Incomparable SurveyResponses: Evaluating and Selecting Anchoring Vignettes,” PoliticalAnalysis, 15, 1 (Winter, 2007): 46–66.
Gary King; Christopher J.L. Murray; Joshua A. Salomon; and AjayTandon. “Enhancing the Validity and Cross-cultural Comparability ofMeasurement in Survey Research,” American Political ScienceReview, Vol. 98, No. 1 (February, 2004): 191–207.
Papers, FAQ, examples, software, conferences, videos:http://GKing.Harvard.edu/vign
Gary King () Anchoring Vignettes 2 / 45
![Page 3: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/3.jpg)
Readings
Gary King and Jonathan Wand. “Comparing Incomparable SurveyResponses: Evaluating and Selecting Anchoring Vignettes,” PoliticalAnalysis, 15, 1 (Winter, 2007): 46–66.
Gary King; Christopher J.L. Murray; Joshua A. Salomon; and AjayTandon. “Enhancing the Validity and Cross-cultural Comparability ofMeasurement in Survey Research,” American Political ScienceReview, Vol. 98, No. 1 (February, 2004): 191–207.
Papers, FAQ, examples, software, conferences, videos:http://GKing.Harvard.edu/vign
Gary King () Anchoring Vignettes 2 / 45
![Page 4: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/4.jpg)
Readings
Gary King and Jonathan Wand. “Comparing Incomparable SurveyResponses: Evaluating and Selecting Anchoring Vignettes,” PoliticalAnalysis, 15, 1 (Winter, 2007): 46–66.
Gary King; Christopher J.L. Murray; Joshua A. Salomon; and AjayTandon. “Enhancing the Validity and Cross-cultural Comparability ofMeasurement in Survey Research,” American Political ScienceReview, Vol. 98, No. 1 (February, 2004): 191–207.
Papers, FAQ, examples, software, conferences, videos:http://GKing.Harvard.edu/vign
Gary King () Anchoring Vignettes 2 / 45
![Page 5: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/5.jpg)
The Importance of Survey Research
In political science: 1/2 of all quantitative articles
Other social sciences and related professional areas: Widely used
A large fraction of our information base over the last half century
A multi-billion dollar industry
Of widespread public interest
Gary King () Anchoring Vignettes 3 / 45
![Page 6: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/6.jpg)
The Importance of Survey Research
In political science: 1/2 of all quantitative articles
Other social sciences and related professional areas: Widely used
A large fraction of our information base over the last half century
A multi-billion dollar industry
Of widespread public interest
Gary King () Anchoring Vignettes 3 / 45
![Page 7: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/7.jpg)
The Importance of Survey Research
In political science: 1/2 of all quantitative articles
Other social sciences and related professional areas: Widely used
A large fraction of our information base over the last half century
A multi-billion dollar industry
Of widespread public interest
Gary King () Anchoring Vignettes 3 / 45
![Page 8: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/8.jpg)
The Importance of Survey Research
In political science: 1/2 of all quantitative articles
Other social sciences and related professional areas: Widely used
A large fraction of our information base over the last half century
A multi-billion dollar industry
Of widespread public interest
Gary King () Anchoring Vignettes 3 / 45
![Page 9: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/9.jpg)
The Importance of Survey Research
In political science: 1/2 of all quantitative articles
Other social sciences and related professional areas: Widely used
A large fraction of our information base over the last half century
A multi-billion dollar industry
Of widespread public interest
Gary King () Anchoring Vignettes 3 / 45
![Page 10: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/10.jpg)
The Importance of Survey Research
In political science: 1/2 of all quantitative articles
Other social sciences and related professional areas: Widely used
A large fraction of our information base over the last half century
A multi-billion dollar industry
Of widespread public interest
Gary King () Anchoring Vignettes 3 / 45
![Page 11: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/11.jpg)
Examples of the Problem
Presidential Approval: the longest public opinion time series
On 9/10/2001, 55% of Americans approved of the way George W.Bush was “handling his job as president”.
The next day — which the president spent in hiding — 90% approved.
Was this massive opinion change, or was the same questioninterpreted differently?
Gary King () Anchoring Vignettes 4 / 45
![Page 12: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/12.jpg)
Examples of the Problem
Presidential Approval: the longest public opinion time series
On 9/10/2001, 55% of Americans approved of the way George W.Bush was “handling his job as president”.
The next day — which the president spent in hiding — 90% approved.
Was this massive opinion change, or was the same questioninterpreted differently?
Gary King () Anchoring Vignettes 4 / 45
![Page 13: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/13.jpg)
Examples of the Problem
Presidential Approval: the longest public opinion time series
On 9/10/2001, 55% of Americans approved of the way George W.Bush was “handling his job as president”.
The next day — which the president spent in hiding — 90% approved.
Was this massive opinion change, or was the same questioninterpreted differently?
Gary King () Anchoring Vignettes 4 / 45
![Page 14: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/14.jpg)
Examples of the Problem
Presidential Approval: the longest public opinion time series
On 9/10/2001, 55% of Americans approved of the way George W.Bush was “handling his job as president”.
The next day — which the president spent in hiding — 90% approved.
Was this massive opinion change, or was the same questioninterpreted differently?
Gary King () Anchoring Vignettes 4 / 45
![Page 15: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/15.jpg)
Examples of the Problem
Presidential Approval: the longest public opinion time series
On 9/10/2001, 55% of Americans approved of the way George W.Bush was “handling his job as president”.
The next day — which the president spent in hiding — 90% approved.
Was this massive opinion change, or was the same questioninterpreted differently?
Gary King () Anchoring Vignettes 4 / 45
![Page 16: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/16.jpg)
Examples of the Problem
The O.J. Simpson trial: most publicized murder trial in history
The facts of the case seemed clear
Did he do it? Whites: 62% say “yes”. Blacks: 14% say “yes”.
Did black and white Americans have genuinely opposing views aboutwhether Simpson committed murder, or did the two groups interpretthe same survey question differently?
Gary King () Anchoring Vignettes 5 / 45
![Page 17: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/17.jpg)
Examples of the Problem
The O.J. Simpson trial: most publicized murder trial in history
The facts of the case seemed clear
Did he do it? Whites: 62% say “yes”. Blacks: 14% say “yes”.
Did black and white Americans have genuinely opposing views aboutwhether Simpson committed murder, or did the two groups interpretthe same survey question differently?
Gary King () Anchoring Vignettes 5 / 45
![Page 18: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/18.jpg)
Examples of the Problem
The O.J. Simpson trial: most publicized murder trial in history
The facts of the case seemed clear
Did he do it? Whites: 62% say “yes”. Blacks: 14% say “yes”.
Did black and white Americans have genuinely opposing views aboutwhether Simpson committed murder, or did the two groups interpretthe same survey question differently?
Gary King () Anchoring Vignettes 5 / 45
![Page 19: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/19.jpg)
Examples of the Problem
The O.J. Simpson trial: most publicized murder trial in history
The facts of the case seemed clear
Did he do it? Whites: 62% say “yes”. Blacks: 14% say “yes”.
Did black and white Americans have genuinely opposing views aboutwhether Simpson committed murder, or did the two groups interpretthe same survey question differently?
Gary King () Anchoring Vignettes 5 / 45
![Page 20: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/20.jpg)
Examples of the Problem
The O.J. Simpson trial: most publicized murder trial in history
The facts of the case seemed clear
Did he do it? Whites: 62% say “yes”. Blacks: 14% say “yes”.
Did black and white Americans have genuinely opposing views aboutwhether Simpson committed murder, or did the two groups interpretthe same survey question differently?
Gary King () Anchoring Vignettes 5 / 45
![Page 21: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/21.jpg)
Examples of the Problem
The most common measure of the health of populations: “Howhealthy are you? Excellent, Good, Fair, or Poor”
Suppose an otherwise healthy 25-year-old woman with a cold and abackache answers “fair” and a 90-year-old man just able to get out ofbed says “excellent”
Is the young woman less healthy or are the two interpreting the samequestion differently?
In some countries, responses to this survey question correlatenegatively with objective measures of health status (Sen, 2002).
Gary King () Anchoring Vignettes 6 / 45
![Page 22: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/22.jpg)
Examples of the Problem
The most common measure of the health of populations: “Howhealthy are you? Excellent, Good, Fair, or Poor”
Suppose an otherwise healthy 25-year-old woman with a cold and abackache answers “fair” and a 90-year-old man just able to get out ofbed says “excellent”
Is the young woman less healthy or are the two interpreting the samequestion differently?
In some countries, responses to this survey question correlatenegatively with objective measures of health status (Sen, 2002).
Gary King () Anchoring Vignettes 6 / 45
![Page 23: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/23.jpg)
Examples of the Problem
The most common measure of the health of populations: “Howhealthy are you? Excellent, Good, Fair, or Poor”
Suppose an otherwise healthy 25-year-old woman with a cold and abackache answers “fair” and a 90-year-old man just able to get out ofbed says “excellent”
Is the young woman less healthy or are the two interpreting the samequestion differently?
In some countries, responses to this survey question correlatenegatively with objective measures of health status (Sen, 2002).
Gary King () Anchoring Vignettes 6 / 45
![Page 24: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/24.jpg)
Examples of the Problem
The most common measure of the health of populations: “Howhealthy are you? Excellent, Good, Fair, or Poor”
Suppose an otherwise healthy 25-year-old woman with a cold and abackache answers “fair” and a 90-year-old man just able to get out ofbed says “excellent”
Is the young woman less healthy or are the two interpreting the samequestion differently?
In some countries, responses to this survey question correlatenegatively with objective measures of health status (Sen, 2002).
Gary King () Anchoring Vignettes 6 / 45
![Page 25: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/25.jpg)
Examples of the Problem
The most common measure of the health of populations: “Howhealthy are you? Excellent, Good, Fair, or Poor”
Suppose an otherwise healthy 25-year-old woman with a cold and abackache answers “fair” and a 90-year-old man just able to get out ofbed says “excellent”
Is the young woman less healthy or are the two interpreting the samequestion differently?
In some countries, responses to this survey question correlatenegatively with objective measures of health status (Sen, 2002).
Gary King () Anchoring Vignettes 6 / 45
![Page 26: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/26.jpg)
Gary King () Anchoring Vignettes 7 / 45
![Page 27: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/27.jpg)
Anchoring Vignettes & Self-Assessments:Political Efficacy (about voting)
“[Alison] lacks clean drinking water. She and her neighbors are supporting anopposition candidate in the forthcoming elections that has promised to address theissue. It appears that so many people in her area feel the same way that theopposition candidate will defeat the incumbent representative.”
“[Jane] lacks clean drinking water because the government is pursuing anindustrial development plan. In the campaign for an upcoming election, anopposition party has promised to address the issue, but she feels it would be futileto vote for the opposition since the government is certain to win.”
“[Moses] lacks clean drinking water. He would like to change this, but he can’tvote, and feels that no one in the government cares about this issue. So he suffersin silence, hoping something will be done in the future.”
How much say [does ‘name’ / do you] have in getting the government to address issues
that interest [him / her / you]?
(a) Unlimited say, (b) A lot of say, (c) Some say, (d) Little say, (e) No say at all
Gary King () Anchoring Vignettes 8 / 45
![Page 28: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/28.jpg)
Anchoring Vignettes & Self-Assessments:Political Efficacy (about voting)
“[Alison] lacks clean drinking water. She and her neighbors are supporting anopposition candidate in the forthcoming elections that has promised to address theissue. It appears that so many people in her area feel the same way that theopposition candidate will defeat the incumbent representative.”
“[Jane] lacks clean drinking water because the government is pursuing anindustrial development plan. In the campaign for an upcoming election, anopposition party has promised to address the issue, but she feels it would be futileto vote for the opposition since the government is certain to win.”
“[Moses] lacks clean drinking water. He would like to change this, but he can’tvote, and feels that no one in the government cares about this issue. So he suffersin silence, hoping something will be done in the future.”
How much say [does ‘name’ / do you] have in getting the government to address issues
that interest [him / her / you]?
(a) Unlimited say, (b) A lot of say, (c) Some say, (d) Little say, (e) No say at all
Gary King () Anchoring Vignettes 8 / 45
![Page 29: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/29.jpg)
Anchoring Vignettes & Self-Assessments:Political Efficacy (about voting)
“[Alison] lacks clean drinking water. She and her neighbors are supporting anopposition candidate in the forthcoming elections that has promised to address theissue. It appears that so many people in her area feel the same way that theopposition candidate will defeat the incumbent representative.”
“[Jane] lacks clean drinking water because the government is pursuing anindustrial development plan. In the campaign for an upcoming election, anopposition party has promised to address the issue, but she feels it would be futileto vote for the opposition since the government is certain to win.”
“[Moses] lacks clean drinking water. He would like to change this, but he can’tvote, and feels that no one in the government cares about this issue. So he suffersin silence, hoping something will be done in the future.”
How much say [does ‘name’ / do you] have in getting the government to address issues
that interest [him / her / you]?
(a) Unlimited say, (b) A lot of say, (c) Some say, (d) Little say, (e) No say at all
Gary King () Anchoring Vignettes 8 / 45
![Page 30: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/30.jpg)
Anchoring Vignettes & Self-Assessments:Political Efficacy (about voting)
“[Alison] lacks clean drinking water. She and her neighbors are supporting anopposition candidate in the forthcoming elections that has promised to address theissue. It appears that so many people in her area feel the same way that theopposition candidate will defeat the incumbent representative.”
“[Jane] lacks clean drinking water because the government is pursuing anindustrial development plan. In the campaign for an upcoming election, anopposition party has promised to address the issue, but she feels it would be futileto vote for the opposition since the government is certain to win.”
“[Moses] lacks clean drinking water. He would like to change this, but he can’tvote, and feels that no one in the government cares about this issue. So he suffersin silence, hoping something will be done in the future.”
How much say [does ‘name’ / do you] have in getting the government to address issues
that interest [him / her / you]?
(a) Unlimited say, (b) A lot of say, (c) Some say, (d) Little say, (e) No say at all
Gary King () Anchoring Vignettes 8 / 45
![Page 31: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/31.jpg)
Does R1 or R2 have More Political Efficacy?
High←Alison1
←Jane1
Self1→
←Moses1
High
←Alison2
←Jane2Self2→
←Moses2
HighAlison2→
Jane2→
←Self2
Moses2→Low Low Low
The only reason for vignette assessments to change over respondentsis DIF
Assumption holds because investigator creates the anchors (Alison,Jane, Moses)
Our simple (nonparametric) method works this way.
Gary King () Anchoring Vignettes 9 / 45
![Page 32: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/32.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 33: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/33.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 34: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/34.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 35: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/35.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 36: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/36.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 37: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/37.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 38: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/38.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 39: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/39.jpg)
A Simple, Nonparametric Method
Define self-assessment answers relative to vignettes answers.
For respondents who rank vignettes, zi1 < zi2 < · · · < ziJ ,
Ci =
1 if yi < zi1
2 if yi = zi1
3 if zi1 < yi < zi2...
...
2J + 1 if yi > ziJ
Apportion C equally among tied vignette categories
(This is wrong, but simple; we will improve shortly)
Treat vignette ranking inconsistencies as ties
Requires vignettes and self-assessments asked of all respondents
(Our parametric method doesn’t)
Gary King () Anchoring Vignettes 10 / 45
![Page 40: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/40.jpg)
Comparing China and Mexico
Gary King () Anchoring Vignettes 11 / 45
![Page 41: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/41.jpg)
Comparing China and Mexico
Gary King () Anchoring Vignettes 11 / 45
![Page 42: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/42.jpg)
Mexico
Opposition leader Vicente Fox elected President.71-year rule of PRI party ends.
Peaceful transition of power begins.
Plenty of political efficacy
Gary King () Anchoring Vignettes 12 / 45
![Page 43: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/43.jpg)
China: How much say do you have in getting thegovernment to address issues that interest you?
Gary King () Anchoring Vignettes 13 / 45
![Page 44: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/44.jpg)
Nonparametric Estimates of Political Efficacy
No Say Little Some A lot Unlimited
Mexico
China
0.0
0.1
0.2
0.3
0.4
0.5
1 2 3 4 5 6 7 8 9 11
Mexico
China
0.0
0.1
0.2
0.3
0.4
0.5
The left graph is a histogram of the observed categoricalself-assessments.
The right graph is a histogram of C , our nonparametric DIF-correctedestimate of the same distribution.
Gary King () Anchoring Vignettes 14 / 45
![Page 45: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/45.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.
(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.(b) The quantity being estimated exists.(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 46: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/46.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.
(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.(b) The quantity being estimated exists.(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 47: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/47.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.(b) The quantity being estimated exists.(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 48: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/48.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.(b) The quantity being estimated exists.(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 49: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/49.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.
(b) The quantity being estimated exists.(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 50: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/50.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.(b) The quantity being estimated exists.
(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 51: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/51.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.(b) The quantity being estimated exists.(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 52: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/52.jpg)
Key Measurement Assumptions
1. Response Consistency: Each respondent uses the self-assessment andvignette categories in approximately the same way across questions.(DIF occurs across respondents, not across questions for any onerespondent.)
2. Vignette Equivalence:
(a) The actual level for any vignette is the same for all respondents.(b) The quantity being estimated exists.(c) The scale being tapped is perceived as unidimensional.
3. In other words: we allow response-category DIF but assume stemquestion equivalence.
Gary King () Anchoring Vignettes 15 / 45
![Page 53: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/53.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C
1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 54: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/54.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C
1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 55: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/55.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}
2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 56: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/56.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}
3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 57: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/57.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}
4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 58: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/58.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}
5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 59: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/59.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}
Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 60: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/60.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:
6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 61: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/61.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}
7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 62: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/62.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}
8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 63: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/63.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}
Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 64: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/64.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:
9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 65: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/65.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}
10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 66: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/66.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}
11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 67: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/67.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}
12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 68: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/68.jpg)
Ties and Inconsistencies Produce Ranges
Survey 1: 2: 3: 4: 5:Example Responses y < z1 y = z1 z1 < y < z2 y = z2 y > z2 C1 y < z1 < z2 T {1}2 y = z1 < z2 T {2}3 z1 < y < z2 T {3}4 z1 < y = z2 T {4}5 z1 < z2 < y T {5}Ties:6 y < z1 = z2 T {1}7 y = z1 = z2 T T {2,3,4}8 z1 = z2 < y T {5}Inconsistencies:9 y < z2 < z1 T {1}10 y = z2 < z1 T T {1,2,3,4}11 z2 < y < z1 T T {1,2,3,4,5}12 z2 < y = z1 T T {2,3,4,5}13 z2 < z1 < y T {5}
Gary King () Anchoring Vignettes 16 / 45
![Page 69: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/69.jpg)
Analyzing the DIF-Free Variable: More Efficiencies
How to analyze a variable with scalar and vector responses?
We define a new method (censored ordered probit), a direct extensionof ordinal probit allowing for ranges of responses
Useful for vignettes; also useful for survey questions that allow rangesof responses
Gary King () Anchoring Vignettes 17 / 45
![Page 70: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/70.jpg)
Analyzing the DIF-Free Variable: More Efficiencies
How to analyze a variable with scalar and vector responses?
We define a new method (censored ordered probit), a direct extensionof ordinal probit allowing for ranges of responses
Useful for vignettes; also useful for survey questions that allow rangesof responses
Gary King () Anchoring Vignettes 17 / 45
![Page 71: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/71.jpg)
Analyzing the DIF-Free Variable: More Efficiencies
How to analyze a variable with scalar and vector responses?
We define a new method (censored ordered probit), a direct extensionof ordinal probit allowing for ranges of responses
Useful for vignettes; also useful for survey questions that allow rangesof responses
Gary King () Anchoring Vignettes 17 / 45
![Page 72: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/72.jpg)
Analyzing the DIF-Free Variable: More Efficiencies
How to analyze a variable with scalar and vector responses?
We define a new method (censored ordered probit), a direct extensionof ordinal probit allowing for ranges of responses
Useful for vignettes; also useful for survey questions that allow rangesof responses
Gary King () Anchoring Vignettes 17 / 45
![Page 73: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/73.jpg)
Improved Efficiency in Practice
1 2 3 4 5 6 7 8 9 10 11
Uniform
C
0.0
0.1
0.2
0.3
0.4
Mexico
China
1 2 3,4 5,6 7,8 9,10 11
Unconditional
C
0.0
0.1
0.2
0.3
0.4
Mexico
China
1 2 3,4 5,6 7,8 9,10 11
Conditional
C
0.0
0.1
0.2
0.3
0.4
Mexico
China
Gary King () Anchoring Vignettes 18 / 45
![Page 74: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/74.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power
(E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)
Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization scheme
Operational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 75: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/75.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power
(E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)
Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization scheme
Operational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 76: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/76.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power
(E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization scheme
Operational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 77: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/77.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power (E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)
Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization scheme
Operational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 78: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/78.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power (E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization scheme
Operational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 79: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/79.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power (E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization scheme
Operational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 80: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/80.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power (E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization schemeOperational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 81: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/81.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power (E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization schemeOperational use:
Run a pretest with lots of vignettes
Compute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 82: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/82.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power (E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization schemeOperational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,
Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 83: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/83.jpg)
Optimally Choosing Vignettes
Ultimate Goal: Define categories with vignettes to learn about acontinuous unobserved variable (health, efficacy).
Worst choice: All in one category, no discriminatory power (E.g., “Bobran two marathons last week. . . ” does not discriminate amongrespondents)Best choice: Largest number of categories, equal proportions acrosscategories
Immediate Goal: Measure information in a categorization schemeOperational use:
Run a pretest with lots of vignettesCompute C and H(C) for each possible subset,Choose vignettes for the main survey based on H and cost of surveyquestions.
Gary King () Anchoring Vignettes 19 / 45
![Page 84: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/84.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):
1. H(C ) should be 0 when all answers are in one category;
at a maximumwhen proportions are equal across categories
2. H(C ) should increase monotonically with the number of vignettes (andthus categories)
3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 85: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/85.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):
1. H(C ) should be 0 when all answers are in one category;
at a maximumwhen proportions are equal across categories
2. H(C ) should increase monotonically with the number of vignettes (andthus categories)
3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 86: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/86.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category;
at a maximumwhen proportions are equal across categories
2. H(C ) should increase monotonically with the number of vignettes (andthus categories)
3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 87: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/87.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category; at a maximum
when proportions are equal across categories
2. H(C ) should increase monotonically with the number of vignettes (andthus categories)
3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 88: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/88.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category; at a maximum
when proportions are equal across categories2. H(C ) should increase monotonically with the number of vignettes (and
thus categories)
3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 89: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/89.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category; at a maximum
when proportions are equal across categories2. H(C ) should increase monotonically with the number of vignettes (and
thus categories)3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 90: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/90.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category; at a maximum
when proportions are equal across categories2. H(C ) should increase monotonically with the number of vignettes (and
thus categories)3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 91: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/91.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category; at a maximum
when proportions are equal across categories2. H(C ) should increase monotonically with the number of vignettes (and
thus categories)3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 92: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/92.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category; at a maximum
when proportions are equal across categories2. H(C ) should increase monotonically with the number of vignettes (and
thus categories)3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 93: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/93.jpg)
A measure of information H(C )?
Three Criteria for a measure, H(C ):1. H(C ) should be 0 when all answers are in one category; at a maximum
when proportions are equal across categories2. H(C ) should increase monotonically with the number of vignettes (and
thus categories)3. Assume consistent decomposition as we add vignettes
Lots of candidates exist (all inequality measures): Gini index,variance, absolute deviations, Herfindahl index, etc.
Only one measure satisfies all three criteria: entropy.
Thus, formally, we set:
H(p1, . . . , p2J+1) = −2J+1∑j=1
pj ln(pj)
Only question remaining: How do we calculate entropy when C is nota scalar?
Gary King () Anchoring Vignettes 20 / 45
![Page 94: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/94.jpg)
Step 2: Defining H(C ) for scalar and vector C
Without ties or inconsistencies, we simply compute entropy
With ties and inconsistencies, 2 options:
Estimated entropy: using the censored ordinal probit modelKnown (minimum) entropy: information in the data we know exists forcertain (inferences do not depend on the model)
Result is easy to use: one measure indicating information in surveyquestion and vignettes
Gary King () Anchoring Vignettes 21 / 45
![Page 95: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/95.jpg)
Step 2: Defining H(C ) for scalar and vector C
Without ties or inconsistencies, we simply compute entropy
With ties and inconsistencies, 2 options:
Estimated entropy: using the censored ordinal probit modelKnown (minimum) entropy: information in the data we know exists forcertain (inferences do not depend on the model)
Result is easy to use: one measure indicating information in surveyquestion and vignettes
Gary King () Anchoring Vignettes 21 / 45
![Page 96: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/96.jpg)
Step 2: Defining H(C ) for scalar and vector C
Without ties or inconsistencies, we simply compute entropy
With ties and inconsistencies, 2 options:
Estimated entropy: using the censored ordinal probit modelKnown (minimum) entropy: information in the data we know exists forcertain (inferences do not depend on the model)
Result is easy to use: one measure indicating information in surveyquestion and vignettes
Gary King () Anchoring Vignettes 21 / 45
![Page 97: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/97.jpg)
Step 2: Defining H(C ) for scalar and vector C
Without ties or inconsistencies, we simply compute entropy
With ties and inconsistencies, 2 options:
Estimated entropy: using the censored ordinal probit model
Known (minimum) entropy: information in the data we know exists forcertain (inferences do not depend on the model)
Result is easy to use: one measure indicating information in surveyquestion and vignettes
Gary King () Anchoring Vignettes 21 / 45
![Page 98: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/98.jpg)
Step 2: Defining H(C ) for scalar and vector C
Without ties or inconsistencies, we simply compute entropy
With ties and inconsistencies, 2 options:
Estimated entropy: using the censored ordinal probit modelKnown (minimum) entropy: information in the data we know exists forcertain (inferences do not depend on the model)
Result is easy to use: one measure indicating information in surveyquestion and vignettes
Gary King () Anchoring Vignettes 21 / 45
![Page 99: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/99.jpg)
Step 2: Defining H(C ) for scalar and vector C
Without ties or inconsistencies, we simply compute entropy
With ties and inconsistencies, 2 options:
Estimated entropy: using the censored ordinal probit modelKnown (minimum) entropy: information in the data we know exists forcertain (inferences do not depend on the model)
Result is easy to use: one measure indicating information in surveyquestion and vignettes
Gary King () Anchoring Vignettes 21 / 45
![Page 100: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/100.jpg)
Political Efficacy (Mex & China)
0.8 1.0 1.2 1.4 1.6 1.8 2.0
0.8
1.0
1.2
1.4
1.6
1.8
2.0
Known (Minimum) Entropy
Est
imat
ed E
ntro
py
12
3
4
5
123124125
134135
145
234235
245
345
12345
12 131415
23
24253435
45
12341235
1245
13452345
Gary King () Anchoring Vignettes 22 / 45
![Page 101: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/101.jpg)
One vignette can be better than three: Sleep (China)
0.5 1.0 1.5 2.0
0.5
1.0
1.5
2.0
Sleep
Known (Minimum) Entropy
Est
imat
ed E
ntro
py
12
34
5
123124125
134135145
234235245
345
12345
12
1314
152324
25
34
3545
123412351245
1345
2345
Gary King () Anchoring Vignettes 23 / 45
![Page 102: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/102.jpg)
Some vignette sets are uninformative: Self-Care (China)
0.5 1.0 1.5 2.0
0.5
1.0
1.5
2.0
Self−Care
Known (Minimum) Entropy
Est
imat
ed E
ntro
py
12
3
4
123124125134135145
234235245
345
12345121314
15
232425
3435
45
1234123512451345
2345
Gary King () Anchoring Vignettes 24 / 45
![Page 103: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/103.jpg)
Some covariates are unhelpful: Pain (China)
0.5 1.0 1.5 2.0
0.5
1.0
1.5
2.0
Pain
Known (Minimum) Entropy
Est
imat
ed E
ntro
py
1
2
3
4
123124125134135
145
234
235245345
12345
121314
15
2324
2534
35
45
1234123512451345
2345
Gary King () Anchoring Vignettes 25 / 45
![Page 104: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/104.jpg)
Categorizing Years of Age
Respondent 1
90
80
70
60Elderly
50
40 ←τ3
30 Middle aged
20 ←τ2Young adult
10 ←τ1Child
0
Respondent 2
90 Elderly
80 ←τ3
70 Middle aged
60
50←τ2
40 Young adult
30
20←τ1
10 Child
0
If thresholds vary, categorical answers are meaningless.
Our parametric model works by estimating the thresholds.
Vignettes provide identifying information for the τ ’s.
Gary King () Anchoring Vignettes 26 / 45
![Page 105: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/105.jpg)
Self-Assessments v. Medical Tests
Self-Assessment:In the last 30 days, how much difficulty did [you/name] have in seeing andrecognizing a person you know across the road (i.e. from a distance ofabout 20 meters)?
(A) none, (B) mild, (C) moderate, (D) severe, (E)extreme/cannot do
The Snellen Eye Chart Test:
Gary King () Anchoring Vignettes 27 / 45
![Page 106: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/106.jpg)
Self-Assessments v. Medical Tests
Self-Assessment:In the last 30 days, how much difficulty did [you/name] have in seeing andrecognizing a person you know across the road (i.e. from a distance ofabout 20 meters)? (A) none, (B) mild, (C) moderate, (D) severe, (E)extreme/cannot do
The Snellen Eye Chart Test:
Gary King () Anchoring Vignettes 27 / 45
![Page 107: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/107.jpg)
Self-Assessments v. Medical Tests
Self-Assessment:In the last 30 days, how much difficulty did [you/name] have in seeing andrecognizing a person you know across the road (i.e. from a distance ofabout 20 meters)? (A) none, (B) mild, (C) moderate, (D) severe, (E)extreme/cannot do
The Snellen Eye Chart Test:
Gary King () Anchoring Vignettes 27 / 45
![Page 108: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/108.jpg)
Fixing DIF in Self-Assessments of Visual (Non)acuity
Snellen Eye Chart Ordinal Probit ChopitMean (s.e.) µ (s.e.) µ (s.e.)
Slovakia 8.006 (.272) .660 (.127) .286 (.129)China 10.780 (.148) .673 (.073) .749 (.081)
Difference −2.774 (.452) −.013 (.053) −.463 (.053)
The medical test shows Slovakians see much better than the ChineseOrdinal probit finds no differenceChopit reproduces the same result as the medical test (though on differentscale)
Gary King () Anchoring Vignettes 28 / 45
![Page 109: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/109.jpg)
Fixing DIF in Self-Assessments of Visual (Non)acuity
Snellen Eye Chart Ordinal Probit ChopitMean (s.e.) µ (s.e.) µ (s.e.)
Slovakia 8.006 (.272) .660 (.127) .286 (.129)China 10.780 (.148) .673 (.073) .749 (.081)
Difference −2.774 (.452) −.013 (.053) −.463 (.053)
The medical test shows Slovakians see much better than the Chinese
Ordinal probit finds no differenceChopit reproduces the same result as the medical test (though on differentscale)
Gary King () Anchoring Vignettes 28 / 45
![Page 110: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/110.jpg)
Fixing DIF in Self-Assessments of Visual (Non)acuity
Snellen Eye Chart Ordinal Probit ChopitMean (s.e.) µ (s.e.) µ (s.e.)
Slovakia 8.006 (.272) .660 (.127) .286 (.129)China 10.780 (.148) .673 (.073) .749 (.081)
Difference −2.774 (.452) −.013 (.053) −.463 (.053)
The medical test shows Slovakians see much better than the ChineseOrdinal probit finds no difference
Chopit reproduces the same result as the medical test (though on differentscale)
Gary King () Anchoring Vignettes 28 / 45
![Page 111: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/111.jpg)
Fixing DIF in Self-Assessments of Visual (Non)acuity
Snellen Eye Chart Ordinal Probit ChopitMean (s.e.) µ (s.e.) µ (s.e.)
Slovakia 8.006 (.272) .660 (.127) .286 (.129)China 10.780 (.148) .673 (.073) .749 (.081)
Difference −2.774 (.452) −.013 (.053) −.463 (.053)
The medical test shows Slovakians see much better than the ChineseOrdinal probit finds no differenceChopit reproduces the same result as the medical test (though on differentscale)
Gary King () Anchoring Vignettes 28 / 45
![Page 112: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/112.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording,
Accurate translation, Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 113: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/113.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording,
Accurate translation, Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 114: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/114.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording,
Accurate translation, Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 115: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/115.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording,
Accurate translation, Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 116: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/116.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording,
Accurate translation, Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 117: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/117.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording,
Accurate translation, Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 118: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/118.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording, Accurate translation,
Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 119: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/119.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording, Accurate translation, Question order,
Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 120: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/120.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording, Accurate translation, Question order, Sampling design,
Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 121: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/121.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording, Accurate translation, Question order, Sampling design, Interview length,
Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 122: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/122.jpg)
Conclusions
Our approach can fix DIF, if response consistency and vignette equivalence hold —and the survey questions are good
Anchoring vignettes will not eliminate all DIF, but problems would have to occurat unrealistically extreme levels to make the unadjusted measures better than theadjusted ones.
Expense can be held down to a minimum by assigning each vignette to a smallersubsample. E.g., 4 vignettes asked for 1/4 of the sample each adds only onequestion/respondent.
Writing vignettes aids in the clarification and discovery of additional domains ofthe concept of interest — even if you do not do a survey.
We do not provide a solution for other common survey problems: Questionwording, Accurate translation, Question order, Sampling design, Interview length,Social backgrounds of interviewer and respondent, etc.
Gary King () Anchoring Vignettes 29 / 45
![Page 123: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/123.jpg)
For More Information
http://GKing.Harvard.edu/vign
Includes:
Academic papers
Anchoring vignette examples by researchers in many fields,
Frequently asked questions,
Videos
Conferences
Statistical software
Gary King () Anchoring Vignettes 30 / 45
![Page 124: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/124.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:
Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:
Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 125: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/125.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:
Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:
Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 126: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/126.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).
Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:
Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 127: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/127.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)
Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:
Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 128: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/128.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:
Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 129: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/129.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:
Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 130: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/130.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).
Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 131: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/131.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)
Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 132: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/132.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )
Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 133: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/133.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 134: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/134.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 135: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/135.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 136: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/136.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 137: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/137.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias.
They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 138: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/138.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias. They willsometimes not make a difference.
They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 139: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/139.jpg)
Anchoring Vignettes Measure DIF, not Vision: A Heuristic
Define µ as the quantity of interest; D as DIF.
1. If model assumptions hold:Self-assessments estimate: (µ + D).Vignettes estimate: D (they vary over i only due to DIF)Vignette-corrected self-assessments: (µ + D)− D = µ
2. If model assumptions do not hold:Self-assessments estimate: (µ + Ds).Vignettes estimate: Dv (which may differ from Ds)Vignette-corrected self-assessments: (µ + Ds)− Dv = µ + (Ds − Dv )Which is larger?
(a) Self-assessment bias: Ds
(b) Vignette-corrected self-assessment bias: (Ds − Dv )
Since the same person generates both Ds and Dv , (b) will usually besmaller.
3. Conclusion: Anchoring vignettes will usually help reduce bias. They willsometimes not make a difference. They will almost never exacerbatebias.
Gary King () Anchoring Vignettes 31 / 45
![Page 140: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/140.jpg)
Self-Assessment Component: for i = 1, . . . , n
Actual level: µi = Xiβ + ηi , with random effect ηi ∼ N(0, ω2)
Perceived level: Y ∗i1 ∼ N(µi , 1) . . . Y ∗iS ∼ N(µi , 1)
Reported Level:
yi1 = k if τk−1i1 ≤ Y ∗i1 < τk
i1
...
yiS = k if τk−1is ≤ Y ∗is < τk
is
where
τ1is = γ1Vi
τkis = τk−1
is + eγkVi (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 32 / 45
![Page 141: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/141.jpg)
Self-Assessment Component: for i = 1, . . . , n
Actual level: µi = Xiβ + ηi , with random effect ηi ∼ N(0, ω2)
Perceived level: Y ∗i1 ∼ N(µi , 1) . . . Y ∗iS ∼ N(µi , 1)
Reported Level:
yi1 = k if τk−1i1 ≤ Y ∗i1 < τk
i1
...
yiS = k if τk−1is ≤ Y ∗is < τk
is
where
τ1is = γ1Vi
τkis = τk−1
is + eγkVi (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 32 / 45
![Page 142: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/142.jpg)
Self-Assessment Component: for i = 1, . . . , n
Actual level: µi = Xiβ + ηi , with random effect ηi ∼ N(0, ω2)
Perceived level: Y ∗i1 ∼ N(µi , 1) . . . Y ∗iS ∼ N(µi , 1)
Reported Level:
yi1 = k if τk−1i1 ≤ Y ∗i1 < τk
i1
...
yiS = k if τk−1is ≤ Y ∗is < τk
is
where
τ1is = γ1Vi
τkis = τk−1
is + eγkVi (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 32 / 45
![Page 143: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/143.jpg)
Self-Assessment Component: for i = 1, . . . , n
Actual level: µi = Xiβ + ηi , with random effect ηi ∼ N(0, ω2)
Perceived level: Y ∗i1 ∼ N(µi , 1) . . . Y ∗iS ∼ N(µi , 1)
Reported Level:
yi1 = k if τk−1i1 ≤ Y ∗i1 < τk
i1
...
yiS = k if τk−1is ≤ Y ∗is < τk
is
where
τ1is = γ1Vi
τkis = τk−1
is + eγkVi (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 32 / 45
![Page 144: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/144.jpg)
Self-Assessment Component: for i = 1, . . . , n
Actual level: µi = Xiβ + ηi , with random effect ηi ∼ N(0, ω2)
Perceived level: Y ∗i1 ∼ N(µi , 1) . . . Y ∗iS ∼ N(µi , 1)
Reported Level:
yi1 = k if τk−1i1 ≤ Y ∗i1 < τk
i1
...
yiS = k if τk−1is ≤ Y ∗is < τk
is
where
τ1is = γ1Vi
τkis = τk−1
is + eγkVi (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 32 / 45
![Page 145: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/145.jpg)
Vignette Component: for ` = 1, . . . , N
Actual level: θ1, . . . , θJ
Perceived level: Z ∗`1 ∼ N(θ1, σ2) . . . Z ∗`J ∼ N(θJ , σ
2)
Reported Level: z`j = k if τk−1`1 ≤ Z ∗`j < τk
`1
where
τ1`s = γ1V`
τk`s = τk−1
`s + eγkV` (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 33 / 45
![Page 146: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/146.jpg)
Vignette Component: for ` = 1, . . . , N
Actual level: θ1, . . . , θJ
Perceived level: Z ∗`1 ∼ N(θ1, σ2) . . . Z ∗`J ∼ N(θJ , σ
2)
Reported Level: z`j = k if τk−1`1 ≤ Z ∗`j < τk
`1
where
τ1`s = γ1V`
τk`s = τk−1
`s + eγkV` (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 33 / 45
![Page 147: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/147.jpg)
Vignette Component: for ` = 1, . . . , N
Actual level: θ1, . . . , θJ
Perceived level: Z ∗`1 ∼ N(θ1, σ2) . . . Z ∗`J ∼ N(θJ , σ
2)
Reported Level: z`j = k if τk−1`1 ≤ Z ∗`j < τk
`1
where
τ1`s = γ1V`
τk`s = τk−1
`s + eγkV` (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 33 / 45
![Page 148: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/148.jpg)
Vignette Component: for ` = 1, . . . , N
Actual level: θ1, . . . , θJ
Perceived level: Z ∗`1 ∼ N(θ1, σ2) . . . Z ∗`J ∼ N(θJ , σ
2)
Reported Level: z`j = k if τk−1`1 ≤ Z ∗`j < τk
`1
where
τ1`s = γ1V`
τk`s = τk−1
`s + eγkV` (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 33 / 45
![Page 149: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/149.jpg)
Vignette Component: for ` = 1, . . . , N
Actual level: θ1, . . . , θJ
Perceived level: Z ∗`1 ∼ N(θ1, σ2) . . . Z ∗`J ∼ N(θJ , σ
2)
Reported Level: z`j = k if τk−1`1 ≤ Z ∗`j < τk
`1
where
τ1`s = γ1V`
τk`s = τk−1
`s + eγkV` (k = 2, . . . ,Ks)
Gary King () Anchoring Vignettes 33 / 45
![Page 150: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/150.jpg)
The Likelihood Function: Self-Assessment Component
If ηi were observed:
P(yi |ηi ) =n∏
i=1
S∏s=1
Ks∏k=1
[F (τ k
is |Xiβ + ηi , 1)− F (τ k−1is |Xiβ + ηi , 1)
]1(yis=k)
(S ordered probits with varying thresholds).
Since ηi is unobserved,
Ls(β, ω2, γ|y) ∝n∏
i=1
∫ ∞
−∞
S∏s=1
Ks∏k=1
[F (τ k
is |Xiβ + η, 1)
− F (τ k−1is |Xiβ + η, 1)
]1(yis=k)N(η|0, ω2)dη
In the special case where S = 1, this simplifies to
Ls(β, ω2, γ|y) =n∏
i=1
K1∏k=1
[F (τ k
i1|Xiβ, 1 + ω2)− F (τ k−1i1 |Xiβ, 1 + ω2)
]1(yi1=k)
Gary King () Anchoring Vignettes 34 / 45
![Page 151: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/151.jpg)
The Likelihood Function: Self-Assessment Component
If ηi were observed:
P(yi |ηi ) =n∏
i=1
S∏s=1
Ks∏k=1
[F (τ k
is |Xiβ + ηi , 1)− F (τ k−1is |Xiβ + ηi , 1)
]1(yis=k)
(S ordered probits with varying thresholds). Since ηi is unobserved,
Ls(β, ω2, γ|y) ∝n∏
i=1
∫ ∞
−∞
S∏s=1
Ks∏k=1
[F (τ k
is |Xiβ + η, 1)
− F (τ k−1is |Xiβ + η, 1)
]1(yis=k)N(η|0, ω2)dη
In the special case where S = 1, this simplifies to
Ls(β, ω2, γ|y) =n∏
i=1
K1∏k=1
[F (τ k
i1|Xiβ, 1 + ω2)− F (τ k−1i1 |Xiβ, 1 + ω2)
]1(yi1=k)
Gary King () Anchoring Vignettes 34 / 45
![Page 152: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/152.jpg)
The Likelihood Function: Self-Assessment Component
If ηi were observed:
P(yi |ηi ) =n∏
i=1
S∏s=1
Ks∏k=1
[F (τ k
is |Xiβ + ηi , 1)− F (τ k−1is |Xiβ + ηi , 1)
]1(yis=k)
(S ordered probits with varying thresholds). Since ηi is unobserved,
Ls(β, ω2, γ|y) ∝n∏
i=1
∫ ∞
−∞
S∏s=1
Ks∏k=1
[F (τ k
is |Xiβ + η, 1)
− F (τ k−1is |Xiβ + η, 1)
]1(yis=k)N(η|0, ω2)dη
In the special case where S = 1, this simplifies to
Ls(β, ω2, γ|y) =n∏
i=1
K1∏k=1
[F (τ k
i1|Xiβ, 1 + ω2)− F (τ k−1i1 |Xiβ, 1 + ω2)
]1(yi1=k)
Gary King () Anchoring Vignettes 34 / 45
![Page 153: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/153.jpg)
The Likelihood Function: Adding the Vignette Component
The vignette component is a J-variate ordinal probit with varyingthresholds:
Lv (θ, σ2, γ|z) ∝N∏
`=1
J∏j=1
K1∏k=1
[F (τk
`1|θj , 1)− F (τk−1`1 |θj , σ
2)]1(z`j=k)
The joint likelihood shares parameter γ:
L(β, σ2, ω2, θ, γ|y , z) = Ls(β, σ2, ω2, γ|y)× Lv (θ, γ|z).
and nests the ordinal probit model as a special case.
Gary King () Anchoring Vignettes 35 / 45
![Page 154: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/154.jpg)
The Likelihood Function: Adding the Vignette Component
The vignette component is a J-variate ordinal probit with varyingthresholds:
Lv (θ, σ2, γ|z) ∝N∏
`=1
J∏j=1
K1∏k=1
[F (τk
`1|θj , 1)− F (τk−1`1 |θj , σ
2)]1(z`j=k)
The joint likelihood shares parameter γ:
L(β, σ2, ω2, θ, γ|y , z) = Ls(β, σ2, ω2, γ|y)× Lv (θ, γ|z).
and nests the ordinal probit model as a special case.
Gary King () Anchoring Vignettes 35 / 45
![Page 155: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/155.jpg)
Fixing DIF in China and Mexico
Ordinal Probit ChopitEqn. Variable Coeff. (s.e.) Coeff. (s.e.)
µ China .670 (.081) −.362 (.090)age .004 (.003) .006 (.003)male .087 (.076) .113 (.081)
education .020 (.008) .019 (.008)
Vignettes θ1 1.393 (.190)θ2 1.304 (.190)θ3 .953 (.189)θ4 .902 (.188)θ5 .729 (.188)
lnσ −.238 (.042)
Gary King () Anchoring Vignettes 36 / 45
![Page 156: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/156.jpg)
The Source of DIF in China and Mexico: ThresholdVariation
Density
Pol
itica
l Effi
cacy
−2
−1
01
23
0.5 0.3 0.1 1 2 3 3 2 1 0 0.2 0.4
θ1
θ5
Mexico China
y*y*
τ1
τ2
τ3
τ4
µµ
Gary King () Anchoring Vignettes 37 / 45
![Page 157: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/157.jpg)
Computing Quantities of Interest
1. Effect Parameters
The effect parameters β are interpreted as in a linear regression ofactual levels µi on Xi and ηi .
2. Actual Levels, without a Self-Assessment
Choose hypothetical values of the explanatory variables, Xc
The posterior density of µc is similar to regression:
P(µc |y) = N(µc |Xc β̂, X ′c V̂ (β̂)Xc + ω̂2)
E.g., we can use the mean, Xc β̂ as a point estimate of the actual levelwhen X = Xc .
Gary King () Anchoring Vignettes 38 / 45
![Page 158: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/158.jpg)
Computing Quantities of Interest
1. Effect Parameters
The effect parameters β are interpreted as in a linear regression ofactual levels µi on Xi and ηi .
2. Actual Levels, without a Self-Assessment
Choose hypothetical values of the explanatory variables, Xc
The posterior density of µc is similar to regression:
P(µc |y) = N(µc |Xc β̂, X ′c V̂ (β̂)Xc + ω̂2)
E.g., we can use the mean, Xc β̂ as a point estimate of the actual levelwhen X = Xc .
Gary King () Anchoring Vignettes 38 / 45
![Page 159: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/159.jpg)
Computing Quantities of Interest
1. Effect ParametersThe effect parameters β are interpreted as in a linear regression ofactual levels µi on Xi and ηi .
2. Actual Levels, without a Self-Assessment
Choose hypothetical values of the explanatory variables, Xc
The posterior density of µc is similar to regression:
P(µc |y) = N(µc |Xc β̂, X ′c V̂ (β̂)Xc + ω̂2)
E.g., we can use the mean, Xc β̂ as a point estimate of the actual levelwhen X = Xc .
Gary King () Anchoring Vignettes 38 / 45
![Page 160: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/160.jpg)
Computing Quantities of Interest
1. Effect ParametersThe effect parameters β are interpreted as in a linear regression ofactual levels µi on Xi and ηi .
2. Actual Levels, without a Self-Assessment
Choose hypothetical values of the explanatory variables, Xc
The posterior density of µc is similar to regression:
P(µc |y) = N(µc |Xc β̂, X ′c V̂ (β̂)Xc + ω̂2)
E.g., we can use the mean, Xc β̂ as a point estimate of the actual levelwhen X = Xc .
Gary King () Anchoring Vignettes 38 / 45
![Page 161: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/161.jpg)
Computing Quantities of Interest
1. Effect ParametersThe effect parameters β are interpreted as in a linear regression ofactual levels µi on Xi and ηi .
2. Actual Levels, without a Self-Assessment
Choose hypothetical values of the explanatory variables, Xc
The posterior density of µc is similar to regression:
P(µc |y) = N(µc |Xc β̂, X ′c V̂ (β̂)Xc + ω̂2)
E.g., we can use the mean, Xc β̂ as a point estimate of the actual levelwhen X = Xc .
Gary King () Anchoring Vignettes 38 / 45
![Page 162: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/162.jpg)
Computing Quantities of Interest
1. Effect ParametersThe effect parameters β are interpreted as in a linear regression ofactual levels µi on Xi and ηi .
2. Actual Levels, without a Self-Assessment
Choose hypothetical values of the explanatory variables, Xc
The posterior density of µc is similar to regression:
P(µc |y) = N(µc |Xc β̂, X ′c V̂ (β̂)Xc + ω̂2)
E.g., we can use the mean, Xc β̂ as a point estimate of the actual levelwhen X = Xc .
Gary King () Anchoring Vignettes 38 / 45
![Page 163: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/163.jpg)
Computing Quantities of Interest
1. Effect ParametersThe effect parameters β are interpreted as in a linear regression ofactual levels µi on Xi and ηi .
2. Actual Levels, without a Self-Assessment
Choose hypothetical values of the explanatory variables, Xc
The posterior density of µc is similar to regression:
P(µc |y) = N(µc |Xc β̂, X ′c V̂ (β̂)Xc + ω̂2)
E.g., we can use the mean, Xc β̂ as a point estimate of the actual levelwhen X = Xc .
Gary King () Anchoring Vignettes 38 / 45
![Page 164: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/164.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 165: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/165.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 166: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/166.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 167: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/167.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X values
By Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 168: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/168.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).
Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 169: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/169.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 170: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/170.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.
Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 171: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/171.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlier
We should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 172: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/172.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 173: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/173.jpg)
Estimating Actual Levels, with a Self-Assessment
1. If we know yi , why not use it?
2. For example,
Suppose John and Esmeralda have the same X valuesBy Method 1, they give the same inferences: P(µJ |y) = P(µE |y).Suppose John’s yJ value is near µ̂J and but Esmeralda’s is far away.
Under Method 1, nothing’s new. Predictions are unchanged.Intuitively, John is average and Esmeralda is an outlierWe should adjust our prediction from µ̂E toward yE .
So the new method takes roughly the weighted average of the modelprediction µ̂E and the observed yE , with weights determined by the howgood a prediction it is.
Gary King () Anchoring Vignettes 39 / 45
![Page 174: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/174.jpg)
More formally, we use Bayes theorem
P(µi |y , yi ) ∝ P(yi |µi , y)P(µi |y),
the likelihood with ηi observed times the Method 1 posterior:
P(µi |y , yi ) ∝S∏
s=1
Ks∏k=1
[F (τ̂k
is |µi , 1)− F (τ̂k−1is |µi , 1)
]1(yis=k)
× N(µi |Xi β̂, Xi V̂ (β̂)X ′i + ω̂2)
Key Difference:P(µi |y) works for out-of-sample predictionP(µi |y , yi ) works better when yi is available
Gary King () Anchoring Vignettes 40 / 45
![Page 175: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/175.jpg)
More formally, we use Bayes theorem
P(µi |y , yi ) ∝ P(yi |µi , y)P(µi |y),
the likelihood with ηi observed
times the Method 1 posterior:
P(µi |y , yi ) ∝S∏
s=1
Ks∏k=1
[F (τ̂k
is |µi , 1)− F (τ̂k−1is |µi , 1)
]1(yis=k)
× N(µi |Xi β̂, Xi V̂ (β̂)X ′i + ω̂2)
Key Difference:P(µi |y) works for out-of-sample predictionP(µi |y , yi ) works better when yi is available
Gary King () Anchoring Vignettes 40 / 45
![Page 176: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/176.jpg)
More formally, we use Bayes theorem
P(µi |y , yi ) ∝ P(yi |µi , y)P(µi |y),
the likelihood with ηi observed times the Method 1 posterior:
P(µi |y , yi ) ∝S∏
s=1
Ks∏k=1
[F (τ̂k
is |µi , 1)− F (τ̂k−1is |µi , 1)
]1(yis=k)
× N(µi |Xi β̂, Xi V̂ (β̂)X ′i + ω̂2)
Key Difference:P(µi |y) works for out-of-sample predictionP(µi |y , yi ) works better when yi is available
Gary King () Anchoring Vignettes 40 / 45
![Page 177: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/177.jpg)
More formally, we use Bayes theorem
P(µi |y , yi ) ∝ P(yi |µi , y)P(µi |y),
the likelihood with ηi observed times the Method 1 posterior:
P(µi |y , yi ) ∝S∏
s=1
Ks∏k=1
[F (τ̂k
is |µi , 1)− F (τ̂k−1is |µi , 1)
]1(yis=k)
× N(µi |Xi β̂, Xi V̂ (β̂)X ′i + ω̂2)
Key Difference:P(µi |y) works for out-of-sample predictionP(µi |y , yi ) works better when yi is available
Gary King () Anchoring Vignettes 40 / 45
![Page 178: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/178.jpg)
More formally, we use Bayes theorem
P(µi |y , yi ) ∝ P(yi |µi , y)P(µi |y),
the likelihood with ηi observed times the Method 1 posterior:
P(µi |y , yi ) ∝S∏
s=1
Ks∏k=1
[F (τ̂k
is |µi , 1)− F (τ̂k−1is |µi , 1)
]1(yis=k)
× N(µi |Xi β̂, Xi V̂ (β̂)X ′i + ω̂2)
Key Difference:P(µi |y) works for out-of-sample predictionP(µi |y , yi ) works better when yi is available
Gary King () Anchoring Vignettes 40 / 45
![Page 179: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/179.jpg)
Unconditional Posterior
−2 −1 0 1 2
0.0
1.0
2.0
3.0
µc
Den
sity
Truth ChopitOrderedProbit
Unconditional posterior for a hypothetical 65-year-old respondent incountry 1, based on one simulated data set.
Gary King () Anchoring Vignettes 41 / 45
![Page 180: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/180.jpg)
Conditional Posteriors
−6 −4 −2 0 2 4 6
0.00
0.10
0.20
0.30
µc
Den
sity
µ1 µ2
P(µ1|y1) P(µ2|y2)
Conditional posteriors for two different 21 year old respondents. Person 1gave responses (1,1) on the two self-evaluation questions; Person 2 gaveresponses (4,3). The unconditional posterior, drawn with a dashed line,gives less specific predictions. Each curve was computed from onesimulated data set.
Gary King () Anchoring Vignettes 42 / 45
![Page 181: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/181.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach,
uses a modification of a standard statisticalmodel, and robust to misspecification.
But
it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 182: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/182.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach,
uses a modification of a standard statisticalmodel, and robust to misspecification.
But
it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 183: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/183.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach,
uses a modification of a standard statisticalmodel, and robust to misspecification.
But
it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 184: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/184.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach,
uses a modification of a standard statisticalmodel, and robust to misspecification.
But
it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 185: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/185.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach, uses a modification of a standard statisticalmodel,
and robust to misspecification.
But
it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 186: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/186.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach, uses a modification of a standard statisticalmodel, and robust to misspecification.
But
it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 187: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/187.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach, uses a modification of a standard statisticalmodel, and robust to misspecification.
But
it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 188: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/188.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach, uses a modification of a standard statisticalmodel, and robust to misspecification.
But it assumes the probit specification is correct.
Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 189: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/189.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach, uses a modification of a standard statisticalmodel, and robust to misspecification.
But it assumes the probit specification is correct. Normally this is ok,
but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 190: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/190.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach, uses a modification of a standard statisticalmodel, and robust to misspecification.
But it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 191: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/191.jpg)
Estimated Entropy
Measures the informativeness of the vignettes,
as supplemented by the predictive information in the covariates
A reasonable approach, uses a modification of a standard statisticalmodel, and robust to misspecification.
But it assumes the probit specification is correct. Normally this is ok,but decisions here are more consequential since they affect datacollection decisions and thus can preclude asking some questions
Thus, we also want “known entropy”.
Gary King () Anchoring Vignettes 43 / 45
![Page 192: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/192.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 193: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/193.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 194: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/194.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 195: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/195.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})
All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 196: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/196.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)
Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 197: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/197.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)
Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 198: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/198.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 199: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/199.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 200: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/200.jpg)
Computing Known Entropy (no assumptions required)
Scalar-valued Ci observations are set to observed values.
Vector-valued Ci :
Elements of all possible vector responses are parameterized: (e.g.,p1, p2, p3 for Ci = {2, 3, 4})All mass is restricted to within the vector (e.g., p1 + p2 + p3 = 1)Choose all p’s to minimize entropy (i.e., adjust the p’s to see how spikythe distribution can become)Some tricks make this easy with a genetic optimizer.
Then form the histogram (summing the p’s) and compute entropy.
We now compute estimated entropy and known entropy for all possiblesubsets of vignettes.
Gary King () Anchoring Vignettes 44 / 45
![Page 201: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/201.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 202: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/202.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 203: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/203.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 204: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/204.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.
For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 205: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/205.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.
For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 206: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/206.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.
Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 207: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/207.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.
Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 208: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/208.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 209: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/209.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,
extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45
![Page 210: What to do about Biases in Survey ResearchAnalysis, 15, 1 (Winter, 2007): 46–66. Gary King; Christopher J.L. Murray; Joshua A. Salomon; and Ajay Tandon. “Enhancing the Validity](https://reader033.vdocument.in/reader033/viewer/2022050406/5f8342f8145cfd524e00e8f7/html5/thumbnails/210.jpg)
Robust Analysis via Conditional Model
Condition on observed value of ci :
Pr(C = c |x0, ci ) =
{ Pr(C=c|x0)Pa∈ci
Pr(C=a|x0)for c ∈ ci
0 otherwise
Advantages compared to unconditional probabilities:
Conditions on ci by normalizing the probability to sum to one withinthe set ci and zero outside that set.For scalar values of ci , this expression simply returns the observedcategory: Pr(C = c |xi , ci ) = 1 for category c and 0 otherwise.For vector valued ci , it puts probability density over categories withinci , which in total sum to one.Probabilities can be interpreted for causal effects or summed toproduce a histogram.Result:
highly robust to model mispecification,extracts considerably more information from anchoring vignette data.
Gary King () Anchoring Vignettes 45 / 45