quantitative evaluation of questionnaire translation with ... · quantitative evaluation of...
TRANSCRIPT
![Page 1: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/1.jpg)
Sunghee Lee
Julia Lee
University of Michigan
Quantitative Evaluation
of Questionnaire
Translation with
Bilingual Speakers
AAPOR 2012
![Page 2: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/2.jpg)
Literal Equivalency
Functional Equivalency
Measuring the same intended construct
Cultural relevance
Question wording + Response scale
QUESTIONNAIRE TRANSLATION
2
![Page 3: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/3.jpg)
Yes/No
Vague quantifiers
RESPONSE SCALE
3
![Page 4: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/4.jpg)
Mostly qualitative works
To quantitatively evaluate the
translation, need randomized
experiments
- Perfect bilingual speakers
- Randomizing interview language
- Any observed differences attributable
imperfect translation
DIFFICULTY IN EVALUATION
4
![Page 5: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/5.jpg)
National Latino and Asian American
Study (NLAAS), 2003
Target populations: Hispanics and Asians
Our Focus: English-Spanish bilingual
Hispanics (Self-rated equal proficiency
on both languages)
5
DATA
![Page 6: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/6.jpg)
Total %
Spanish Only 72 4.5%
Mostly Spanish 521 32.7%
20.8%
Mostly English 647 40.6%
English Only 21 1.3%
Total 1,593
DATA
6
Interview
language
randomization
182 English
150 Spanish
NLAAS Hispanic Sample
Spanish/English 332
![Page 7: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/7.jpg)
One-to-One Translation
Poor, fair, good, and excellent
Spanish/English Proficiency
One-to-Two Translation
Frequency scale
Often, Sometimes, Rarely, Never
Response Style
7
ANALYSIS
![Page 8: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/8.jpg)
SELF-RATED SPANISH PROFICIENCY
8
![Page 9: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/9.jpg)
SELF-RATED ENGLISH PROFICIENCY
9
![Page 10: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/10.jpg)
Translation incurred measurement error
RESPONSE SCALE DIFFERENCES
10
English
Spanish
Poor Fair Good Excellent
Pobre Regular Bien Excelente
Incomparable measurement
Measurement error
![Page 11: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/11.jpg)
FREQUENCY SCALE
11
English
Spanish
Translation
A
Spanish
Translation
B
Often Muchas veces Muchas veces
Sometimes Alguna veces Alguna veces
Rarely Pocas veces Casi nunca
Never Nunca Nunca
![Page 12: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/12.jpg)
How often does your (spouse/partner) make too many demands on you, often, sometimes, rarely, or never ?
muchas veces, algunas veces, pocas veces o nunca?
How often do your relatives or children make too many demands on you, often, sometimes, rarely, or never?
muchas veces, algunas veces, casi nunca o nunca?
FREQUENCY SCALE
12
![Page 13: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/13.jpg)
FREQUENCY SCALE: MARRIAGE Qs
13
![Page 14: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/14.jpg)
FREQUENCY SCALE: OTHER Qs
14
![Page 15: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/15.jpg)
Literature suggests Hispanics’ higher tendency on
Extreme response style
In NLAAS, 10 questions with “Strongly Agree,” to “Strongly Disagree” scale
Bilingual Spanish respondents consistently have higher frequency on “Strongly Agree”
RESPONSE STYLE
15
![Page 16: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/16.jpg)
Why did we find these differences?
Functionally equivalent?
Cultural norm invoked by language?
Translation differences?
DISCUSSION
16
![Page 17: Quantitative Evaluation of Questionnaire Translation with ... · Quantitative Evaluation of Questionnaire Translation with Bilingual Speakers AAPOR 2012 Literal Equivalency Functional](https://reader033.vdocument.in/reader033/viewer/2022060402/5f0e69ec7e708231d43f20cc/html5/thumbnails/17.jpg)
Analyze variables with different
response scales.
Implement various measurement
models to adjust for incomparable
measurement error.
17
WHAT’S NEXT?