biases in questionnaires
Post on 23-Feb-2016
73 Views
Preview:
DESCRIPTION
TRANSCRIPT
BIASES IN QUESTIONNAIRES
Introduction to Medicine IISchool Year 2010/2011Class 8, First Year
Regent Teacher: Altamiro Costa PereiraSupervisor Teachers: Cristina da Costa Santos and
Luís Pinto
FICTION OR REALITY?
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References
The importance of questionnaires
People’s Opinion Politics Religion Sports ScienceDemography Medicine …
Introduction
Collected through Have applications
in…
Questionnaires
(1) Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281-91
Biases
Introduction
Sistematic mistakes or Biases:
The way each question of a questionnaire is written may or may not influence the answer of the inquired person
Errors related to not measuring the situation that is intended to be
measured.
Biased Questionnaire
(2) Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34:265-71.
(5)
Question Design
Questionnaire Design
Administration of
Questionnaire
• Insensitive measure
• Forced choice
• Sensitive question
• Vague word
• Overlapping interval
• Ambiguous question
• Juxtaposed scale• Horizontal response format
• End aversion
• Primacy and recency
Types of Biases
(3) Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health. 2004. 58(8): 635-641.(4) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.
(5) Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Biases
Introduction
“A catalog of biases in
questionnaires”
… it wasn’t found any article which proves the influence of those biases in questionnaires.
There are several articles identifying possible biases.
BUT…
(6) Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; 1982.(7) Foddy W. Constructing questions for interviews and questionnaires: theory and practive in social research. Cambridge (UK): Cambridge University Press; 1993.(8) Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; 1998. p. 331-8. (9) Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63.
Importance of the Work
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References
Find out if the types of biases such as:
Question Design; Questionnaire Design; Administration of Questionnaire…
…Influence the answers given in a questionnaire.
Aim
Could the way each question is made affect, in a significant way, the answers given by the
inquired person?
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References
Participants and Methods
“Utilização dos telemóveis pelos estudantes do primeiro ano do Mestrado Integrado em Medicina (MIM) da Faculdade de Medicina da Universidade do Porto (FMUP)”
All the students of the first year of MIM from FMUP
Question Design Questionnaire Design Administration of Questionnaires Other questions
Define the theme of the questionnaire
Define the target
population
Types of biased
questions
Elaboration of two versions
of a questionnaire
Random distribution of questionnaire
s by per of classes
Collect, organize and
analyse
A1
A2
Participants and Methods
IBM SPSS STATISTICS 1
Collect, organize and analyse
Participants and MethodsWebsite
http://www.wix.com/fmupturma08/im11
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References
Questionnaires (9)
Questionnaire A2
Visual Analogical Scale
Questionnaire A1
Questionnaire A2
Questionnaire A1
QuestionnairesQuestion design - Forced Choice
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1
Questionnaire A2
QuestionnairesQuestion design - Sensitive question
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A2
Questionnaire A1
Questionnaires
(10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.
Questionnaire design - Juxtaposed scale
Questionnaire A1
Questionnaire A2
QuestionnairesQuestion Design – Forced Choice
5.
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1
Questionnaire A2
QuestionnairesQuestion Design - Vague Word
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1
Questionnaire A2
QuestionnairesQuestion Design - Overlapping interval
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A2
Questionnaire A1
QuestionnairesQuestion Design - Ambiguous question
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A2
Questionnaire A1
QuestionnairesQuestionnaire Design - Horizontal response format
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1
Questionnaires
Questionnaire A2
Administration of Questionnaire – Primacy and recency
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References
Expected Results
Answers
Questions of Questionnaires
A1
Questions of Questionnaires
A2≠
Expected Results
The way each question is made affects, in a significant way, the answers given by the inquired
person.
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References
Results and DiscussionQuestionnaire A1 Questionnaire A2
Total* of inquired students (N) 107 of 118 98 of 120
Percentage of inquired students 91% 82%
* The total number of students who should answer the questionnaires AI and A2 was 118 and 120, respectively. However, only 107 and 98 answered because the others weren’t present when the questionnaires were delivered.
Questionnaire A2Questionnaire A1
Questionnaire A1 (N=107)
Questionnaire A2 (N=97) p Value
Quão importante é o telemóvel para ti?Median (min-max) 7,4 (0-10) 8,0 (3-10) < 0,001
Tests: Mann-Whitney
Results and Discussion
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A2Questionnaire A1
RESPOSTAS VÁLIDAS:Questionnair
e A1(N=85)
QuestionnaireA2
(N=98)p Value
Conseguia fazer a vida normal sem telemóvel. *
N (%)
29 (34%) 32 (33%) 0,834* Neutral questions (“Não sei”) weren’t considered.
Tests: Chi-square
Results and Discussion
21% “não sei”CI95%=[13%,29%]
0%Mmissing answers
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1 Questionnaire A2
Questionnaire A1 (N=107)
QuestionnaireA2
(N=98)p Value
Considera-se viciado no uso do telemóvel
N (%)
21 (20%) 19 (19%) 0,966
Tests: Chi-square
Results and Discussion
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A2Questionnaire A1
(10) Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.
Questionnaire A1 (N=106)
Questionnaire A2 (N=98) p Value
Quão importante/satisfei
to está com as aplicações do
telemóvel.Mediana (1º quartil – 3º
quartil)
Aplicações Importância
Satisfação
Importância
Satisfação
Importância Satisfação
Funcionalidades
3 (2-4) 4 (4-5) 3 (3-5) 4 (4-5) 0,011 0,655
Duração da bateria
4 (4-5) 3 (3-4) 5(4-5) 3 (2-4) 0,006 0,214
Tarifário 5 (4-5) 4 (3-5) 5(4-5) 4 (3-5) 0,177 0,778
Design 3 (2-4) 4 (3-5) 3(2-4) 4 (4-5) 0,104 0,817Tests: Mann-Whitney
Results and Discussion
Questionnaire A1 Questionnaire A2
“Concordo/Discordo” not considered Questionnaire A1 (N=45)
Questionnaire A2 (N=97) p Value
Concorda com o desbloqueio ilegal dos telemóveis. *
N (%)
Concordo
32 (71%) 66 (68%) 0,713
Tests: Chi-Square
Results and Discussion
58% “Concordo/Discor
do”CI95%=[49%;68
%]
1Missing answer
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1 Questionnaire A2
Questionnaire A1 (N=107)
QuestionnaireA2
(N=98)p Value
Em média, frequência de envio
de mensagensN (%)
Nunca (0 mensagens diárias)
1 (1%) 0 (0%)
Raramente (1-2 mensagens diárias)
5 (5%) 2 (2%)
Ocasionalmente (3-10 mensagens diárias)
27 (25%) 20 (20%)
Frequentemente (11-30 mensagens diárias)
39 (36%) 61 (62%)
Constantemente (mais de 30 mensagens diárias)
35 (33%) 15 (15%)
Results and Discussion
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1 Questionnaire A2
Questionnaire A1 (N=107)
Questionnaire A2 (N=98) p Value
Mensagens enviadas numa aula teórica
Zero 6 (6%) 1 (1%)
Uma a três 24 (23%) 25 (26%)
Quatro a dez 40 (38%) 37 (38%)
Mais de dez 36 (34%) 35 (36%)
Results and Discussion
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A2Questionnaire A1
Questionnaire A1 (N=107)
Questionnaire A2 (N=97) p Value
Uso de telemóveis prejudica as relações
sociais dos jovensN (%)
Sim 29 (27%) 21 (22%) 0,366
Tests: Chi-Square
Results and Discussion
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A2Questionnaire A1
Questionnaire A1 (N=107)
Questionnaire A2 (N=98) p Value
Quando utilizas o telemóvel, o que fazes mais frequentemente?
N (%)
Música 4 (5%) 6 (7%)Jogos 0 (0%) 1 (1%)
Chamadas 15 (18%) 14 (16%)Internet 2 (2%) 1 (1%)
SMS/MMS 59 (72%) 67 (75%)Fotos 0 (0%) 0 (0%)
Messenger 0 (0%) 0 (0%)Outros 2 (2%) 0 (0%)
Results and Discussion
23% of missing answers (IC95%=
15,0;31,0)
9% of missing answers (IC95%=
3,3;14,7)[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
Questionnaire A1 Questionnaire A2
Questionnaire A1 (N=107)
Questionnaire A2 (N=98) p Value
Quais são os três tipos de pessoas com quem mais falas ao
telemóvel?N (%)
Pai 36 (34%) 21 (23%) 0,051
Mãe 78 (74%) 59 (64%) 0,054
Amigos 83 (79%) 71 (77%) 0,397
Namorado(a) 43 (41%) 44 (48%) 0,495Professores 1 (1%) 2 (2%) 0,607
Avós 0 (0%) 0 (0%) -Irmão/Irmã 14 (13%) 18 (20%) 0,298
Primos 4 (4%) 2 (2%) 0,685Colegas de Faculdade
56 (53%) 59 (64%) 0,257
Pessoas que não conheces
pessoalmente0 (0%) 0 (0%) -
Tests: Chi-Square
Results and Discussion
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
It’s very hard to write a biased questionnaire with only one bias per question.
Limitations
Results and Discussion
We want to understand if there are significant differences between the questionnaire considered biased
and the one considered neutral.
The same inquired person should answer both questionnaires
In what way is that a
limitation?
Limitations
Results and Discussion
Minimize selection bias
The questionnaires were randomly distributed to the inquired people
Limitations
Results and Discussion
Limitations
Results and Discussion
The homogeneity of the population:
Same age Same course Same college
Limitations
Results and Discussion
Lack of sample size in some options
Some statistical tests
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results and Discussion Conclusion References
Conclusion
Could the way each question is made affect, in a significant way, the answers
given by the inquired person?
Yes
Questionnaire A1
ConclusionQuestionnaire A1Questionnaire
DesignJuxtaposed Scale
Visual Analogical Scale
Conclusion
Some recommendations:
Use visual analogical scales;
Presence of neutral option;
Mixing concepts;
Attention to the order of the options.
Summary Introduction
Importance of questionnaires Biases Types of biases Importance of our work
Aim Participants and Methods Questionnaires Expected Results Results Discussion Conclusion References
References[1] Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf). 2005;27(3):281-91.
[2] Choi BC, Noseworthy AL. Classification, direction, and prevention of bias in epidemiologic research. J Occup Med 1992;34:265-71.
[3] Rodríguez MD, Llorca J. Bias. J Epidemiol Community Health. 2004. 58(8): 635-641.
[4]Hunt DM, Magruder S, Bolon DS. Questionnaire format bias: when are juxtaposed scales appropriate: a call for further research. Psychol Reports 1995;77:931-41.
[5] Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005 Jan;2(1):A13.
[6] Sudman S, Bradburn NM. Asking questions: a practical guide to questionnaire design. San Francisco (CA): Jossey-Bass Pulishers; 1982.
[7] Foddy W. Constructing questions for interviews and questionnaires: theory and practice in social research. Cambridge (UK): Cambridge University Press; 1993.
[8] Choi BC, Pak AW. Bias, overview. In: Armitage P, Colton T, editors. Encyclopedia of biostatistics. Vol 1. Hoboken (NJ): John Wiley&Sons, Inc; 1998. p. 331-8.
[9] Sackett DL. Bias in analytic research. J Chron Dis. 1979;32:51–63.
[10] Salant P, Dillman DA. How to conduct your own survey. New York: John Wiley & Sons, Inc; 1994.
Class 8 Ana Raquel Roca Andreia Freitas Bárbara Vaz Bárbara Neto Joana Costa João Alves Jorge Nogueiro Lucas Lopes Margarida Capitão Maria Durães Mariana Silva Raquel Cardoso
Email: fmup.turma08@gmail.com
top related