Faculdade de Medicina da Universidade do Porto

Curso de Mestrado Integrado em Medicina2010/2011

INTRODUÇÃO À MEDICINA II

TURMA 21REGENTE: Prof Dr. Altamiro Costa Pereira

PROFESSORA E ASSISTENTE: Prof Drª. Cristina Santos e Luís Pinto

http://jpn.icicom.up.pt/imagens/up/up_fmedicina.jpg

Biases in questionnaire construction:

how much do they influence the

answers given?

Types of errorsRandom Errors

Affect questionnaires’ precision

Related to the variability inherent to a phenomenon that is being measured [1]

Systematic Errors Affect

questionnaires’ validity

Related to the wrong association between an exposure anda certain effect [1]

Commonly known as “biases”

1. M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-641

Scenario Analysis of questionnaires’

results based on false/inaccurate information

Leads To

Descriptions and catalogues of common

biases [2,5]

To inform investigators To minimize biases

Description of biases in literature

2. M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-6415. Bernard C.K. Choi and Anita W.P. Pak. A Catalog of Biases in Questionnaires. Preventing Chronic Diseases.

2005; 1; 13

Described biasesQuestion Design

• Ambiguous questions • Technical jargon  • Uncommon words

Problems with wording

• Leading questionLeading questions

• Missing intervalFaulty scale

• Sensitive questionsIntrusiveness

• Belief vs. behavior

Missing or inadequate data for intended purpose

Described biasesAdministration of questionnaire

The “Don’t know”, “Undecided”, and “Neutral response” options

• Faking Good

Respondent’s conscious reaction

Described biases

Questionnaire Design

 

Formatting problem

• Horizontal response format

Research Question Do these so called

‘biases’ really influence the answers given in

questionnaires?

http://static4.depositphotos.com/1000423/471/i/110/depositphotos_4715431-3D-man-with-question-mark.jpg

Check if the way we change a normal question influences the given answers in general.

Be able to sort out at what extent can each of the types of biases mentioned above manipulate the results obtained.

To realize whether there is evidence that the described

biases interfere in questionnaires’ results

Aims

Methods

Two versions of the same

questionnaireA and B

Both with biased and unbiased questions

Target Population

First year students from FMUP

Versions A and B• Randomly assigned

questions to versions A and B of the questionnaire

Questionnaire

arquitecture

• 19 different classes• Distribution prevented them from

detecting two different versions of the same questionnaire.

Questionnaire

Distribution• Gathering the answers given• Creation of a data base in SPSS• Comparison between the supposedly

biased and respective non-biased questions using Qui-square test and T-student test

• Analysing if the documented biases are indeed biases (observing p value)

Questionnaire Analysis

Data AnalysisThe analysis of the answers given was

only possible because we consider that the odds of obtaining a certain answer is similar in both groups of each class.

Our outcome was the rates in the answers given to supposedly biased and unbiased questions, comparing the rates to see if there was any significant differences.

To analyse our results we use the statistical analysis software SPSS 19.0.

Final Results and Discussion

Question A1 vs. B1-2  Biased question (A1) :

Do you lend your tooth paste or tooth brush to other people?

Yes [ ]

No [ ]Unbiased question (B) :

1) Do you lend your tooth paste to other people?Yes [ ]No [ ] 

2) Do you lend your tooth brush to other people?Yes [ ]No [ ]

A B1 p

No 77 (77,8%) 29 (30,9%)<0,0001

Yes 22 (22,2%) 65 (69,1)

A B2 p

No 77 (77,8%) 93 (100%)<0,0001

Yes 22 (22,2%) 0 (0%)

Ambiguous question

Question A2 vs. B3  Biased Question (A2):

When you take a bath, do you wash your genital area?

Yes [ ]

No [ ] Unbiased question (B3):

When you take a bath, do you wash your hair?

Yes [ ]

No [ ]

A B p

No 0 (0%) 1 (1%)

0,301Yes 99 (100%) 92 (95,8%)

Missing 0 (0%) 3 (3,2%)

Sensitive question

Question A3 vs. B4Biased question (B4):

You have a shower everyday, don´t you?

Yes [ ]

No [ ]

Unbiased question (A3):

How often do you have a shower in the week: More than 6 days per week[ ]

5-6 days per week [ ]

2-4 days per week [ ]

1 day per week[ ]

I never have a shower [ ]

A B p

(A)More than 6 days per week/(B)Yes

67(67,7%)

82(87,2%)

0.001A)I never have a

shower; 1, 2-4, 5-6 days per week/ (B) No

32(32,3%)

12(12,8%)

Faking good

Question A4 vs. B5Biased question (A4):

Do you reckon that physical exercise is a germane aspect of your health?

Yes [ ]

No [ ] Unbiased question (B5):

Do you consider that physical exercise is a relevant aspect of your health?

Yes [ ]

No [ ]A B Confidence

Intervals

No 7 (7,07%)[-0,00493;0,019125]

3 (3,1%)[-0,00486;0,011062]

_

Yes 85 (85,8%)[0,809155; 0,908845]

91 (94,8%)[0,916201;0,979799]

Statistically Significant

Missing 7 (7,07%)[-0,00493;0,01907]

2 (2,1%)[-0,00446;0,008656]

_

Uncommon word

Question A5 vs. B6Biased question (B6):

Do you think it is important to practice physical exercise at least 3 times a week?

Yes [ ]

No [ ] Unbiased question (A5):Do you practice physical exercise at least 3 times a week?

Yes [ ]No [ ]

A B p

No 69 (69,7%) 15 (16,1%)

<0,0001Yes 30 (30,3%) 78 (83,9%)

Belief vs. behaviour

Question A6 vs. B7Biased question (B7):Your eating habits are: Excellent [ ] Very good [ ] Good [ ] Fair [ ] Somehow deficient [ ] Poor [ ] Very poor [ ]

Unbiased question (A6):Your eating habits are:

Excellent [ ] Very good [ ]Good [ ] Fair [ ] Somehow deficient [ ]Poor [ ] Very poor [ ]

A B p

Very Poor 2 (2%) 0 (0%)

0,463

Poor 2 (2%) 2 (2,1%)

Somehow deficient

5 (5,1%) 2 (2,1%)

Fair 14 (14,1%) 20 (21,3%)

Good 55 (55,6%) 56 (59,6%)

Very Good 17 (17,2%) 11 (11,7%)

Excellent 4 (4%) 3 (3,2%)

Horizontal response format

Question A7 vs. B8Biased question (A7):

Do you agree that McDonalds is responsible for the increase of hypertension rates?

Agree [ ]

Neutral opinion [ ]

Disagree [ ]

Unbiased question (B8):

Do you agree that McDonalds is responsible for the increase of hypertension rates?

Agree [ ]Disagree [ ]

A BConfidence

Intervals

Disagree12 (12,1%)

[-0,00356;0,027759]33 (35,9%)

[0,008297;0,060503] _

Agree49 (49,5%)

[0,018434;0,080566]59 (64,1%)

[0,027091;0,095909]_

“Neutral response” options

Question A8 vs. B9Biased question (B9):

How many days per week do you dedicate to study anatomy?

2 days [ ]

5 days [ ]

7days [ ]

Unbiased question (A8):

How many days per week do you dedicate to study anatomy?

1 day [ ]

2 days [ ]

3 days [ ]

4 days [ ]

5 days [ ]

6 days [ ]

7 days [ ]Mean

Standard deviation

p

A 4,11 1,4350,462

B 3,95 1,649

Faulty scale

Question A9 vs. B10Biased question (B10):

Do you arrange activities with your friends, such as cinema, every week?

Yes [ ]No [ ]

Unbiased question (A9):Do you arrange activities with your friends every week?

Yes [ ]No [ ]

A B p

No 30 (30,9%) 25 (36,6%)

0,509Yes 67 (69,1%) 69 (73,4%)

Leading question

Question A10 vs. B11Biased question (A10):

Did you exercise your Wernicke’s area while responding to this questionnaire?

Yes [ ]

No [ ]

Unbiased question (B11):

Did you understand what you read in this questionnaire?

Yes [ ]

No [ ]

A BConfidence

Intervals

Yes53 (53,5%)

[0,463564;0,606436]93 (96,9%)

[0,944177;0,993823]Statistically Significant

No20 (20,2%)

[0,144497;0,259503]1 (1%)

[-0,00353;0,005527]Statistically Significant

Missing26 (26,3%)

[0,003381;0,049219]2 (2,1%)

[-0,00446;0,008656]_

Technical jargon

ConclusionSumming up, there are a lot of

biases described in literature. Through this study, it has been concluded that some of them might work and others not. We cannot, however, extrapolate these results to the general population.

http://us.cdn2.123rf.com/168nwm/texelart/texelart1011/texelart101100030/8220784-3d-human-by-the-symbol-of-victory-with-his-fingers.jpg

References (Presentation)

1. M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-641

2. Bernard C.K. Choi and Anita W.P. Pak. A Catalog of Biases in Questionnaires. Preventing Chronic Diseases. 2005; 1; 13

3. http://www.statpac.com/surveys/undecided-category.htm

Afonso QuelhasAndré SantosBárbara AraújoCarolina SilvaCatarina MarquesCláudia OliveiraLaura GonçalvesLuís SousaMarta Sanhá Rita CostaTânia Costa