questionnaire design: general principles lu ann aday, ph.d. the university of texas school of public...
TRANSCRIPT
QUESTIONNAIRE DESIGN: GENERAL PRINCIPLES
Lu Ann Aday, Ph.D.The University of Texas School of Public Health
MEASUREMENT: Definition of Variables
TYPE DESCRIPTIONCONCEPTUALDEFINITION
Major concept or issue of interest, e.g., obesity
OPERATIONALDEFINITION
Specific questions asked on the survey to obtain information on the concept
VARIABLEDEFINITION
Variable(s) constructed from the questions to actually be used in the analyses, e.g., Body Mass Index
MEASUREMENT: Levels of Measurement
LEVEL/PROPERTY
Nominal
Ordinal Interval
Ratio
Classify X X X X
Rank X X X
Distance between points
X X
Distance from zero
X
MEASUREMENT: Example Concept: Obesity
LEVEL/DEFINITION
Nominal Ordinal Interval
OPERATIONALDEFINITION
Do you consider yourself overweight, underweight, or just about right?
Would you say you are very overweight, somewhat overweight, or only a little overweight?
About how tall are you without shoes?About how much do you weigh without shoes?
VARIABLEDEFINITION
1=overweight2=underweight3=about right
1=very overweight2=somewhat overweight3=only a little overweight
Body Mass Index (BMI)=weight divided by height, squared
MEASUREMENT: Example Concept: Family
Income
LEVEL/DEFINITION
Nominal Ordinal Ratio
OPERATIONALDEFINITION
Did anyone in your family have income from wages or salaries in the past 12 months?
Which of the following best describes your family’s total income from wages & salaries in the past 12 months?
What was your family’s total income from wages & salaries during the past 12 months?
VARIABLEDEFINITION
1=Yes2=No
1=<$35,0002=$35,000-$49,9993=$50,000+
$ __,____/year
PRINCIPLES FOR WRITING QUESTIONS
(Dillman, 2000)
1. Choose simple over specialized words.
2. Choose as few words as possible to pose the question.
3. Use complete sentences to ask questions.
4. Avoid vague quantifiers when more precise estimates can be obtained.
PRINCIPLES FOR WRITING QUESTIONS
(Dillman, 2000)
5. Avoid specificity that exceeds the respondent’s potential for having an accurate, ready-made answer.
6. Use equal numbers of positive and negative categories for scalar questions.
7. Distinguish undecided from neutral by placement at the end of the scale.
PRINCIPLES FOR WRITING QUESTIONS
(Dillman, 2000)
8. Avoid bias from unequal comparisons.
9. State both sides of attitude scales in the question stems.
10. Eliminate check-all-that-apply question formats to reduce primacy effects.
11. Develop response categories that are mutually exclusive.
PRINCIPLES FOR WRITING QUESTIONS
(Dillman, 2000)
12. Use cognitive design techniques to improve recall.
13. Provide appropriate time referents.
14. Be sure each question is technically accurate.
15. Choose question wordings that allow essential comparisons to be made with previously collected data.
PRINCIPLES FOR WRITING QUESTIONS
(Dillman, 2000)
16. Avoid asking respondents to say yes when they mean no.
17. Avoid double-barreled questions.
18. Soften the impact of potentially objectionable questions.
19. Avoid asking respondents to make unnecessary calculations.
SCALES: Definition
Scales are measurement instruments that are collections of items [questions/statements] intended to reveal theoretical variables [abstract concepts] not readily observable by direct means, e.g., patient satisfaction, locus of control.
SCALES:Underlying Structure
ABSTRACT CONCEPT, e.g., Unsatisfactory Patient Encounters Specific Items, e.g., series of
statements regarding the nature of Patient Encounters
Uniform Response Categories, e.g., scale of 1 to 5, where 1 is very typical and 5 is not at all typical of patient encounters
SCALES: Example—Unsatisfactory Patient Encounters(Aday & Cornelius, 2006, NDMS, Q. 3)
VeryTypical
1 (Circle one response for each item.)
Not at allTypical
5
1. The patient was trying to manipulate me.
1 2 3 4 5
11. I felt that the patient didn’t believe me.
1 2 3 4 5
18. The patient and I seemed to come from different worlds.
1 2 3 4 5
SCALES:Methodological Issues
ABSTRACT CONCEPT Internal consistency reliability: Verify that the specific items in
the scale relate to the CONCEPT. Construct validity: Verify that the specific items in
the scale define and distinguish the CONCEPT from other concepts.
SCALES:Methodological Issues
Specific Items Take reading level of respondents
into account in phrasing items. Avoid jargon and colloquialisms. Express only one idea in each
item. Use both positively & negatively
worded items, but avoid use of negatives (e.g., “not”) in negatively wording an item.
SCALES:Methodological Issues
Uniform Response Categories Decide on a reasonable number of
response categories, e.g., 3-10, taking concept and respondents’ ability to discriminate into account.
Select meaningful labels/terms for categories, e.g., strongly agree, agree, disagree, strongly disagree; excellent, good, fair, poor.
MEASUREMENT MATRIX: National Dental Malpractice Survey (Aday & Cornelius, 2006, Table 15.1)
QUESTION CONCEPT LEVEL OBJECTIVE
3 Doctor-patient communication: characteristics of unsatisfactory patient encounters (Likert scale)
ordinal(interval) 3
10 Practice characteristics: avg. office waiting time for patient
ordinal 3
28 Practice finances: % patients with insurance coverage
interval 3
35 Malpractice insurance: no. of complaints
ordinal 1, 2, 3
54 Demographics: gender
nominal 2, 3
SURVEY ERRORS: General Principles for Formulating Questions
Systematic Errors
Variable Errors
Questionnaireeffects: under- or over-reporting
Respondent effects: yeah-saying
Mode effects: systematic
Questionnaire effects: order & context
Interviewer effects: interviewer variability
Mode effects: variable
Solutions to errors
See Chapters Ten and Twelve.
See Chapters Ten, Eleven, and Twelve.
See Chapters Five and Ten.
See Chapters Ten, Eleven, and Twelve.
See Chapter Thirteen.
See Chapter Five.
REFERENCES DeVellis, Robert F. (2003). Scale
Development: Theory and Applications. Second Edition. Thousand Oaks, CA: Sage.
Dillman, Don A. (2000). Mail and Internet Surveys: The Tailored Design Method. Second Edition. New York: John Wiley & Sons, Inc.