Constructing the questionnaire

STAT-472SURVEY DESIGN

AGREE/DISAGREE, RANKINGS OR RATINGS? Survey researchers who measure

values and attitudes have two issues about the responses offered

1) Should questionnaire items make a statement and ask respondents whether they agree or disagree with it, or should it offer respondents specific alternatives?

2) Should the questionnaire include a set of items and ask respondents to rate them (e.g., approve, disapprove), or should it give them a list of items and force them to rank order items (e.g., from most favored to least favored)?

It is best to offer respondents explicit alternativesFor example, Men are better suited to work. Agree / Disagree Less educated respondents are more likely to agree

instead ask, Do you think men are better suited, women are better suited, or both are equally suited to work? (forced-choice)

forced-choice alternatives encourage thought and avoid the response bias

Researchers create bias if question wording gives respondents a reason for choosing one alternative

Example: Do you support or opposed a law on energy conservation?

The results changed when respondents heard,

Do you support the law or do you oppose it because the law would be difficult to enforce?

instead of simply,

Do you support or oppose the law?

It is better to ask respondents to choose among alternatives by ranking instead of rating items along an imaginary continuum.

Respondents can rate several items equally high, but will place them in a hierarchy if asked to rank them.

Schartz and Knauper et al suggest: That researchers attach positive and negative numbers

to response categories (i.e., -5 to 5) only when they conceptualize a variable in bipolar term

Example: How do you feel about defense spending?

And Use positive numbers (e.g., 0 to 10) when they

conceptualize it along a single continuum

Example: How successful would you say you are?