Items & Scales

Overarching Theme: Write Understandable Items that Can Be

Answered Easily.

Warning…(Sheatsley, 1983, p. 200)

“Because questionnaires are usually written by educated persons who have a special interest in and understanding of the topic of their inquiry, and because these people usually consult with other educated and concerned persons, it is much more common for questionnaires to be overwritten, overcomplicated, and too demanding of the respondent than they are to be simpleminded, superficial, and not demanding enough”

Some Basic Principles (Dillman, 2000)

Principle 1: Choose Simple Over Specialized Words

Tired Honest Most Important Work Correct

Exhausted Candid Top Priority Employment Rectify

Principle 2: Choose as Few Words as Possible

Bad: Do you strongly favor, somewhat favor, somewhat oppose, strongly oppose, or have no opinion on whether advertisers should be required to have advertising aimed at children approved by a national board?

Better: Which of the following best describes your position on whether advertisers should be required to have advertising aimed at children approved by a national board?

Principle 3: Avoid Being Too Specific

Bad: How many books have you read for leisure in the last year?

Better: About how many books have you read for leisure during the past year?

A. None B. 1-2 C. 3-5 D. 6-10 E. 11 or more

Principle 4: Avoid Biased Questions or Responses

Bad: Which of the following do you feel is most responsible for recent outbreaks of violence in America’s schools?

A. Irresponsible parents B. School policies C. Television programs

A Less Biased and More Informative Way To Ask The Question

To what extent do you feel that the way children are raised by parents is responsible for recent outbreaks of violence in America’s schools?

A. Completely Responsible B. Mostly Responsible C. Somewhat Responsible D. Not At All Responsible

Other Tips

Distinguish Undecided from Neutral. One suggestion: Put No Opinion at the End.

To what extent do you agree with this statement: “Living in rural places is better for a person’s emotional health.”

A. Strongly Agree B. Somewhat Agree C. Neither Agree Nor Disagree D. Somewhat Disagree E. Strongly Disagree F. No Opinion

Avoid Asking Respondents to Say “Yes” to Really Mean “No”

Bad: Do you favor or oppose not allowing the state to raise taxes without approval of 60% of the voters?

A. Favor B. Oppose Better: Do you favor or oppose requiring 60%

approval by voters in order to raise state taxes? A. Favor requiring 60% approval B. Oppose requiring 60% approval

A similar Problem

Please tell me whether you agree or disagree with the following statement about high school teachers in the public schools.

Teachers should not be required to supervise students in the halls, the lunchroom, and school parking lots.

Agree or Disagree?

Avoid double-barreled questions

Bad: Should the city build a new swimming pool that includes lanes for swimming laps that is not enclosed for winter use?

Better: Should the city build a new swimming pool? Should the pool include lanes for swimming laps? Should the pool be enclosed for winter use?

Moral of the Story: Make it Easy on Participants.

Some Problems with Items (Gorsuch, 1997)

1. Items have lower reliabilities than scales. If you have a 20 item scale with an internal consistency of .80 and work backward then the reliability of 1-item is .17.

2. Items often contain a good deal of confounding variance. “Almost every item of every scale has a word or phrase that may be interpreted by someone in an idiosyncratic manner”

What to do? Use scales: Aggregate across items.

Summated Rating Scales

This format is often called a “Likert-type” scale. This format is very common in Psychology (items on 5- or 7-Point Scales)

Must have Multiple Items. Sum or Average the responses to form a

measurement scale. The original method proposed by Likert has been

slightly distorted over time.

Domain Sampling

Imagine A Population of All Items Relevant to Our Construct

Dimensions We Assess

Domain sampling: Should try to asks items that represent each subcomponent of the construct.

Issue – Acquiescent Response Style

General Tendency to Agree to Statements Regardless of Content.

Solution: Included Reverse Scored Items. Strive for Balanced Scales.

Other Types of Scales

Graphic Rating Scales

• Person indicates how much by putting a mark at the appropriate point on a line that runs from one extreme to the other

• The person scales “how much” for his or herself

Guttman cumulative scales

Should agree up to a point, then disagree with all remaining options

Usually a small set of unidimensional items with increasing complexity.

Example

Attitudes about spanking children(Monette et al., 1994)

• Slapping a child’s hand is an appropriate way to teach the meaning of “no!”

• Spanking is sometimes necessary• Sometimes discipline requires using a belt

or paddle.• Some children need a good beating to

help keep them in line.

Other scales

Thurstone scales Semantic differential scales

– Know in book