measurement of attitudes

Measurement of Attitudes

Dr. K. A. KorbUniversity of Jos

Overview

A majority of Americans favor having Arabs, even those who are U.S. citizens, being subjected to separate, more intensive security procedures at airports. About half of Americans favor requiring Arabs, even those who are citizens of the United States, to carry special ID.

Gallup.org

Overview(Mike Frone) Please tell me if you would favor or oppose

each of the following as a means of preventing terrorist attacks in the United States. Requiring Arabs, including those who are U.S.

citizens, to undergo special, more intensive security checks before boarding airplanes in the U.S.

Requiring Arabs, including those who are U.S. citizens, to carry a special ID.

Problems Leading stem Double-barreled questions


Overview

Considerations in Measurement Types of Attitude Measurement

Explicit Likert Scale Interview

Implicit Implicit Association Test


Reliability: Consistency of results

Reliable Reliable Unreliable


Reliability Theory

Actual score on test = True score + Error True Score: Hypothetical actual score on

test The more reliable the test is, the less

error in measurement


Reliability: Sources of Error Error in Test Construction

Error in Item Sampling: Items measure more than one construct

Error in Test Administration Test environment Test-taker variables Examiner-related variables

Error in Test Scoring


Validity: Measuring what is supposed to be measured

Valid Invalid Invalid


Validity

Three types of validity necessary for all psychological tests: Construct validity: Measure the

appropriate psychological construct Criterion validity: Predict appropriate

outcomes Content validity: Adequate sample of

content


Construct Validity Definition: Appropriateness of inferences

drawn from test scores regarding an individual’s status of the psychological construct of interest

Two considerations: Construct underrepresentation Construct irrelevant variance


Construct Validity Construct underrepresentation: A

test does not measure all important aspects of the construct.

Construct-irrelevant variance: Test scores are affected by other unrelated processes


Sources of Construct Validity Evidence

Homogeneity: The test measures a single construct Evidence: Good reliability

Convergence: Test is related to other measures of the same construct and related constructs Evidence: Highly correlations with other

measures


Sources of Construct Validity Evidence

Theory: The test behaves according to theoretical propositions about the construct Evidence by changes in test scores

according to age: Scores on the measure should change by age as predicted by theory.

Evidence from treatments: Scores on the measure change as predicted by theory from a treatment between pretest and posttest.


Criterion Validity Definition: Correlation between the measure

and a criterion. Criterion: Other accepted measures of the construct

or measures of other constructs similar in nature. A criterion can consist of any standard with which

the test should be related Examples:

Behavior Judgments Other instruments related to attitudes


Criterion Validity

Three types: Convergent validity: High correlations

with measures of similar constructs taken at the same time.

Divergent validity: Low correlations with measures of different constructs taken at the same time.

Predictive validity: High correlation with a criterion in the future


Criterion Validity Example

Criterion Validity Evidence for Attitudes toward Coke

Correlation between Attitudes toward Coke and…

Attitude for Pepsi .83

Attitude for Mountain Dew .75

Attitude for Water .34

Attitude for Juice .24

High correlations with other minerals indicates good criterion validity.

Low correlations with unrelated beverages indicates good criterion validity.


Content Validity

Attitudes Toward Course:

Teacher

Content

Assignments

Exam

Definition: Sampling the entire domain of the construct

Attitudes Toward Course:

Teacher

Content

Assignments

Exam


Measuring Attitude Overview

Three Methods to Assess Attitudes Explicit (Self Report)

Questionnaire Interview

Implicit Behavioral Measures Physiological Response Reaction Time


Scaling of Attitudes Types of Scales

Semantic Differential Scales Reactions to stimulus on a bipolar scale

with opposite adjectives at the end Thurstone Scaling

Use expert judges to create an ordered list of attitudes toward a topic

Likert Scaling Evaluation of individual items according to

level of agreement or disagreement


Semantic Differential Scales

Tick your opinion of UniJos: Very Bad _:_:_:_:_:_:_ Very Good

Very Weak _:_:_:_:_:_:_ Very Strong

-3 0 +3

-3 0 +3


Thurstone Scale Tick if you agree with the item.

1. People with AIDS should be considered the lowest class of human beings.

2. People with AIDS must be kept apart in social affairs.

3. I am not interested in how people with AIDS rate socially.

4. A refusal to accept a person with AIDS is a prejudice which should be overcome.

5. I believe that people with AIDS deserve the same social privileges.


Likert Scaling Define the attitude to be measured Develop about 25 items thought to

measure the attitude Participants indicate the degree of agreement to

each item Single items work for very specific attitudes Multiple items have the best reliability and

validity for general attitudes (at least 10) Level of Measurement: 4, 5, 7, 10


Likert Scaling

ItemStrongly Agree

Agree Disagree Strongly Disagree

1. The class is beneficial.

1 2 3 4

2. The teacher is informative.

1 2 3 4

3. The assignments do NOT help me learn.

1 2 3 4

4. The classroom is comfortable.

1 2 3 4


Likert Scaling Factor Analyze the items to determine whether they

measure the attitude Factor Loading tells how well the item relates to the

overall score Interpret Factor Loadings as a correlation

Calculate the Reliability of the final items to establish internal consistency Coefficient Alpha > .70

Analyze Sum responses to each attitude construct separately Average responses to each attitude construct

separately


Likert Scaling

ItemFactor

Loading1. The class is beneficial. .96

2. The teacher is informative. .75


-.79

4. The classroom is comfortable. .34


Likert Scaling

Item Factor 1 Factor 21. The class is beneficial. .96 .222. The teacher is informative. .75 .393. The assignments do NOT help me learn.

-.79 -.10

4. The classroom is comfortable. .34 .785. The classroom is well lit. .17 .866. The classroom is too hot. -.24 -.73


Likert ScalingStrongly Agree


1. The class is beneficial.

1 2 3 4

2. The teacher is informative.

1 2 3 4


1 2 3 4

Summed Score = 2+3+1 = 6

Averaged Score = 2+3+1 = 6 6 3 = 2


Likert Scales Potential Problems

Acquiescence bias: Agree with statements as presented Solution: Develop equal numbers of positive

and negative items Reverse score negative items

Central tendency bias: Avoid extreme categories Strongly Agree, Strongly Disagree

Social desirability: Tendency to portray oneself in favorable light Solution: Bogus Pipeline


Acquiescence Bias

SA U SD1. 1 2 3 4 5 6 7

2. 1 2 3 4 5 6 7

3. 1 2 3 4 5 6 7

4. 1 2 3 4 5 6 7

5. 1 2 3 4 5 6 7

6. 1 2 3 4 5 6 7

7. 1 2 3 4 5 6 7

8. 1 2 3 4 5 6 7

9. 1 2 3 4 5 6 7

10. 1 2 3 4 5 6 7


Central Tendency Bias

SA U SD1. 1 2 3 4 5 6 7

2. 1 2 3 4 5 6 7

3. 1 2 3 4 5 6 7

4. 1 2 3 4 5 6 7

5. 1 2 3 4 5 6 7

6. 1 2 3 4 5 6 7

7. 1 2 3 4 5 6 7

8. 1 2 3 4 5 6 7

9. 1 2 3 4 5 6 7

10. 1 2 3 4 5 6 7


Social DesirabilityStrongly Agree


1. There have been times when I was quite jealous of others’ good fortune.

1 2 3 4

2. I never resent being asked to return a favor.

1 2 3 4

3. I have never intensely disliked anyone.

1 2 3 4


Interviews Define the purpose of the interview Select an interview format

Standardized Open-Ended Interview: Specific set of questions with open-ended response

Semi-structured: Structured questions followed by open-ended questions to probe deeper

Structured Interview: Participants select pre-determined responses


Standardized Open-Ended Interview

What issues do you think the Plateau State governor needs to spend more effort on?

How effective do you think the new road construction plans are?

How well do you think the public schools are being managed in Plateau State?


Semi-Structured Interview

What teacher has most influenced you? Why? She encouraged me to pursue my

dreams. How did she encourage you to pursue

your dreams?


Structured Interview How well do you think that Plateau State

governor compares to other Nigerian governors in providing funds for education? Better than most Nigerian governors Same as most Nigerian governors Worse than most Nigerian governors

How well do you think that Plateau State governor compares to other Nigerian governors in fixing the roads?

How well do you think that Plateau State governor compares to other Nigerian governors in providing jobs?


Interviews

Data Analysis Tally responses for Structured Interview Open-Ended questions require a

categorization system Cared for me Encouraged to excellence Knew the course material


Questionnaires vs. Interviews Questionnaire Advantages

Sample more participants Quicker Standardized Anonymous

Interview Advantages Probe deeper Adaptability Use with illiterate participants


Quality of Attitude Items(Trochim, 2006)

Is the question useful? Only ask questions that will be used for

analysis Are several questions needed?

Double-barreled question: Question has two parts Plateau State should give more money

to education and road upkeep. I do not like pounded yam because it

tastes slimy.



Will all respondents have the needed information to answer the question? Knowledge Vocabulary

Does the question need to be more specific?

Is the question biased? Question leads to a particular response



Will the respondent answer truthfully? Social Desirability Bias in Questioning

Can the question be misunderstood? Conclusion: Read each question as if

you were the respondent. How could a respondent misunderstand the question?


Implicit Measures Purpose: Measure unconscious

thoughts and feelings Assumptions

People may be unwilling to report their true attitudes

People may not know their attitudes Implicit Attitudes: Attitudes that

people are either unwilling or unable to report


Implicit Measures

Behavioral Measures Distance from another person

Physiological Measures Facial EMG: Measuring imperceptible

muscle movements in the face EEG: Brain wave patterns

Reaction Time Implicit Priming Procedure


Implicit Priming

Good Bad Male Female

Wonderful Awful Amos Martha

Pleasant Evil John Julie

Nice Wicked Barnabas Nancy

Lovely Terrible Joseph Sabina

Agreeable Poor Peter Anna


Implicit Priming

Good Bad

NiceAwfulPleasantLovely


Implicit Priming

GoodBoys

BadGirls

NiceMarthaPleasantJohn


Implicit Priming

GoodGirls

BadBoys

NiceMarthaPleasantJohn


Implicit Measures

Implicit Association Test https://implicit.harvard.edu/implicit/de

mo/selectatest.html Gender-Career Skin Tone Disability Weight Religion


Implicit Measures Race IAT Predicts:

Time spent speaking: .51 Smiling: .39 Social comments: .32

Potential Uses Research Diversity training

Misuses Decisions of placement


Explicit vs. Implicit Measures

Meta-analysis correlated attitudes to criterion measures Implicit measures were best predictors of

stereotyping and prejudice (r = .25 vs .13)

Explicit measures were best predictors for products and political preferences (r = .67 vs .41)


Choosing a Measure

Research Questions/Hypotheses Time and Resources Available Participants


Revision

What are the four major quality checks for measures?

How can attitudes be explicitly measured?

How can attitudes be implicitly measured


measurement of attitudes

Documents