Research Methods in Social Relations

Professor Mike Gallivan

Georgia State University

Atlanta, Georgia, USA

Class 5: June 22, 2009

Overview of Class 5 today Remind students about tomorrow’s assigned reading We will discuss some material that I did not have time

to present last Thursday / Friday Different modes of data collection Scale uni-dimensionality vs. multi-dimensionality

New material for today: Multiple regression analysis

select only certain groups “dummy coding” for nominal variables

Different types of measurement scales summative (Likert) scales are most popular also Differential, Guttman and Semantic differential

Explain how qualitative research process differs

Date Readings in 6th ed.

red book

Readings in 7th ed.

green book

Special comments

       

June 22

7: 146-169(omit 7: pp. 159-162)  

7: 158-177(omit 7: pp. 167-170) 

Finish data analysis team project. Focus on Likert scales (Semantic, differential and Guttman scales are used less)

June 23

6: 128-142article about response rate on website You can omit:p. 360 (formulas)p. 369-370 (Adv. CFA and SEM)

8: 181-195article about response rates on website. You can omit:p. 360 (formulas)p. 369-370 (Adv. CFA and SEM)

Paper from JAIS by Sivo, Saunders et al. (2006). Discuss issues related to response rate. Student will read and critique a study about its response rate

June 24

19: 463-47619: 457-460 (repeat)

20: 521-53619: 483-491 

Writing academic research papers (skip section about meta-analysis in 7th ed.)

Questions you asked about U.S.A. What is your favorite NBA sports team?

My hometown is Boston So, my favorite team is Celtics!

Are all American people good at dancing? No, definitely not! Many Americans are very bad dancers!

Review of last week’s classes Please take 1 minute to write one idea or

lesson you learned Thursday or Friday that you will remember when course is finished!

Review of Discriminant Validity in SPSSExploratory factor analysis (EFA)

This shows which items go together In this method, items that load onto the same

factor can prove convergent validity In this method, items that load onto separate

factors can prove discriminant validity Convergent + discriminant validity = construct validity

Performing EFA using SPSS Procedure is called “Data reduction – Factor” Results are easier to understand if you do:

select “Options” and then “sorted by size” also “suppress values with absolute value < 0.40 also select Rotation: “Varimax” or “Direct oblimin” difficult to interpret if construct is multi-dimensional

Review of Multi-dimensionalityMulti-dimensional vs. uni-dimensional

Some constructs have > 1 dimensionFamily’s socio-economic status

Amount of salary income (from formal job) Amount of investment income (from bank) Amount of other income (from gambling, eBay) Level of parents’ education

Individual creative style construct Originality (number of original ideas generated) Rule-conformity (willing to follow rules) Efficiency (perform tasks in time-efficient way)

Multi-dimensional constructs in data set Section #9 – Psychological and physical strain Items about psychological strain factor separately

Topics we did not cover in Class 3Summary of Contents from Ch. 10

(Ch. 5 in green book, pp. 96-110)

Modes of direct questioning Paper-and-pencil questionnaire Face-to-face (in-person) interviews Telephone interviews Web-based surveys on the Internet Experience sampling (“diary methods”)

Other modes of measurement (indirect) Collateral data (another person also gives data)

Observing others (it may be unobtrusive) We will not discuss these methods in detail

Topics we did not cover in Class 3Summary of Contents from Ch. 10

(Ch. 5 in green book, pp. 96-110)

Paper-and-pencil questionnaire Advantages

Authors claim that paper surveys are inexpensive Minimize interviewer (researcher) bias Less time pressure for subjects Subject has feeling of being anonymous

Disadvantages Subjects cannot ask a question if they are confused Often a very low response rate (e.g., < 10%) You may not know how non-respondents are similar to or different

from the survey respondents Perhaps the respondents are 60% female, 40% male, but the non-

respondents are 30% female, 70% male

Topics we did not cover in Class 3Summary of Contents from Ch. 10

(Chapter 5 in green book) Face-to-face (in-person) interviews

Advantages Can “probe” the subject further for more information Subject can ask the researcher for clarification This may create a higher quality of information Can provide visual aids (maps, photos, examples)

Disadvantages Potential for large interviewer effect (bias) Especially for sensitive or personal questions May not want to give an honest answer to question

“Have you ever had an unwanted pregnancy?” This is called social desirability bias

Some subjects might want to impress the interviewer “How much alcohol do you normally drink in 1 week?”

Both of these effects might hurt the information quality

Data collection modes Topics we did not cover in Class 3Summary of Contents from Ch. 10

(Chapter 5 in green book) Telephone interviews Advantages

Combines advantages of both surveys + interviews Can probe subjects for further information Cost is low, compared to face-to-face interviews

Disadvantages More expensive than paper-and-pencil surveys Chance for response bias

People with phones might differ from those without Famous example in American presidency in 1960s Especially for sensitive or personal questions

If you rely on a telephone directory In U.S., telephone directories do not list mobile phones Young people only have mobile phones (no land phone) So, you might have an age bias effect in your results The solution to this problem is random-digit dialing

Data collection modes Topics we did not cover in Class 3Summary of Contents from Ch. 10

Web-based surveys on Internet Advantages

1: Avoid the problem of transcript errors 2: Very, very cheap to do 3 Easy to conduct 4: No effort required to mail survey back 5: More voluntary than a face-to-face 6 Faster, easier and “more fun” for subjects

Disadvantages Response rates are often very low (e.g., <1%) How many subjects did you send survey to? This information is required to compute response rate Not a good idea to just post survey to a website unless you know how many

total people will see the survey

Experience sampling (“diary methods”) Can use a pager to remind subjects to record data These methods can be used in medical research

Summary of Contents from Chapter 3: pp: 53-66

(Ch. 4 in green book, pp. 85-90 and Ch. 7: 152-158)

Single-item vs. multiple item scales Single-item is acceptable sometimes

Some examples of single-item measures How old are you in years?

Do we need to also ask “how old are you in months?” What is your sex? (male or female) What is your major? What is the city where you were born?

Advantages to multiple-item scales multiple items required to calculate internal reliability journal reviewers prefer multiple-item scales you can compute amount of measurement error

measurement error = 1 – Cronbach a

Summary of Contents from Chapter 7: pp: 146-169

(Ch. 7 in green book, pp. 158-177)

Different types of rating scales Graphic rating scales Itemized rating scales Comparative rating scales

Source of the ratings Self-ratings (individual)

Often the individual is best judge of their own opinions

Parent ratings of child’s behavior This is common in medical surveys for young children

Peer or co-worker or supervisor ratings Sometimes more accurate data from another person

Summary of Contents from Chapter 7: pp: 146-169

(Ch. 7 in green book, pp. 158-177

Problems with all measurement Social desirability bias – try to give answer subject

thinks the interviewer wants to receive “Do you ever drink > 3 drinks at dinner or party?” “Have you had premarital sex?” The direction of bias effect might differ depending:

If the researcher is a peer member (e.g., same age) If the researcher is much older or a medical doctor

Sometimes the wording creates bias

My doctor asked me last month: “You always wear sunscreen, don’t you?” What would be a better way to ask question?

Summary of Contents from Chapter 7: pp: 146-169

(Ch. 7 in green book, pp. 158-177

Remember definition of measurement in Class 2: Any measurement item is comprised of 3 parts:

The construct of interest (general intelligence) Other constructs not of interest (English language skill) Random measurement error (mistakes in answer)

Observed score (e.g., each survey item) = True score + systematic error + random error Variable of interest + variable not of interest + random error

Multiple item scales help calculate random error Halo bias (generosity error) is a systematic error

Example of personal attractiveness (an attractive person is often considered smarter, more competent, trusted)

Example of website attractiveness (a website that is more visually attractive is often trusted more by users)

Summary of Contents from Chapter 7: pp: 146-169

(Ch. 7 in green book, pp. 158-177

Developing multiple-item scales Domain sampling

start with very broad range of items have some positive-worded items

“I like Presidential candidate Obama”

also have some negative-worded items “I don’t trust Presidential candidate Obama”

Process of pilot testing the survey Small sample to identify wording problems Modify or delete confusing items Then test with a much larger sample of subjects

Example of Survey Development: Personal Innovativeness with IT

Initial survey items:

Measuring multi-item scales Examine Cronbach a (internal reliability)

Delete any items that reduce a < 0.70 For all items retained in the final scale:

Average the items together

Examine Exploratory Factor Analysis This is a more complex method

Some researchers prefer this method of “factor scores” Be sure to select “Options”

Missing values: exclude cases “pairwise” “Suppress absolute values < 0.40”

Then choose “Scores” and “Save as Variables” This allows each item to have a different weight

(e.g., item with 0.84 factor loading will have twice as much weight as another item with 0.42 factor loading)

Example of multi-item scales using Psychological & Physical Strain items

The next 2 slides show items from the survey

Section #9: During the past week…

(Circle one number per item)

 

1. I was bothered by things that usually don't bother me.

 

2. I did not feel like eating; my appetite was poor.[1]

Rarely or None of the Time

(Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

 

3. I felt that I could not shake off the blues even with help from my family or friends.[1]

Rarely or None of the Time

(Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

 

4. I had trouble keeping my mind on what I was doing.

[1]Rarely or None of the

Time (Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

[1]Rarely or None of the

Time (Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

[1]Rarely or None of the

Time (Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

5. I felt depressed.

[1]Rarely or None of the

Time (Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

6. I felt I that everything I did was an effort.

[1]Rarely or None of the

Time (Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days) 

9. I talked less than usual.[1]

Rarely or None of the Time

(Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

 

10. I felt lonely.[1]

Rarely or None of the Time

(Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

 

11. I had crying spells.[1]

Rarely or None of the Time

(Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

 

12. I felt sad.[1]

Rarely or None of the Time

(Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

13. I could not “get going.”[1]

Rarely or None of the Time

(Less than 1 Day)

[2]Some of the Time

(1-2 Days)

[3]Occasionally or a

Moderate Amount of Time(3-4 Days)

[4]Most or All of the Time

(5-7 Days)

8. My sleep was restless.

Results from Factor AnalysisThis is first result table showing the number of factors

Initial Eigen-values

Compo-nent

Total % of Variance

Cumulative %

1 5.931 45.627 45.627

2 1.087 8.359 53.986

3 1.026 7.892 61.878

4 .905 6.963 68.841

5 .703 5.405 74.246

6 .676 5.200 79.447

Exploratory Factor Analysis resultsThis is a very messy 3-factor resultMost items load on Factor 1 and 2

A few items load separately on Factor 3

Rotated Component Matrixa

.814

.812

.736

.661 .426

.658

.529 .522

.501 .477

.885

.512 .638

.509

.744

.716

.449 .534

WEK_MIND

WEK_NOGO

WEK_EFRT

WEK_DEPR

WEK_CRY

WEK_BLUS

WEK_LONE

WEK_SLEP

WEK_SAD

WEK_BOTH

WEK_EAT

WEK_FEAR

WEK_TALK

1 2 3

Component

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Rotation converged in 5 iterations.a.

Review Data Analysis AssignmentWhich surveys items are Nominal?

Other names for “nominal” items Categorical or dichotomous (if just 2 categories) Chi-squared (“crosstabs”) analysis that Dr. Doug

Rice had you do Friday was a dichotomous item Items? (section # and item #)

Which surveys items are Ordinal? Items? (section # and item #)