Psych 231: Research Methods in Psychology

Announcements

I will aim to have the exams graded and entered into ReggieNet by Monday

Remember that piloting is happening in labs this week

Non-Experimental designs

Sometimes you just can’t (or don’t want to) perform a fully controlled experiment Because of the issue of interest Limited resources (not enough subjects, observations are too

costly, etc.).

There are other kinds of designs that may be used• Surveys

• Correlational studies

• Quasi-Experiments• Developmental designs

• Small-N designs

This does NOT imply that they are bad designs Just remember the advantages and disadvantages of each

Non-Experimental designs

Sometimes you just can’t (or don’t want to) perform a fully controlled experiment Because of the issue of interest Limited resources (not enough subjects, observations are too

costly, etc.).

There are other kinds of designs that may be used• Surveys

• Correlational studies

• Quasi-Experiments• Developmental designs

• Small-N designs

This does NOT imply that they are bad designs Just remember the advantages and disadvantages of each

Surveys

What are they? Questionnaires and interviews that ask people to

provide information about themselves Why conduct them?

Best way to collect some kinds of information:• Descriptive, behavioral, and preferential

• e.g., demographic information, recreational behavior, personal preferences, and attitudes

To compliment experimental work• Good/common first step, can collect a lot of data about a

lot of variables• Do not have to directly observe behaviors

http://www.people-press.org/

A good resource:

Check out the Methodology Pages

Surveys

Advantages One can investigate internal events

• e.g., attitudes & opinions

Can generalize about an entire population based on relatively small samples of individuals

Large amounts of data can be collected quickly with relatively little cost (effort, time, etc.)

• But they’re often not as “cheap” as you may think

Surveys

Disadvantages Correlational: causal claims shouldn’t be made

• Often used for descriptive and predictive research questions

• Note: surveys are sometimes used as part of an experimental design, which may allow testing of causal claims

Non-response bias• Why doesn’t everybody respond?• Does response rate interact with variables of interest?

Self-reports may not be truthful• Response set - tendency to respond from a particular

perspective• Social desirability bias (e.g., how a “moral” person would answer)

Large data sets are sometimes difficult to analyze

11 Stages of survey research

Stage 1) Identify the focus of the study and select your research method What are the objectives of the research? Is a survey method the best approach? What kind of survey should be used?

• Group administration• Mail surveys• Internet surveys• Telephone surveys• Face-to-face interviews• Focus group interviews

11 Stages of survey research

Stage 2) Determining the research schedule and budget

Stage 3) Establishing an information base Find out what’s been done, what’s known

• E.g., Find other related surveys

Stage 4) Identify the sampling frame The actual population that the sample is drawn

from (as opposed to the ideal population)• Think of it as operationalizing the conceptual level

population• Be aware of potential coverage error – when the

sampling doesn’t lead to a good representativeness

11 Stages of survey research

Stage 5) Determining the sample method and sampling size Review Probability and Non-Probability methods

• Voluntary response method

Importance of sample size

Voluntary response methods

A kind of convenience sampling methods commonly used

• Problem: Typically only individuals with strong opinions respond, so the results are often extremely biased (A coverage error issue? Non-response issue?)

Should leftover Halloween candy be given out to students who get an A on the exam?

Should leftover Halloween candy be given out to students who get an A on the exam?

Call 123-NONO if you think NOCall 123-4YES if you think YES

Daily show clip(5:30 mins)

Importance of sample size

Confidence intervals• An estimate of the mean or percentage of the population,

based on the sample data • “John Doe has 55% of the vote, with a margin of error ± 3%”

• Margin of error (that “± 3%” part)• The larger your sample size, the smaller your margin

of error will be. • Which would you be more likely to believe

• “We asked 10 people …”• “We asked 1000 people …”

Sampling error - how is the sample different from the population?

Often focus on this partBut this part is important too

Importance of sample size

Sampling error - how is the sample different from the population?

Response rate • What proportion of the sample actually responded to

the survey?• Hidden costs here - what can you do to increase

response rates

• Non-response error (bias)• Is there something special about the data that you’re

missing (From the people who didn’t respond)?

11 Stages of survey research

Stage 6) Designing the survey instrument Question construction: How the questions are

written is very important• Clearly identify the research objectives

• Do your questions really target those research objectives (think Internal and External Validity)?

• Take care wording of the questions• Keep it simple, don’t ask two things at once, avoid

loaded or biased questions, etc.• How should questions be answered (question type)?

Good and poor questions

GoodPoorWas the FDC negligent by ignoring the warnings about Vioxx during testingand approving it for sale?

a) Yesb) Noc) Unsure

Do you favor eliminating the wasteful excess in the public school budget?

a) Yesb) Noc) Unsure

If the FDC knew that Vioxx caused serious side effects during testing,what should it have done?

a)Ban it from ever being soldb)Require more testing before approving itc)Unsure

Do you favor reducing the public school budget?

a)Yesb)Noc)Unsure

Problem: emotionally charged words

Problem: emotionally charged words

Good and poor questions

GoodPoorShould senior citizens be given moremoney for recreation centers and food assistance programs?

a)Yesb)Noc)Unsure

Should senior citizens be given more money for recreation centers?

a)Yesb)Noc)Unsure

Should senior citizens be given more money for food assistance programs?

a)Yesb)Noc)Unsure

Problem: asks two different questions

Problem: asks two different questions

Good and poor questions

GoodPoorAre you against same sex marriage and in favor of a constitutional amendment to ban it?

a) Yesb) Noc) Unsure

What is your view on same sex marriage?

a) I think marriage is a matter of personal choice

b) I’m against it but don’t want a constitutional amendment

c) I want a constitutional amendment banning it

Problem: Biased inmore than one direction

Problem: Biased inmore than one direction

Problem: Asks two questions

Problem: Asks two questions

Survey Questions

Question types Open-ended (fill in the blank, short answer)

• Can get a lot of information, but• Coding is time intensive and potentially

ambiguous

Close-ended (pick best answer, pick all that apply)• Easier to code• Same response alternatives for everyone• Take care with your labels

• Decide what kind of scale

• Decide number/label of response alternatives

What is the best thing about ISU? (choose one)

1. Location 2. Academics 3. Dorm food 4. People who sell

things between Milner and the Bone

What is the best thing about ISU?

Decide what kind of rating scales• Rating: e.g., Likert scale

Survey Questions: Close-ended

PSY 231 is an important course in the major. 1 2 3 4 5 Strongly Agree Neutral Disagree Strongly Agree Disagree

• Semantic differential: Rate how you feel about PSY 231 on these dimensions

Important _____: _____: _____: _____: _____: Unimportant

Boring _____: _____: _____: _____: _____: Interesting

• Nonverbal scale for children:

Point to the face that shows how you feel about the toy.

Decide number/label of response alternatives• Use odd number (mid point and equal # of responses above

and below the mid point)• Questions should be uni-dimensional (each concerned with

only one thing)• Labels should be clear

Survey Questions: Close-ended

11 Stages of survey research

Stage 7) Pre-testing the survey instrument Fix what doesn’t seem to be working

Stage 8) Selecting and training interviewers For telephone and in-person surveys Need to avoid interviewer bias

Stage 9) Implementing the survey Stage 10) Coding and entering the data Stage 11) Analyzing the data and preparing a

final report