Research Design Overview

Heather M. Gray, Ph.D.January 26, 2010

Research Methods for the Social Sciences: An Introductory Course

Today’s Plan

• Lecture: The 5 basic types of research approaches in the social sciences

• Interactive exercise #1: Does watching too much TV kill you?

• Interactive exercise #2: Name that Method (time permitting)

Primary Sources

• Rosenthal, R. & Rosnow, R. L. (1991). Essentials of Behavioral Research: Methods and Data Analysis

• Research Methods: The Laboratory http://www.mcli.dist.maricopa.edu/proj/res_meth/login.html

5 Basic Approaches

Experiments Everything Else

Correlation NaturalisticObservation

Survey CaseStudy

Experiments

• Description: • Manipulating a variable to see if it

changes a second variable• Independent variable• Dependent variable

X Y

Experiments

• Strengths: Cause-and-effect relationships, IF:• Statistical conclusion validity• Construct validity• Internal validity• External validity

Experiments

• Limitations:• Necessary trade-offs• “Truth accrues, error cancels out”

(attributed to Robert Rosenthal)

Rigor Relevance

Experiments

• Limitations:• Not always practical• Not always ethical

5 Basic Approaches

Experiments Everything Else

Correlation NaturalisticObservation

Survey CaseStudy

Correlation

• Description: • No manipulation• Designed to determine degree and

direction of relationship between two variables

X Y

Correlation

• Strength: • No manipulation• Useful for prediction

Correlation

• Limitations: • Correlation cannot prove causation

5 Basic Approaches

Experiments Everything Else

Correlation NaturalisticObservation

Survey CaseStudy

Naturalistic Observation

• Description:• Careful observation and recording of

some behavior or phenomenon• Over a prolonged time• In its natural setting• Without interference

Naturalistic Observation

• Strengths:• Observation of behavior as it occurs

naturally• Limited opportunity for experimenter

effects• Can yield hypotheses for future

experimental investigation

Naturalistic Observation

• Limitations:• Descriptive method, not explanatory• Time intensive• Difficulty of observing behavior without

disrupting it• Coding issues

5 Basic Approaches

Experiments Everything Else

Correlation NaturalisticObservation

Survey CaseStudy

Survey

• Description:• Descriptive study• Inferences drawn from interviews or

questionnaires• Different types of questions

• Forced-choice• Open-ended

Survey

• Strength:• When constructs are difficult to

observe directly• Standardization• Cost-effective• Relatively quick

Survey

• Limitations:• No cause-and-effect conclusions• Reliance on self-reports

• Deception• Poor memory• Misunderstanding of question• Lack of insight• Predicting behavior?

5 Basic Approaches

Experiments Everything Else

Correlation NaturalisticObservation

Survey CaseStudy

Case Study

• Description:• In-depth descriptive record of an

individual or small group of individuals• Biographical data, medical records,

family history, observations, interviews, psychological tests

Case Study

• Strengths:• Provide detailed, contextual view • If long term, helpful in understanding

developmental issues • Helpful in generating hypotheses for

future testing

Case Study

• Limitations:• Not explanatory; no cause-and-effect

relationships• Behavior can be observed but not

explained• Lack of generalizability• Issues re: retrospective data

Take-home point #1

• On choosing your own research method• Don’t be limited to just one approach• Use combination of approaches to help

correct for inherent weaknesses

Take-home point #2

• On being an educated consumer of research• Are conclusions warranted by the study

design?• Example…

Interactive Exercise #1

• How else could we have addressed this question?• Naturalistic observation• Survey• Case study• Experiment?

Interactive Exercise #2

• Name that Method