Experimental Research Designs

Experimental Design

AdvantagesBest establishes cause-and-effect

relationships Disadvantages

Artificiality of experimentsFeasibilityUnethical

Causality

Temporal precedence Covariation between IV and DV Eliminate alternative explanations

Types of Experimental Designs

Simple True Experimental Complex True Experimental Quasi-Experimental

Types of Experimental Designs

Simple True Experimental Complex True Experimental Quasi-Experimental

Simple True Experimental

Characteristics Types Variations

Characteristics of True Designs

Manipulation (treatment) Randomization Control group

Characteristics of simple true designs One IV with 2 levels (T, C) One DV

Types

Randomized posttest control group design Randomized pretest-posttest control group

design

Randomized posttest control group design

R T PostR C Post

Randomized pretest-posttest control group design

R Pre T PostR Pre C Post

Advantages & Disadvantages

Advantages of pretest designEquivalency of groupsCan measure extent of changeDetermine inclusionAssess reasons for and effects of mortality

Disadvantages of pretest designTime-consumingSensitization to pre-test

Solomon four-group design

R Pre T PostR Pre C PostR T PostR C Post

Variations

Independent groups (between groups) Repeated measures (within groups)

Repeated Measures Design

Advantages:Fewer subjects needed (less costly)Sensitive to finding statistical differences

Disadvantages:Order effect (practice, fatigue, carry-over)

Dealing with Order Effects

Counterbalancingn!Latin squares

Latin Squares

1 2 3 4

Row 1A

(60)B(0)

D(120)

C(180)

Row 2B(0)

C(180)

A(60)

D(120)

Row 3C

(180)D

(120)B(0)

A(60)

Row 4D

(120)A

(60)C

(180)B(0)

Dealing with Order Effects

Counterbalancingn!Latin squaresRandomized blocks

Time interval between treatments

Variations

Independent groups (between) vs. repeated measures (within) designs

Consider external validity when deciding which design to use.

Types of Experimental Designs

Simple True Experimental Complex True Experimental Quasi-Experimental Pre-Experimental

Characteristics of True Designs

Manipulation (treatment) Randomization Control group

Characteristics of simple true designs One IV with 2 levels (T, C) One DV

Complex True Experimental

Randomized matched control group design

Increased levels of IV Factorial design Multiple DVs

Complex True Experimental

Randomized matched control group design

Increased levels of IV Factorial design Multiple DVs

Randomized matched control group design

M R T PostM R C Post

• Used in small samples cost in time & money

Complex True Experimental

Randomized matched control group design

Increased levels of IV Factorial design Multiple DVs

Increased Levels of IV

Provides more complete information about the relationship between the IV & DV

Detects curvilinear relationships Examines effects of multiple treatments

Per

form

ance

leve

l (%

com

plet

e)

Amount of reward promised ($)

DV

IV

$0 $1 $2 $3Reward Amount

Increased Levels of IVP

erfo

rman

ce le

vel (

% c

ompl

ete)

Amount of reward promised ($)

DV

IV

Complex True Experimental

Randomized matched control group design

Increased levels of IV Factorial design Multiple DVs

Factorial Design

>1 IV (factor) Simultaneously determine effects of 2 or

more factors on the DV (real world) Between Factor vs. Within Factor ID’d by # of factors and levels of factors

2X2

Do differing exercise regimens (hi, med, lo intensity) have the same effect on men as they do on women?

3 X 2 (Exercise Regimen X Gender)2 factorsExercise Regimen – 3 levelsGender – 2 levelsBetween factorsDV?Experimental IVs or Participant IVs?

Male FemaleHighMediumLowE

xerc

ise

Inte

nsity

Gender

Do strength gains occur at the same rate in men as they do in women over a 6 mo. training period? Measurements are taken at 0, 2, 4, 6 mo.

2 X 4 (Gender X Time)? factorsTime – 4 levelsGender – 2 levelsBetween or within factors?DV?Experimental IVs or Participant IVs?

0 mo. 2 mo. 4 mo. 6 mo.

Male

FemaleGen

der

Time

0 mo. 2 mo. 4 mo. 6 mo.

Male 50 70 90 130 85

Female 30 60 75 90 64

40 65 83 110 74

Gen

der

Time

Cell means, Margin meansMain Effects, Interactions

Cell meansMargin means Grand mean

0 mo. 2 mo. 4 mo. 6 mo.

Male 50 70 90 130 85

Female 30 60 75 90 64

40 65 83 110 74

Gen

der

Time

Parallel lines indicate no interaction.

Interaction of Exercise Intensity and Gender

35

40

45

50

55

60

65

70

High Medium Low

Exercise Intensity

VO2

Max

(ml/k

g/m

in)

MaleFemale

Is there a main effect?

Interaction of Exercise Intensity and Gender

35

40

45

50

55

60

65

70

Male Female

Gender

VO2

Max

(ml/k

g/m

in)

HighMediumLow

Is there a main effect?

Non-parallel lines indicate an interaction.

Interaction of Gender and Time

0

20

40

60

80

100

120

140

0 mo. 2 mo. 4 mo. 6 mo.

Time

Wei

ght L

ifted

(lbs

.)

MaleFemale

Is there a main effect?

Interaction Between Gender and Time

0

20

40

60

80

100

120

140

Male Female

Gender

Wei

ght L

ifted

(lbs

.)

0 mo.2 mo.4 mo.6 mo.

Is there a main effect?

Interpretation

Always interpret the interaction first (graphical)

If no significant interaction, interpret main effects

Advantages of factorial designs:Greater protection against Type I errorMore efficient Can examine the interaction

Disadvantages: subject # for between factor designs

Consider external validity when deciding which design to use.

IV A: Exposure to Violence – violent vs. nonviolent videoIV B: Gender – male vs. femaleDV: # ads recalled (0-8)

1 21 1 5 3

2 9 5 7

5 5

A

B

1

5

9

1 2

A

B1

B2

IV A: Exposure to Violence – violent vs. nonviolent videoIV B: Gender – male vs. femaleDV: # ads recalled (0-8)

1

5

9

1 2

A

B1

B2

A: YesB: NoAxB: Yes

Complex True Experimental

Randomized matched control group design

Increased levels of IV Factorial design Multiple DVs

0 mo. 2 mo. 4 mo. 6 mo.

Male 50 70 90 130 85

Female 30 60 75 90 64

40 65 83 110 74

Gen

der

Time

Do strength gains occur at the same rate in men as they do in women over a 6 mo. training period? Measurements are taken at 0, 2, 4, 6 mo.

Types of Experimental Designs

Simple True Experimental Complex True Experimental Quasi-Experimental

Characteristics of True Designs

Manipulation (treatment) Randomization Control group

Less control More real-world Program evaluation

Randomized posttest control group design

R T PostR C Post

Randomized pretest-posttest control group design

R Pre T PostR Pre C Post

Quasi-experimental Designs One group posttest-only design One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Quasi-experimental Designs One group posttest-only design One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Randomized posttest control group design

R T PostR C Post

One group posttest-only design (One shot study)

T Post

No control of IV threatsUse?

Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Randomized pretest-posttest control group design

R Pre T PostR Pre C Post

One group pretest-posttest design

Pre T Post

•History•Maturation•Testing•Instrument decay•Regression

Use control group

Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Randomized posttest control group design

R T PostR C Post

Non-equivalent control group design

(Static group comparison design)T PostC Post

•Selection bias

Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Randomized pretest-posttest control group design

R Pre T PostR Pre C Post

Non-equivalent control group pretest-posttest design

Pre T PostPre C Post

•Can check selection bias

Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Time series

Pre Pre Pre Pre T Post Post Post Post

Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest

design Time series Single subject designs (Case study) Developmental designs

Developmental Research Designs

Longitudinal

Powerful (within subject)

Time consuming Attrition Testing effect

Cross Sectional

Less time consuming Cohorts problem

Choosing a Research Design

Best addresses the problem Ethics Cost in time and money Validity (internal & external)

Experimental design

5 m

Subambient CO2 tunnel Elevated CO2 tunnel

250 ppm

500 ppm370 ppm

370 ppm