Design and Control in Experimental Research

Lawrence R. Gordon

Psychology Research Methods I

Remind me again…

Why am I taking this class???

Academic Skills in Psychology Majors Kruger & Zechmeister (2001) Interested in seeing the skills that

psychology majors develop during college Graduating senior and first-year psychology

majors completed a scale measuring 10 skill areas

Academic Skills

Written/oral communication

Information gathering Group/team work Interpersonal Behavior management

Diversity Critical thinking Methods and statistics Ethics and values Technology and

computers

Review

Independent variable?– What type is it?

Dependent variable? What type of research is this? Cause-effect conclusions?

Results

Seniors reported significantly more skill in 7 of the 10 academic skill areas than did the first-years

No differences in group/team work, behavior management, or diversity

Look at the graph...

Mean Scores in Each Skill Area

0

1

2

3

4

5

6

7W

ritt

en

Info

Gro

up

Inte

rper

s

Beh

avio

r

Div

ersi

ty

Cri

tica

l

Met

h&

Sta

t

Eth

ics

Tec

h

1st YearsSeniors

Academic Skill Gains?

How else could you measure skill gains in psychology majors during college from first-year to senior year?

P.S. They also compared Psych majors to majors in natural sciences, social sciences, & humanities. Psych majors had greater exposure in five areas & were most similar to natural science majors.

Research Designs

Between-subjects vs. Within-subjects Between-subjects

– participants participate in only one condition of the independent variable(s)

– their results are compared to other groups in other conditions of the independent variable

– another group serves as the control

EXAMPLE: Memory Experiment?

Why do we use Between-subjects designs?

Often, independent variables are subject variables, and it is impossible to be in more than one level

With some manipulated IVs, it is not appropriate to participate in additional conditions after having been in one

May sometimes reduce session time

Problems with Between-subjects designs

Will need more participants Control groups and experimental groups

contain different people– To deal with this, create equivalent groups

• Random assignment (also block)

• Matching

Research Designs (cont.)

Within-subjects– participants serve in all conditions of the

independent variable(s)– their results in one condition are compared to

their results in other conditions of the independent variable

– hence, participants serve as their own control

EXAMPLE: Memory Experiment?

Why use Within-subjects designs?

Procedure takes big prep time, but testing is short (E.g., S & P)

Participants are hard to come by (small population)

Need fewer participants

Problems with Within-subjects designs

Sequence or Order effects– Progressive effects (practice, fatigue)– To deal with this

• Once per S: Counterbalancing: complete or partial

• Two+ per S: Reverse counterbalancing, block randomization

– Carryover effects– To deal with this difficult -- test or do btwn-Ss

May increase research time

SPECIAL CASE:Cross-sectional vs. Longitudinal Often want to investigate DVs that you

expect may change over time Two ways to do this:

1. Measure different groups of people at one time (b/w-Ss, cross-sectional)

2. Measure the same group of people at each time (w/in-Ss, longitudinal)

– Example

Treatment of Juvenile Offenders

Why do juveniles act out through crime? Borduin, et al. (1995) Effects of multisystemic therapy (MST) on

prevention of criminal activity in juvenile offenders

Participants were juvenile offenders ages 12-17 in Missouri

Treatment focused on cognitive and systemic factors associated with offending

Treatment of Juvenile Offenders (cont.) Measured

– Individual adjustment (pre-post)• psychiatric symptomatology

• behavior problems

– Family relations (pre-post)– Peer relations (pre-post)– Criminal activity (4 years post)

Treatment of Juvenile Offenders (cont.) Results for individuals

– decrease in psychiatric symptomatology and in behavior problems

– positive changes in family relation on both scale and observation

– no differences in peer relations

Treatment of Juvenile Offenders (cont.) Results for criminal activity

– lower risks for arrest (see graph)– arrested less often– arrested for less serious offenses– less likely to be arrested for violent crimes (e.g.

rape, assault and battery)

How about a picture?

MST vs. IT

0

10

20

30

40

50

60

70

80

90

100

0 200 400 600 800 1000 1200 1400 1600 1800

Days Survived

Su

rviv

al P

rob

abil

ity

MSTIT

MST vs. IT (w/ dropouts)

0

10

20

30

40

50

60

70

80

90

100

0 200 400 600 800 1000 1200 1400 1600 1800

Days Survived

Su

rviv

al P

rob

abil

ity

MST fullMST dropIT fullIT dropRefusers

Problems with longitudinal research?Threats to Validity, again...

For longitudinal studies

1. History

2. Testing

3. Instrumentation

4. Regression to the mean

5. Attrition

6. Maturation

7. Participant selection

Threats to Validity for All Designs

Experimenter bias– Clever Hans example

– To deal with this• Use strict (even automated) procedure

• Use double-blind procedures

• Use multiple experimenters

Threats to Validity for All Designs (cont.)

Participant bias (often from demand characteristics)1. Cooperative participants

• being a “good subject”

2. Defensive participants

3. Uncooperative participants• “screw you effect”

Threats to Validity for All Designs (cont.)

To deal with this• Sometimes appropriate to use mild deception

• Placebo control group where possible

• Manipulation checks

• Field research

One final study

Appreciating difficult methods

Shotland & Heinfold (1985) Bystander response to arterial bleeding What did they do? How did they convince people that the

victim needed help? Did the people believe the scenario?