Download - Design and Control in Experimental Research Lawrence R. Gordon Psychology Research Methods I
Design and Control in Experimental Research
Lawrence R. Gordon
Psychology Research Methods I
Remind me again…
Why am I taking this class???
Academic Skills in Psychology Majors Kruger & Zechmeister (2001) Interested in seeing the skills that
psychology majors develop during college Graduating senior and first-year psychology
majors completed a scale measuring 10 skill areas
Academic Skills
Written/oral communication
Information gathering Group/team work Interpersonal Behavior management
Diversity Critical thinking Methods and statistics Ethics and values Technology and
computers
Review
Independent variable?– What type is it?
Dependent variable? What type of research is this? Cause-effect conclusions?
Results
Seniors reported significantly more skill in 7 of the 10 academic skill areas than did the first-years
No differences in group/team work, behavior management, or diversity
Look at the graph...
Mean Scores in Each Skill Area
0
1
2
3
4
5
6
7W
ritt
en
Info
Gro
up
Inte
rper
s
Beh
avio
r
Div
ersi
ty
Cri
tica
l
Met
h&
Sta
t
Eth
ics
Tec
h
1st YearsSeniors
Academic Skill Gains?
How else could you measure skill gains in psychology majors during college from first-year to senior year?
P.S. They also compared Psych majors to majors in natural sciences, social sciences, & humanities. Psych majors had greater exposure in five areas & were most similar to natural science majors.
Research Designs
Between-subjects vs. Within-subjects Between-subjects
– participants participate in only one condition of the independent variable(s)
– their results are compared to other groups in other conditions of the independent variable
– another group serves as the control
EXAMPLE: Memory Experiment?
Why do we use Between-subjects designs?
Often, independent variables are subject variables, and it is impossible to be in more than one level
With some manipulated IVs, it is not appropriate to participate in additional conditions after having been in one
May sometimes reduce session time
Problems with Between-subjects designs
Will need more participants Control groups and experimental groups
contain different people– To deal with this, create equivalent groups
• Random assignment (also block)
• Matching
Research Designs (cont.)
Within-subjects– participants serve in all conditions of the
independent variable(s)– their results in one condition are compared to
their results in other conditions of the independent variable
– hence, participants serve as their own control
EXAMPLE: Memory Experiment?
Why use Within-subjects designs?
Procedure takes big prep time, but testing is short (E.g., S & P)
Participants are hard to come by (small population)
Need fewer participants
Problems with Within-subjects designs
Sequence or Order effects– Progressive effects (practice, fatigue)– To deal with this
• Once per S: Counterbalancing: complete or partial
• Two+ per S: Reverse counterbalancing, block randomization
– Carryover effects– To deal with this difficult -- test or do btwn-Ss
May increase research time
SPECIAL CASE:Cross-sectional vs. Longitudinal Often want to investigate DVs that you
expect may change over time Two ways to do this:
1. Measure different groups of people at one time (b/w-Ss, cross-sectional)
2. Measure the same group of people at each time (w/in-Ss, longitudinal)
– Example
Treatment of Juvenile Offenders
Why do juveniles act out through crime? Borduin, et al. (1995) Effects of multisystemic therapy (MST) on
prevention of criminal activity in juvenile offenders
Participants were juvenile offenders ages 12-17 in Missouri
Treatment focused on cognitive and systemic factors associated with offending
Treatment of Juvenile Offenders (cont.) Measured
– Individual adjustment (pre-post)• psychiatric symptomatology
• behavior problems
– Family relations (pre-post)– Peer relations (pre-post)– Criminal activity (4 years post)
Treatment of Juvenile Offenders (cont.) Results for individuals
– decrease in psychiatric symptomatology and in behavior problems
– positive changes in family relation on both scale and observation
– no differences in peer relations
Treatment of Juvenile Offenders (cont.) Results for criminal activity
– lower risks for arrest (see graph)– arrested less often– arrested for less serious offenses– less likely to be arrested for violent crimes (e.g.
rape, assault and battery)
How about a picture?
MST vs. IT
0
10
20
30
40
50
60
70
80
90
100
0 200 400 600 800 1000 1200 1400 1600 1800
Days Survived
Su
rviv
al P
rob
abil
ity
MSTIT
MST vs. IT (w/ dropouts)
0
10
20
30
40
50
60
70
80
90
100
0 200 400 600 800 1000 1200 1400 1600 1800
Days Survived
Su
rviv
al P
rob
abil
ity
MST fullMST dropIT fullIT dropRefusers
Problems with longitudinal research?Threats to Validity, again...
For longitudinal studies
1. History
2. Testing
3. Instrumentation
4. Regression to the mean
5. Attrition
6. Maturation
7. Participant selection
Threats to Validity for All Designs
Experimenter bias– Clever Hans example
– To deal with this• Use strict (even automated) procedure
• Use double-blind procedures
• Use multiple experimenters
Threats to Validity for All Designs (cont.)
Participant bias (often from demand characteristics)1. Cooperative participants
• being a “good subject”
2. Defensive participants
3. Uncooperative participants• “screw you effect”
Threats to Validity for All Designs (cont.)
To deal with this• Sometimes appropriate to use mild deception
• Placebo control group where possible
• Manipulation checks
• Field research
One final study
Appreciating difficult methods
Shotland & Heinfold (1985) Bystander response to arterial bleeding What did they do? How did they convince people that the
victim needed help? Did the people believe the scenario?