lecture 9: experimental methods
TRANSCRIPT
Lecture 9: Experimental Methods
Research Design
Gale M. Lucas, PhD
…or “How to run a study”
Schacter & Singer• Claim we don't automatically know when we are
happy, angry. Instead, we label our emotions by considering situational cues. This labeling process depends on two factors:– something in world triggers general, nonspecific
arousal marked by increased heart rate, tightening of the stomach, and rapid breathing.
– people examine perceptual cues and their thoughts about the situation for clues for what has caused the emotion.
• Essentially 2 factor model– Appraisal a purely cognitive process that provides
context for interpreting body state
Overview of “how to run a study”
• Choose your variables, make them concrete
– IV, DV
• Choose your method
– Correlational or experimental
• Analyze data
– Depends on “type” of measurement
Choose your variables, make
them concrete• What do you want to study?
• Variables (or constructs)
• Operational definitions
– A clear, measurable definition of a construct
Our example
• Belonging is a fundamental need, like hunger
• But people feel need to different degrees…
Construct: Need to belong (NTB)
Our example
• Operational definition: Need to Belong
(NTB) Scale (Leary et al., 2001)
• High scores on items 2, 4, 5, 6, 8, 9, 10
• Low scores on items 1, 3, 7
Practice
• Come up with operational definitions for each of the following:
– Trust
– Aggression
– Depression
Choose your variables, make
them concrete• What do you want to study?
• Variables (or constructs)
• Operational definitions
– A clear, measurable definition of a construct
– Specify operational definition for IV and DV
IVs and DVs
• IV: Independent Variable (“manipulated”)
– Variable that you think causes the effect on
another variable
– Variable that is manipulated in an experiment
• DV: Dependent Variable (“measured”)
– The outcome variable (you think it has effect on)
– Variable measured in an experiment
Warning: some
foreshadowing
Overview of “how to run a study”
• Choose your variables, make them concrete
– IV, DV
• Choose your method
– Correlational or experimental
– Depends on your question
?
Our example
• What would that hunger make us do?
– And socially sensitive/aware
– Could ask two questions here
Our example
• “Is NTB associated with social sensitivity?”
• “Does NTB cause heightened sensitivity?”
Correlational
Experimental
NTB Sensitivity
NTB Sensitivity
Correlational vs experimental
• Recall, there was something different
about IVs and DVs in experimental…?
• Correlational
– Measure IV and measure DV
– Our example?
• Experimental
– Manipulate IV and measure DV
– Our example?
?
Is NTB associated with sensitivity?
• What kind of research tests this question?
• Correlational research
• Picket, Gardner, and Knowles
– NTB scale
– DANVA: Number of facial expressions correctly
identified
r = .38What does that mean?
Correlation Coefficient -- r
Correlation
coefficient
Indicates direction
of relationship
(positive or negative)
Indicates strength
of relationship
(0.00 to 1.00)
r = 0.63+r = +0.50
Problems with correlational research
• Only shows there’s an association
• We can’t say NTB causes social sensitivity,
why not…?
• It could be some other variable causing both
– Third variable problem
• Sensitivity could cause NTB
– Directionality problem
NTB Sensitivity
NTB Sensitivity
Third Variable
Problems with correlational research
• Third variable problem
NTBSocial
sensitivity
Motivation
Please
experimenter
Else?
Problems with correlational research
• Directionality problem
NTB Social
sensitivity
Problems with correlational research
• …and there will always another third variable
that you didn’t think of!
• So, conduct experimental research!
– What makes an experiment? What do you have to
do?
– Random assignment to conditions that only
differ on (thus manipulate) one variable
• Our example?Internal
validity!
Experimental NTB research
• Picket, Gardner, and Knowles
– What did they have to do here?
– Manipulate belonging needs
• Measure social sensitivity
Experimental NTB research
• NTB causes an increase in social sensitivity!
0
20
40
60
80
100
120
Control Rejection
So
cia
l s
en
sit
ivit
y
Benefit of experimental research
• Experimental research wins!
• Right?
• Do we only care about outcomes these
kinds of manipulations can effect?
• What important outcomes could NTB
effect? Think big! How might hunger affect
us?
Benefits of correlational research
• Real world outcomes
that really matter
0
0.05
0.1
0.15
0.2
Low High
Males
Females
Age-a
dju
ste
d m
ort
alit
y r
ate
Belonging
External
validity!
Problems with experimental research
• Hard to make manipulations powerful enough
to impact these kinds of outcomes
• Would you really want to?
– Ethical operationalizations?
– Ethically affect those outcomes?
• Even if didn’t have ethics….
– Can’t practically manipulate some things
Overview of “how to run a study”
• Choose your variables, make them concrete
– IV, DV
• Choose your method
– Correlational or experimental
• Analyze data
– Depends on “type” of measurement
Our example
• Please access the link that was emailed to
you and complete the study…
– ---once you are done, you can get up and stretch
• What did you experience?
• What kind of study was this?
• What am I testing? And why these 3 groups?
Data has been collected…
From the raw data it is impossible to tell if the groups differ!
Population
of
students
Sample
Control Life alone
4 5 6 1 3 6
2 7 7 3 2 7
4 6 7 2 5 5
2 3 7 5 7 3
7 6 7 2 3 7
6 1 7 4 2 5
3 1 2 2 2 4
1 1 3 4 2 5
6 1 4 6 5 1
2 7 7 2 7 2
2 7 2 7 1 3
4 7 6 7 5 4
7 2 4 6 4 7
3 6 5 6 3 4
1 7 7 3 5 5
Accident prone
…so run a statistical test
• Which one?
• It depends. …. On whether you’ve run a
correlational or experimental study, right?
– Calculate a correlation coefficient, r, if we ran
a correlational study, right?
– WRONG!
• It depends… on “type” of IV and DV
Overview of “how to run a study”
• Choose your variables, make them concrete
– IV, DV
• Choose your method
– Correlational or experimental
• Analyze data
– Depends on “type” of measurement
“Type” of IVs and DVs
• Both IV and DV can either be “categorical”
or “continuous”
– “groups” vs “how much?”
• Think continuous, think “continuum”
___________________
1 2 3 4 5 6 7
– Ask yourself: “can I put these in order?”
Continuous IV
(e.g., how much)
Categorical IV
(e.g, exp vs control)
Continuous
DV (e.g.,
how much)
Categorical
DV (e.g.,
yes/no)
Continuous IV
(e.g., how much)
Categorical IV
(e.g, exp vs control)
Continuous
DV (e.g.,
how much)t-test
Categorical
DV (e.g.,
yes/no)
Continuous IV
(e.g., how much)
Categorical IV
(e.g, exp vs control)
Continuous
DV (e.g.,
how much)
correlation t-test
Categorical
DV (e.g.,
yes/no)
Continuous IV
(e.g., how much)
Categorical IV
(e.g, exp vs control)
Continuous
DV (e.g.,
how much)
correlation t-test or ANOVA
Categorical
DV (e.g.,
yes/no)
Continuous IV
(e.g., how much)
Categorical IV
(e.g, exp vs control)
Continuous
DV (e.g.,
how much)
correlation or
regressiont-test or ANOVA
Categorical
DV (e.g.,
yes/no)
Continuous IV
(e.g., how much)
Categorical IV
(e.g, exp vs control)
Continuous
DV (e.g.,
how much)
correlation or
regressiont-test or ANOVA
Categorical
DV (e.g.,
yes/no)
chi-square test
Continuous IV
(e.g., how much)
Categorical IV
(e.g, exp vs control)
Continuous
DV (e.g.,
how much)
correlation or
regressiont-test or ANOVA
Categorical
DV (e.g.,
yes/no)
logistical
regression
chi-square test
What test would you use?
• Let’s go back:
– Recall Picket, Gardner, and Knowles’ r = .38
– Was that correct? Why?
– What test here?
0
20
40
60
80
100
120
Control Rejection
So
cia
l s
en
sit
ivit
y
What test would you use?
• Practice question:
– I ran a study to test whether people with high
need to belong smile more
– What do you need to know to figure out what
test I should use?
– IV: NTB scores
– DV: count of smiles during 5 minute
conversation
– If you had high vs low NTB?
– If you had high, med, low NTB?
– Are these last 2 experimental or correlational?
What test would you use?
• Practice question:
– I ran a study to test whether people with high
need to belong smile more
– What do you need to know to figure out what
test I should use?
– IV: NTB scores
– DV: smile or not in photo
– If you had high vs low NTB?
What test would you use?
• …to analyze our data?• ---while I get the data ready, practice & then take a break
• Let’s run that test!
• ...what if I had only randomly assigned you to
“live alone” vs “acceptance” conditions?
• …what if I had you provide your NTB scores
instead of assigning you to conditions?
Could we publish our data?
• Write down: at least one reason why we
could NOT publish our results?
• Lacking IRB approval & informed consent
• Analyze each item separately increases
chance of “false positive”
• Concerns about our sample
– Too small “N” (sample size)
– Findings may not generalize
– Demand characteristics
Summary of “how to experiment”
• Choose your variables, make them concrete
– IV, DV
• Choose your method
– Correlational or experimental
• Analyze data
– Depends on “type” of measurement
Recap
• Which should you choose when?
• Correlational research
– Important, “hard to manipulate” real-life outcomes
– Could not ethically or practically manipulate
• Experimental research
– Make causal claims
– Can manipulate cleanly
Variable Outcome Internal
validity!
• What does it mean to “cleanly” manipulate…?
• Hard to create conditions that only differ on one variable
– Why is it hard? What have you noticed about manipulations?
– …but its important to try…. Why?
Challenge in experimental research
Internal
validity!
• If conditions differ on more than one variable
– Those other differences are potential threats to internal validity
– what things differ between rejected & walk to school other than increased need to belong?
• Pain
• Negativity
• Mood
• Anxiety
– These could be alternative explanations• If rejected were more socially sensitive, could be….
Challenge in experimental research
Solutions in experimental research
• But, are ways to help ensure internal validity
• 1. Control condition similar to experimental
• The conditions are very much alike, helps to control
for some factors (alone vs with others, anxiety, etc)
– But still might differ on more than rejection: pain,
negativity, mood…
Solutions in experimental research
• 2. Other additional conditions that manipulate
these confounds– Pain
– Negativity
– If they don’t increase social sensitivity
but rejection condition does…
– Mood?
– Anxiety?
Solutions in experimental research
• 3. Measure alternative explanation– Pain
– Negativity
– Mood
– Anxiety
– If the rejection condition doesn’t increase these
compared to control…
– Even if it does, can control for them statistically,
and if still find a difference in social sensitivity…
Solutions in experimental research
• 4. Measure variable intended to manipulate
– If experimental condition is higher on that
variable…
– Manipulation check
– Mediation analysis
0
1
2
3
4
5
6
7
Control Rejected
Be
lon
gin
g n
ee
ds
Rejection
manipulationBelonging needs Sensitivity
Solutions in experimental research
• 5. Different manipulations across studies
– If always find difference…
– Multiple ways to increase NTB
• Reliving task
• Cyberball
• Living alone
• Chatroom
• More!
Convergent
validity!
Solutions in experimental research
• 6. Combine with other ways of asking the
question
• Does NTB cause heighten social sensitivity?
• A) NTB scores associated with sensitivity
• B) Heighten need (starve) increases sensitivity
• C) Reduce need (feed) decreases sensitivity?
Convergent
validity!
And, if its like hunger, you could also…?
Solutions in experimental research• Lucas et al.
• Priming acceptance to “satiate”– Acceptance - e.g., together went us they there
– Control - e.g., pretty was chair the skiing
• Social sensitivity
0
1
2
3
4
5
6
7
Acceptance Control
So
cia
l s
en
sit
ivit
y
Questions?
SparkNotes reading for today
• Its either available
online for free
(http://www.sparkn
otes.com/psycholo
gy/psych101/resea
rchmethods/),
where you have to
click through all the
sections yourself
• or download easier
-to-read (for < $5)
• http://www.barnesa
ndnoble.com/w/res
earch-methods-in-
psychology-
sparknotes-
editors/101822024
3?ean=978141149
1663