knr 497 exptl design slide 1 experimental design ch. 8 – let’s not kid ourselves, this is going...
TRANSCRIPT
KNR 497Exptl
DesignSlide 1
Experimental Design
Ch. 8 – Let’s not kid ourselves, this
is going to hurt
KNR 497Exptl
designSlide 2 Experimental Design
How on Earth can you ensure that 2 groups of different people are equal (in all respects, not just on the measure of choice) at the beginning of an experiment? You can’t But you can make it more probable
(and to experimenters, good enough)
Remember though, even if you achieve this, groups can still grow different after they have been formed
KNR 497Exptl
designSlide 3 Experimental Design
Searching for group equivalence What we do:
Random assignment
Does it work? Maybe! Sample size, power &c.
KNR 497Exptl
designSlide 4 Experimental Design
If random assignment is the solution, and increased internal validity is the benefit, is there a cost? Undoubtedly
Sample size big enough? Control of social threats, & mortality Its unreal, so improved internal validity
comes at the cost of external validity
KNR 497Exptl
designSlide 5 Experimental Design
2-group experimental designs
Two-group, post-test only randomized experimental design
KNR 497Exptl
designSlide 6 Experimental Design
More on probabilistic equivalence Random assignment will
distribute folk to groups such that their scores on any measure will be distributed randomly (duh)…this means they will probably be different, but that it is statistically improbable that this will be a significant difference
KNR 497Exptl
designSlide 8 Experimental Design
Random selection Random assignment
External validity control
Internal validity control
KNR 497Exptl
designSlide 9 Experimental Design
Classifying experimental designs Signal enhancing vs. noise
reducing The signal vs. noise idea:
Strong treatment enhances signal
Good measurement reduces noise
KNR 497Exptl
designSlide 10 Experimental Design
Classifying experimental designs Signal enhancing vs. noise
reducing Designs differ in their strengths ~
Factorial designs focus on isolating aspects or combinations of treatments that seem to affect the measurement most (signal enhancer)
Covariance/blocking designs focus on lessening the effects of known sources of noise (noise reducers)
KNR 497Exptl
designSlide 11 Experimental Design
Factorial designs Imagine an educational
program… You are interested in (IV’s)
Time of instruction (1 hour vs. 4 hr) Setting (in-class or pulled out of class)
You measure via study scores (DV)
Note – we are now dealing with 2 independent
variables for the first time
KNR 497Exptl
designSlide 19
Experimental Design: Factorial
A silly example - The marshmallow peeps study Factor 1: Alcohol
(presence/absence) Factor 2: Smoking (yes/no)
KNR 497Exptl
designSlide 20
Experimental Design: Factorial
Does alcohol have an effect? Imbibed liberally Moderate
headache Nausea No permanent
damage
KNR 497Exptl
designSlide 21
Experimental Design: Factorial
Does tobacco have an effect?
No marketing to young chicks Peep grabs a ciggie… …lights up…
…begins smoking… …& continues ‘til
satiated It can give up any
time it wants to…no effect
KNR 497Exptl
designSlide 22
Experimental Design: Factorial
So, alcohol & nicotine are benign?
Wait..what if you combined them? Sum of the parts? More than the sum of the parts?
KNR 497Exptl
designSlide 23
Experimental Design: Factorial
Is there an interaction? Combine the elements
Faint flame…blackening …smell of caramel…
Metamorphosis “ball of charred
goo…” “less sweet” “crunchier” “gross”
KNR 497Exptl
designSlide 28
Experimental Design: Blocking
Reducing noise – Randomized block designs Key point – unexplained variation in a
sample reduces power The solution is to reduce the variation within the
sample by splitting the sample up You split across some factor that you know causes
the sample to differ with respect to the measure of interest (making multiple blocks)
You do not include this as a factor in the experiment, because it is not of interest
Each block will have less variability on the measure, and therefore more power
KNR 497Exptl
designSlide 29
Experimental Design: Blocking
Reducing noise – Randomized block designs
Here is the design
notation for what was
described on the last slide
KNR 497Exptl
designSlide 30
Experimental Design: Blocking
Reducing noise – Randomized block designs
“+’s” show scores for all treatment group
members (average of all “+” gives treatment
group score – average on x-axis is for pretest, and on y-axis is for posttest
“o’s” show scores for all control group members (average of all “o” gives
control group score – average on x-axis is for pretest, and on y-axis is
for posttest
KNR 497Exptl
designSlide 31
Experimental Design: Blocking
Reducing noise – Randomized block designs
Note that, regardless of the block, the spread of
scores on the post-test is less within the block than
across the entire measure
KNR 497Exptl
designSlide 32
Experimental Design: Covariates
Reducing noise – Covariance designs Design can vary, but basic is this –
Lingo – “controlling for”, “removing the effect of” Both terms imply use of covariates
KNR 497Exptl
designSlide 34
Experimental Design: Hybrids
Solomon 4 group
To examine & control testing effects in pre-
post arrangements
KNR 497Exptl
designSlide 35
Experimental Design: Hybrids
Switched replication design
To examine & control social interaction
threats