sampling & experimental control psych 231: research methods in psychology
Post on 21-Dec-2015
219 views
TRANSCRIPT
Sampling & Experimental Control
Psych 231: Research Methods in Psychology
Sampling
Population
Everybody that the research is targeted to be about
The subset of the population that actually participates in the research
Sample
Errors in measurement Sampling error
Why do we do we use sampling methods? Typically don’t have the
resources to test everybody, so we test a subset
Sampling
Sample
Inferential statistics used to generalize back
Sampling to make data collection manageable
Population
Allows us to quantify the Sampling error
Sampling
Goals of “good” sampling:– Maximize Representativeness:
– To what extent do the characteristics of those in the sample reflect those in the population
– Reduce Bias:– A systematic difference between those in
the sample and those in the population
Key tool: Random selection
Sampling Methods
Probability sampling Simple random sampling Systematic sampling Stratified sampling
Non-probability sampling Convenience sampling Quota sampling
Have some element of random selection
Susceptible to biased selection
Simple random sampling
Every individual has a equal and independent chance of being selected from the population
Systematic sampling
Selecting every nth person
Cluster sampling
Step 1: Identify groups (clusters) Step 2: randomly select from each group
Convenience sampling
Use the participants who are easy to get
Quota sampling
Step 1: identify the specific subgroups Step 2: take from each group until desired number of
individuals
Extraneous Variables
Control variables Holding things constant - Controls for excessive random
variability Random variables – may freely vary, to spread variability
equally across all experimental conditions Randomization
• A procedure that assures that each level of an extraneous variable has an equal chance of occurring in all conditions of observation.
Confound variables Variables that haven’t been accounted for (manipulated,
measured, randomized, controlled) that can impact changes in the dependent variable(s)
Co-varys with both the dependent AND an independent variable
Colors and words
Divide into two groups: men women
Instructions: Read aloud the COLOR that the words are presented in. When done raise your hand.
Women first. Men please close your eyes. Okay ready?
BlueGreenRedPurpleYellowGreenPurpleBlueRedYellowBlueRedGreen
List 1
Okay, now it is the men’s turn. Remember the instructions: Read aloud the
COLOR that the words are presented in. When done raise your hand.
Okay ready?
BlueGreenRedPurpleYellowGreenPurpleBlueRedYellowBlueRedGreen
List 2
Our results
So why the difference between the results for men versus women?
Is this support for a theory that proposes: “Women are good color identifiers, men are not” Why or why not? Let’s look at the two lists.
BlueGreenRedPurpleYellowGreenPurpleBlueRedYellowBlueRedGreen
List 2Men
BlueGreenRedPurpleYellowGreenPurpleBlueRedYellowBlueRedGreen
List 1Women
Matched Mis-Matched
What resulted in the perfomance difference? Our manipulated independent variable
(men vs. women) The other variable match/mis-match?
Because the two variables are perfectly correlated we can’t tell
This is the problem with confounds
BlueGreenRedPurpleYellowGreenPurpleBlueRedYellowBlueRedGreen
BlueGreen
RedPurple
YellowGreenPurple
BlueRed
YellowBlueRed
Green
IVDV
Confound
Co-vary together
Experimental Control
Our goal: To test the possibility of a relationship
between the variability in our IV and how that affects the variability of our DV.
• Control is used to minimize excessive variability.• To reduce the potential of confounds.
Sources of variability (noise)
Nonrandom (NR) Variability - systematic variation A. (NRexp) manipulated independent variables (IV)
i. our hypothesis is that changes in the IV will result in changes in the DV
Sources of Total (T) Variability:
T = NRexp + NRother + R
B. (NRother) extraneous variables (EV) which covary with IV i. Condfounds
Sources of variability (noise)
Non-systematic variationC. Random (R) Variability
• Imprecision in manipulation (IV) and/or measurement (DV) • Randomly varying extraneous variables (EV)
Sources of Total (T) Variability:
T = NRexp + NRother + R
Sources of variability (noise)
Sources of Total (T) Variability:
T = NRexp + NRother + R Goal: to reduce R and NRother so that we can detect NRexp.
That is, so we can see the changes in the DV that are due to the changes in the independent variable(s).
Weight analogy
Imagine the different sources of variability as weights
RNRexp
NRother
RNRother
Treatment group control group
The effect of the treatment
Weight analogy
If NRother and R are large relative to NRexp then detecting a difference may be difficult
RNRexp
NRother
RNRother
Difference Detector
Weight analogy
But if we reduce the size of NRother and R relative to NRexp then detecting gets easier
RNRother
RNRexpNR
other
Difference Detector
Things making detection difficult
Potential Problems Confounding Excessive random variability Dissimulation
Difference Detector
Potential Problems
Confounding If an EV co-varies with IV, then NRother component
of data will be "significantly" large, and may lead to misattribution of effect to IV
IVDV
EV
Co-vary together
Confounding
RNRexp
NRother
Hard to detect the effect of NRexp because the effect looks like it could be from NRexp but is really (mostly) due to the NRother
R
Difference Detector
Potential Problems
Excessive random variability If experimental control procedures are not applied
• Then R component of data will be excessively large, and may make NRexp undetectable
So try to minimize this by using good measures of DV, good manipulations of IV, etc.
Excessive random variability
RNRexp
NRother
NRother
R
Hard to detect the effect of NRexp
Difference Detector
Potential Problems
Potential problem caused by experimental control Dissimulation
• If EV which interacts with IV is held constant, then effect of IV is known only for that level of EV, and may lead to overgeneralization of IV effect
This is a potential problem that affects the external validity
Controlling Variability
Methods of Experimental Control Comparison Production Constancy/Randomization
Methods of Controlling Variability
Comparison An experiment always makes a comparison, so it must have at
least two groups• Sometimes there are control groups
• This is typically the absence of the treatment• Without control groups if is harder to see what is really
happening in the experiment• It is easier to be swayed by plausibility or inappropriate
comparisons
• Sometimes there are just a range of values of the IV
Methods of Controlling Variability
Production The experimenter selects the specific values of the
Independent Variables • Need to do this carefully
• Suppose that you don’t find a difference in the DV across your different groups
• Is this because the IV and DV aren’t related?• Or is it because your levels of IV weren’t different enough
Methods of Controlling Variability
Constancy/Randomization If there is a variable that may be related to the DV that you
can’t (or don’t want to) manipulate• Control variable: hold it constant
• Random variable: let it vary randomly across all of the experimental conditions
But beware confounds, variables that are related to both the IV and DV but aren’t controlled
Experimental designs
So far we’ve covered a lot of the about details experiments generally
Now let’s consider some specific experimental designs. Some bad designs Some good designs
• 1 Factor, two levels
• 1 Factor, multi-levels
• Factorial (more than 1 factor)
• Between & within factors
Poorly designed experiments
Example: Does standing close to somebody cause them to move? So you stand closely to people and see how long before
they move
Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”)
Poorly designed experiments
Does a relaxation program decrease the urge to smoke? One group pretest-posttest design Pretest desire level – give relaxation program – posttest desire
to smoke
Poorly designed experiments
One group pretest-posttest design
Problems include: history, maturation, testing, instrument decay, statistical regression, and more
participants Pre-test Training group
Post-testMeasure
Independent Variable
Dependent Variable
Dependent Variable
Poorly designed experiments
Example: Smoking example again, but with two groups. The subjects get to choose which group (relaxation or no program) to be in
Non-equivalent control groups Problem: selection bias for the two groups, need to do random
assignment to groups
Poorly designed experiments
Non-equivalent control groups
participants
Traininggroup
No training (Control) group
Measure
Measure
Self Assignment
Independent Variable
Dependent Variable
“Well designed” experiments
Post-test only designs
participants
Experimentalgroup
Controlgroup
Measure
Measure
Random Assignment
Independent Variable
Dependent Variable
“Well designed” experiments
Pretest-posttest design
participants
Experimentalgroup
Controlgroup
Measure
Measure
Random Assignment
Independent Variable
Dependent Variable
Measure
Measure
Dependent Variable