© 2001 dr. laura snodgrass, ph.d.1 basic experimental design common problems assigning participants...
TRANSCRIPT
© 2001 Dr. Laura Snodgrass, Ph.D. 1
Basic Experimental Design
• Common Problems• Assigning Participants to Groups• Single variable experiments
– bivalent– multivalent– baseline
• Multivariate– factorial– converging series
© 2001 Dr. Laura Snodgrass, Ph.D. 2
Common Problems
• Confounds• Lack of control group(s)• Nonequivalent control groups• Why control groups
– history– maturation– testing– instrument decay– statistical regression
© 2001 Dr. Laura Snodgrass, Ph.D. 3
Assigning Participants to Groups
• Independent or Random Groups Design– between groups
• Repeated Measures– within groups
© 2001 Dr. Laura Snodgrass, Ph.D. 4
Between Groups
• Advantages– generalizable– collect more data at a given level– shorter time for each participant
• Disadvantages– may not be random– unequal N– potential confounds– requires more participants
© 2001 Dr. Laura Snodgrass, Ph.D. 5
Between Groups
• Matching to equate groups and decrease error variance
• How– correlated variables– pairs– yoked controls– performance criterion
© 2001 Dr. Laura Snodgrass, Ph.D. 6
Matching
• Advantages– equates groups– increase power of experiment– decrease number of participants needed
• Disadvantages– extra work– extra testing– lose individual differences - less generalizable
© 2001 Dr. Laura Snodgrass, Ph.D. 7
Repeated Measures
• Advantages– fewer participants needed– impt for special groups– statistically more powerful
• Disadvantages– not naïve after first trials– order effects
• practice and fatigue• non-symmetric or differential transfer
© 2001 Dr. Laura Snodgrass, Ph.D. 8
Counterbalancing
• Vary order of treatment to distribute or measure order effects
• Complete counterbalancing– within participants ABBA– between AB for some, BA for others
• Latin Squares– each cond at each ordinal position– precedes and follows each other once
© 2001 Dr. Laura Snodgrass, Ph.D. 9
Counterbalancing
• Randomized blocks
• Time interval between trials– mortality
© 2001 Dr. Laura Snodgrass, Ph.D. 10
Single Variable Experiments
• Bivalent– one independent variable with two levels
• Multivalent (functional)
– one independent variable with three or more levels
• Baseline
© 2001 Dr. Laura Snodgrass, Ph.D. 11
Bivalent
• Two levels of the independent variable– experimental and control groups– two different levels of the variable
• Post-test only vs. pre-test/post-test
• Advantages– easy to interpret and analyze– decide if IV is worth studying
© 2001 Dr. Laura Snodgrass, Ph.D. 12
Bivalent
• Disadvantages– limited theoretical value– conclusions may be based on arbitrary choice of
levels– negative findings are not conclusive– does not describe shape of relationship therefore you
may over generalize for non-linear relationships• interpolation and extrapolation• plateau or asymptote
© 2001 Dr. Laura Snodgrass, Ph.D. 13
Multivalent (functional)
• Gives more info about the shape of the relationship
• Advantages– better estimate true relationship– individual choice of levels becomes less critical
• Disadvantages– more: time, effort, cost, subjects– more complex statistics and interpretation
© 2001 Dr. Laura Snodgrass, Ph.D. 14
Baseline
• Only works with certain types of variables– will not work with variables that cause permanent
change
• Procedure:– establish baseline or steady-state response level– introduce IV until stable transition– allow subject to return to baseline
© 2001 Dr. Laura Snodgrass, Ph.D. 15
Baseline
• Advantages– rules out most confounds– easy to interpret (often no statistics)
– flexible and replicable– investigate behavior of an individual
• Disadvantages– does not show small changes– may not generalize
© 2001 Dr. Laura Snodgrass, Ph.D. 16
Multivariate Experiments
• Factorial Designs– two or more independent variables, each with two or
more levels– variables can be all between, all within, or mixed in
many combinations
• Converging series– series of small experiments in which a variable
manipulated in an earlier experiment becomes a control variable in a later experiment
© 2001 Dr. Laura Snodgrass, Ph.D. 17
Factorial
• Design matrix– produces a family of functions– study main effects and interactions
• Advantages– study interactions– increases precision and generalizability– decrease statistical error and increase power– theoretical value
© 2001 Dr. Laura Snodgrass, Ph.D. 18
Factorial
• Disadvantages– increases time, money and number of subjects
increases dramatically as number of cells increases– assumptions of ANOVA may not be met– N-way interactions are very difficult to interpret
© 2001 Dr. Laura Snodgrass, Ph.D. 19
Converging series for applied problems
• Optimal designs– e.g. car, medical treatment, office
• Find an optimal level of a variable and turn it into a control variable
– lose higher order interactions
© 2001 Dr. Laura Snodgrass, Ph.D. 20
Converging Operations
• Converge on a single hypothesis– start with several possible hypotheses or
explanations– each experiment eliminates one or more until only
one remains (hopefully)
• For example: – perceptual defense against vulgar words– isolation tank
© 2001 Dr. Laura Snodgrass, Ph.D. 21
Converging Series
• Advantages– flexible, many choice points– efficient, leave out factors that have no effect– built in replications
• Disadvantages– interactions are lost– almost always between subjects– analyze and interpret prior before next experiment
so can take a long time