Experimental DesignSEPTEMBER 26 – 28, 2019
FACULTY OF FORESTRY AND ENVIRONMENTAL MANAGEMENT
UNIVERSITY OF NEW BRUNSWICK
EXPERIMENTAL DESIGN
• Concerned with the analyses of existing data
• ANOVA is the subject of books entitled “Experimental Design”
THREE SCENARIOS
• The Good
• You designed
the experiment
you a file • In the file is a map,
a brief description of an experiment
• and………………… .. the data sheets
• Translating questions into formula
• Formula into treatment levels
• Problem/Statement
• Question
• Hypothesis
• Predictions
• Tests
WHAT IS A RESEARCH HYPOTHESIS?
• A research hypothesis is the statement created by researchers when they speculate upon the outcome of a research or experiment
• The “expected” answer to your research question
• A Research Hypothesis must:
• ………
• Some provide explanations, some provide answers
• Some are qualitative, some are quantitative
• Some are simple, some mechanistic
• Some are conceived quickly, some takes months (or longer)
MCROBERT’S JUSTIFICATION STRUCTURE
GREEN’S 10 PRINCIPLES
2. Take replicate samples within each combination of time, location, and any
other controlled variable. Differences between can only be demonstrated by
comparison to differences within.
• Equal probability of having a treatment applied to it
• The level at which the treatment is applied
• Any measurement made below the level of treatment is a subsample
• A psuedoreplicate is a subsample used as a replicate
DIFFERENCES WITHIN VS BETWEEN
DIFFERENCES WITHIN VS BETWEEN
GREEN’S 10 PRINCIPLES
3. Take an equal number of samples within each combination of time, location,
and any other controlled variable. Putting samples in “representative” or
“typical” places is not sampling.
IMPLICATIONS FOR DESIGNING EXPERIMENTS
supporting and evidence contradicting our hypothesis to satisfy the test criteria
• Falsification experiments concentrate resources to provide maximum opportunity
to detect counter examples
GREEN’S 10 PRINCIPLES
4. To test whether a condition has an effect, collect samples both where the
condition is present and where the condition is absent, but all else is the same.
An effect can only be demonstrated by comparison with a control.
NEST PREDATION AND EDGE EFFECTS
GREEN’S 10 PRINCIPLES
5. Carry out some preliminary sampling to provide a basis for evaluation of
sample design and statistical analysis options. Those who skip this step because
they do not have enough time usually end up losing time.
POWER OF A STATISTICAL TEST
TYPE I & TYPE II ERRORS AND POWER EFFECT SIZE AND SAMPLE SIZE
GREEN’S 10 PRINCIPLES
6. Verify that your sampling device or method is sampling the population you
think you are sampling, and with equal and adequate efficiency over the entire
range of sampling conditions to be encountered. Variation in efficiency of
sampling from area to area biases among-area comparisons.
YELLOWSTONE GRIZZLY BEARS
GREEN’S 10 PRINCIPLES
7. If the area to be sampled has a large-scale environmental pattern, break the
area up into relatively homogeneous subareas and allocate samples to each in
proportion to the size of the subarea. If it is an estimate of total abundance
over the entire area that is desired, make the allocation proportional to the
number of organisms in the subarea.
BLOCKING
GREEN’S 10 PRINCIPLES
8. Verify that your sample unit size is appropriate to the size, densities, and
spatial distributions of the organisms you are sampling. Then estimate the
number of replicate samples required to obtain the precision you want.
SIZE FACTORS
GREEN’S 10 PRINCIPLES
9. Test your data to determine whether the ERROR (residual) variation is
homogeneous, normally distributed, and independent of the mean. If it is not,
as is frequently [occasionally] encountered in field data, then (a) appropriately
transform the data, (b) use a distribution-free [robust] statistical procedure, (c)
use a sequential sampling design, (d) test against simulate H0 data; [or (e) use
a mixed modeling approach].
GREEN’S 10 PRINCIPLES
10. Having chosen the best [most powerful] statistical method to test your
hypothesis, stick with the result. An unexpected or undesired result is NOT a
valid reason for rejecting the method and hunting for a “better” one.
WHAT IS A STATISTICAL HYPOTHESIS?
• A statistical hypothesis is an assumption about a population parameter
• This assumption may or may not be true
• Hypothesis testing refers to the formal procedures used by statisticians to fail to
reject or reject statistical hypotheses
RESEARCH HYPOTHESES
measurements
mean tail length of cats in Europe
• Research hypothesis is about how
nature is or how nature works
• Cats evolved tails because tails have
a survival advantage; specifically,
help them right themselves, so that
they land on their feet
STATISTICAL HYPOTHESES
• Null Hypothesis
• µ1 = µ2 = µ3
• µ1 µ2 = µ3
• µ1 = µ2 µ3
• µ1 µ2 µ3
• Specific Hypothesis
• General Hypothesis
STATISTICAL TEST
• We construct our “Test” statistic assuming the Null hypothesis is true
• If the Null hypothesis is true, the test statistic should be 0 (no difference)
• Most likely (and hopefully) our test statistic will be >> 0
• Because we have a sample we have sampling error
• We learned from Ting-Ru that the sampling error causes differences in our
estimates of the mean
STATISTICAL TEST
• So we could obtain a test statistic >> 0, because of sampling error
• Therefore, we assess, given the variability in our population, what is the
probability that a difference as large as we have observed, is due to sampling error
• If that probability is small, then we assume the difference is not due to sample
error, but due to our treatment, and we conclude that we have significant
treatment effects
TAKE AWAYS
• Skipping steps to save time may cost you more time
• Unexpected results may reflect bad research design, bad hypotheses, or novel discoveries
• Controlling the first 2 lead to the 3rd
REFLECTIVE WRITING #XX
• What logistical blocks do you have in your study? Have you missed or avoided any of
Green’s Principles?