# iufro spdc workshop: research methods

Embed Size (px)

TRANSCRIPT

Experimental DesignSEPTEMBER 26 – 28, 2019

FACULTY OF FORESTRY AND ENVIRONMENTAL MANAGEMENT

UNIVERSITY OF NEW BRUNSWICK

EXPERIMENTAL DESIGN

• Concerned with the analyses of existing data

• ANOVA is the subject of books entitled “Experimental Design”

THREE SCENARIOS

• The Good

• You designed

the experiment

you a file • In the file is a map,

a brief description of an experiment

• and………………… .. the data sheets

• Translating questions into formula

• Formula into treatment levels

• Problem/Statement

• Question

• Hypothesis

• Predictions

• Tests

WHAT IS A RESEARCH HYPOTHESIS?

• A research hypothesis is the statement created by researchers when they speculate upon the outcome of a research or experiment

• The “expected” answer to your research question

• A Research Hypothesis must:

• ………

• Some provide explanations, some provide answers

• Some are qualitative, some are quantitative

• Some are simple, some mechanistic

• Some are conceived quickly, some takes months (or longer)

MCROBERT’S JUSTIFICATION STRUCTURE

GREEN’S 10 PRINCIPLES

2. Take replicate samples within each combination of time, location, and any

other controlled variable. Differences between can only be demonstrated by

comparison to differences within.

• Equal probability of having a treatment applied to it

• The level at which the treatment is applied

• Any measurement made below the level of treatment is a subsample

• A psuedoreplicate is a subsample used as a replicate

DIFFERENCES WITHIN VS BETWEEN

DIFFERENCES WITHIN VS BETWEEN

GREEN’S 10 PRINCIPLES

3. Take an equal number of samples within each combination of time, location,

and any other controlled variable. Putting samples in “representative” or

“typical” places is not sampling.

IMPLICATIONS FOR DESIGNING EXPERIMENTS

supporting and evidence contradicting our hypothesis to satisfy the test criteria

• Falsification experiments concentrate resources to provide maximum opportunity

to detect counter examples

GREEN’S 10 PRINCIPLES

4. To test whether a condition has an effect, collect samples both where the

condition is present and where the condition is absent, but all else is the same.

An effect can only be demonstrated by comparison with a control.

NEST PREDATION AND EDGE EFFECTS

GREEN’S 10 PRINCIPLES

5. Carry out some preliminary sampling to provide a basis for evaluation of

sample design and statistical analysis options. Those who skip this step because

they do not have enough time usually end up losing time.

POWER OF A STATISTICAL TEST

TYPE I & TYPE II ERRORS AND POWER EFFECT SIZE AND SAMPLE SIZE

GREEN’S 10 PRINCIPLES

6. Verify that your sampling device or method is sampling the population you

think you are sampling, and with equal and adequate efficiency over the entire

range of sampling conditions to be encountered. Variation in efficiency of

sampling from area to area biases among-area comparisons.

YELLOWSTONE GRIZZLY BEARS

GREEN’S 10 PRINCIPLES

7. If the area to be sampled has a large-scale environmental pattern, break the

area up into relatively homogeneous subareas and allocate samples to each in

proportion to the size of the subarea. If it is an estimate of total abundance

over the entire area that is desired, make the allocation proportional to the

number of organisms in the subarea.

BLOCKING

GREEN’S 10 PRINCIPLES

8. Verify that your sample unit size is appropriate to the size, densities, and

spatial distributions of the organisms you are sampling. Then estimate the

number of replicate samples required to obtain the precision you want.

SIZE FACTORS

GREEN’S 10 PRINCIPLES

9. Test your data to determine whether the ERROR (residual) variation is

homogeneous, normally distributed, and independent of the mean. If it is not,

as is frequently [occasionally] encountered in field data, then (a) appropriately

transform the data, (b) use a distribution-free [robust] statistical procedure, (c)

use a sequential sampling design, (d) test against simulate H0 data; [or (e) use

a mixed modeling approach].

GREEN’S 10 PRINCIPLES

10. Having chosen the best [most powerful] statistical method to test your

hypothesis, stick with the result. An unexpected or undesired result is NOT a

valid reason for rejecting the method and hunting for a “better” one.

WHAT IS A STATISTICAL HYPOTHESIS?

• A statistical hypothesis is an assumption about a population parameter

• This assumption may or may not be true

• Hypothesis testing refers to the formal procedures used by statisticians to fail to

reject or reject statistical hypotheses

RESEARCH HYPOTHESES

measurements

mean tail length of cats in Europe

• Research hypothesis is about how

nature is or how nature works

• Cats evolved tails because tails have

a survival advantage; specifically,

help them right themselves, so that

they land on their feet

STATISTICAL HYPOTHESES

• Null Hypothesis

• µ1 = µ2 = µ3

• µ1 µ2 = µ3

• µ1 = µ2 µ3

• µ1 µ2 µ3

• Specific Hypothesis

• General Hypothesis

STATISTICAL TEST

• We construct our “Test” statistic assuming the Null hypothesis is true

• If the Null hypothesis is true, the test statistic should be 0 (no difference)

• Most likely (and hopefully) our test statistic will be >> 0

• Because we have a sample we have sampling error

• We learned from Ting-Ru that the sampling error causes differences in our

estimates of the mean

STATISTICAL TEST

• So we could obtain a test statistic >> 0, because of sampling error

• Therefore, we assess, given the variability in our population, what is the

probability that a difference as large as we have observed, is due to sampling error

• If that probability is small, then we assume the difference is not due to sample

error, but due to our treatment, and we conclude that we have significant

treatment effects

TAKE AWAYS

• Skipping steps to save time may cost you more time

• Unexpected results may reflect bad research design, bad hypotheses, or novel discoveries

• Controlling the first 2 lead to the 3rd

REFLECTIVE WRITING #XX

• What logistical blocks do you have in your study? Have you missed or avoided any of

Green’s Principles?

FACULTY OF FORESTRY AND ENVIRONMENTAL MANAGEMENT

UNIVERSITY OF NEW BRUNSWICK

EXPERIMENTAL DESIGN

• Concerned with the analyses of existing data

• ANOVA is the subject of books entitled “Experimental Design”

THREE SCENARIOS

• The Good

• You designed

the experiment

you a file • In the file is a map,

a brief description of an experiment

• and………………… .. the data sheets

• Translating questions into formula

• Formula into treatment levels

• Problem/Statement

• Question

• Hypothesis

• Predictions

• Tests

WHAT IS A RESEARCH HYPOTHESIS?

• A research hypothesis is the statement created by researchers when they speculate upon the outcome of a research or experiment

• The “expected” answer to your research question

• A Research Hypothesis must:

• ………

• Some provide explanations, some provide answers

• Some are qualitative, some are quantitative

• Some are simple, some mechanistic

• Some are conceived quickly, some takes months (or longer)

MCROBERT’S JUSTIFICATION STRUCTURE

GREEN’S 10 PRINCIPLES

2. Take replicate samples within each combination of time, location, and any

other controlled variable. Differences between can only be demonstrated by

comparison to differences within.

• Equal probability of having a treatment applied to it

• The level at which the treatment is applied

• Any measurement made below the level of treatment is a subsample

• A psuedoreplicate is a subsample used as a replicate

DIFFERENCES WITHIN VS BETWEEN

DIFFERENCES WITHIN VS BETWEEN

GREEN’S 10 PRINCIPLES

3. Take an equal number of samples within each combination of time, location,

and any other controlled variable. Putting samples in “representative” or

“typical” places is not sampling.

IMPLICATIONS FOR DESIGNING EXPERIMENTS

supporting and evidence contradicting our hypothesis to satisfy the test criteria

• Falsification experiments concentrate resources to provide maximum opportunity

to detect counter examples

GREEN’S 10 PRINCIPLES

4. To test whether a condition has an effect, collect samples both where the

condition is present and where the condition is absent, but all else is the same.

An effect can only be demonstrated by comparison with a control.

NEST PREDATION AND EDGE EFFECTS

GREEN’S 10 PRINCIPLES

5. Carry out some preliminary sampling to provide a basis for evaluation of

sample design and statistical analysis options. Those who skip this step because

they do not have enough time usually end up losing time.

POWER OF A STATISTICAL TEST

TYPE I & TYPE II ERRORS AND POWER EFFECT SIZE AND SAMPLE SIZE

GREEN’S 10 PRINCIPLES

6. Verify that your sampling device or method is sampling the population you

think you are sampling, and with equal and adequate efficiency over the entire

range of sampling conditions to be encountered. Variation in efficiency of

sampling from area to area biases among-area comparisons.

YELLOWSTONE GRIZZLY BEARS

GREEN’S 10 PRINCIPLES

7. If the area to be sampled has a large-scale environmental pattern, break the

area up into relatively homogeneous subareas and allocate samples to each in

proportion to the size of the subarea. If it is an estimate of total abundance

over the entire area that is desired, make the allocation proportional to the

number of organisms in the subarea.

BLOCKING

GREEN’S 10 PRINCIPLES

8. Verify that your sample unit size is appropriate to the size, densities, and

spatial distributions of the organisms you are sampling. Then estimate the

number of replicate samples required to obtain the precision you want.

SIZE FACTORS

GREEN’S 10 PRINCIPLES

9. Test your data to determine whether the ERROR (residual) variation is

homogeneous, normally distributed, and independent of the mean. If it is not,

as is frequently [occasionally] encountered in field data, then (a) appropriately

transform the data, (b) use a distribution-free [robust] statistical procedure, (c)

use a sequential sampling design, (d) test against simulate H0 data; [or (e) use

a mixed modeling approach].

GREEN’S 10 PRINCIPLES

10. Having chosen the best [most powerful] statistical method to test your

hypothesis, stick with the result. An unexpected or undesired result is NOT a

valid reason for rejecting the method and hunting for a “better” one.

WHAT IS A STATISTICAL HYPOTHESIS?

• A statistical hypothesis is an assumption about a population parameter

• This assumption may or may not be true

• Hypothesis testing refers to the formal procedures used by statisticians to fail to

reject or reject statistical hypotheses

RESEARCH HYPOTHESES

measurements

mean tail length of cats in Europe

• Research hypothesis is about how

nature is or how nature works

• Cats evolved tails because tails have

a survival advantage; specifically,

help them right themselves, so that

they land on their feet

STATISTICAL HYPOTHESES

• Null Hypothesis

• µ1 = µ2 = µ3

• µ1 µ2 = µ3

• µ1 = µ2 µ3

• µ1 µ2 µ3

• Specific Hypothesis

• General Hypothesis

STATISTICAL TEST

• We construct our “Test” statistic assuming the Null hypothesis is true

• If the Null hypothesis is true, the test statistic should be 0 (no difference)

• Most likely (and hopefully) our test statistic will be >> 0

• Because we have a sample we have sampling error

• We learned from Ting-Ru that the sampling error causes differences in our

estimates of the mean

STATISTICAL TEST

• So we could obtain a test statistic >> 0, because of sampling error

• Therefore, we assess, given the variability in our population, what is the

probability that a difference as large as we have observed, is due to sampling error

• If that probability is small, then we assume the difference is not due to sample

error, but due to our treatment, and we conclude that we have significant

treatment effects

TAKE AWAYS

• Skipping steps to save time may cost you more time

• Unexpected results may reflect bad research design, bad hypotheses, or novel discoveries

• Controlling the first 2 lead to the 3rd

REFLECTIVE WRITING #XX

• What logistical blocks do you have in your study? Have you missed or avoided any of

Green’s Principles?