iufro spdc workshop: research methods

39
IUFRO SPDC WORKSHOP: RESEARCH METHODS CURITIBA, BRAZIL SEPTEMBER 26 – 28, 2019 DR. JOHN A. KERSHAW, JR. AND MS. TING-RU YANG FACULTY OF FORESTRY AND ENVIRONMENTAL MANAGEMENT UNIVERSITY OF NEW BRUNSWICK FREDERICTON, NB CANADA

Upload: others

Post on 05-Oct-2021

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IUFRO SPDC WORKSHOP: RESEARCH METHODS

IUFRO SPDC WORKSHOP: RESEARCH METHODSCURITIBA, BRAZIL

SEPTEMBER 26 – 28, 2019

DR. JOHN A. KERSHAW, JR. AND MS. TING-RU YANG

FACULTY OF FORESTRY AND ENVIRONMENTAL MANAGEMENT

UNIVERSITY OF NEW BRUNSWICK

FREDERICTON, NB CANADA

Page 2: IUFRO SPDC WORKSHOP: RESEARCH METHODS

PRINCIPLES OFDESIGNING EXPERIMENTS

Page 3: IUFRO SPDC WORKSHOP: RESEARCH METHODS
Page 4: IUFRO SPDC WORKSHOP: RESEARCH METHODS

TOPICS

• Experimental Design versus Designing Experiments

• Research Hypotheses versus Statistical Hypotheses

Page 5: IUFRO SPDC WORKSHOP: RESEARCH METHODS

EXPERIMENTAL DESIGN

• The statistical models used to assess treatment effects

• Concerned with the analyses of existing data

• ANOVA is the subject of books entitled “Experimental Design”

Page 6: IUFRO SPDC WORKSHOP: RESEARCH METHODS
Page 7: IUFRO SPDC WORKSHOP: RESEARCH METHODS

THREE SCENARIOS

• The Good

• You designed

the experiment

• You have the

data

• Now what?

• The Bad

• Someone else

designed the

experiment

• They explain how

they laid out the

treatments

• You get the data

• Now what?

•The Ugly• Your boss hands

you a file• In the file is a map,

a brief description of an experiment

• and………………….. the data sheets

• Now what?

Page 8: IUFRO SPDC WORKSHOP: RESEARCH METHODS

DESIGNING EXPERIMENTS

• Concerns the “logistics” of the experimental process

• Translating questions into formula

• Formula into treatment levels

• Sampling schemes

• Treatment layout

• Measurements

• Choosing tests

• Determining results (Experimental Design)

Page 9: IUFRO SPDC WORKSHOP: RESEARCH METHODS

MY GO-TO REFERENCE

Page 10: IUFRO SPDC WORKSHOP: RESEARCH METHODS

WHAT IS A RESEARCH HYPOTHESIS?

• Problem/Statement

• Question

• Hypothesis

• Predictions

• Tests

Page 11: IUFRO SPDC WORKSHOP: RESEARCH METHODS

WHAT IS A RESEARCH HYPOTHESIS?

• A research hypothesis is the statement created by researchers when they speculate upon the outcome of a research or experiment

• The “expected” answer to your research question

• A Research Hypothesis must:

• Be concise

• Be testable

• Be falsifiable

• Mean the same to everyone

• ………

Page 12: IUFRO SPDC WORKSHOP: RESEARCH METHODS

RESEARCH HYPOTHESES

• Many ways to Classify them

• Some provide explanations, some provide answers

• Some are qualitative, some are quantitative

• Some are simple, some mechanistic

• Some are conceived quickly, some takes months (or longer)

Page 13: IUFRO SPDC WORKSHOP: RESEARCH METHODS

MCROBERT’S JUSTIFICATION STRUCTURE

Number of hypotheses

Corroboration

Contradiction

Disproof

1 2+

Hypothetico-

Deduction

Multiple

Hypotheses

Falsification Strong

inference

0

Induction or

Retroduction

[Discovery] Justification

Page 14: IUFRO SPDC WORKSHOP: RESEARCH METHODS

HYPOTHESIS -> PREDICTIONS

• Predictions are the currency you test

• Statistical hypotheses are derived from predictions

Page 15: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

2. Take replicate samples within each combination of time, location, and any

other controlled variable. Differences between can only be demonstrated by

comparison to differences within.

Page 16: IUFRO SPDC WORKSHOP: RESEARCH METHODS

REPLICATION

• A replicate is an independent sample unit

• Equal probability of having a treatment applied to it

• The level at which the treatment is applied

• Any measurement made below the level of treatment is a subsample

• A psuedoreplicate is a subsample used as a replicate

Page 17: IUFRO SPDC WORKSHOP: RESEARCH METHODS

DIFFERENCES WITHIN VS BETWEEN

Page 18: IUFRO SPDC WORKSHOP: RESEARCH METHODS

DIFFERENCES WITHIN VS BETWEEN

Page 19: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

3. Take an equal number of samples within each combination of time, location,

and any other controlled variable. Putting samples in “representative” or

“typical” places is not sampling.

Page 20: IUFRO SPDC WORKSHOP: RESEARCH METHODS

IMPLICATIONS FOR DESIGNING EXPERIMENTS

• Corroboration designs requires resources allocated to acquiring both evidence

supporting and evidence contradicting our hypothesis to satisfy the test criteria

• Falsification experiments concentrate resources to provide maximum opportunity

to detect counter examples

Page 21: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

4. To test whether a condition has an effect, collect samples both where the

condition is present and where the condition is absent, but all else is the same.

An effect can only be demonstrated by comparison with a control.

Page 22: IUFRO SPDC WORKSHOP: RESEARCH METHODS

NEST PREDATION AND EDGE EFFECTS

Page 23: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

5. Carry out some preliminary sampling to provide a basis for evaluation of

sample design and statistical analysis options. Those who skip this step because

they do not have enough time usually end up losing time.

Page 24: IUFRO SPDC WORKSHOP: RESEARCH METHODS

POWER OF A STATISTICAL TEST

TYPE I & TYPE II ERRORS AND POWER EFFECT SIZE AND SAMPLE SIZE

Page 25: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

6. Verify that your sampling device or method is sampling the population you

think you are sampling, and with equal and adequate efficiency over the entire

range of sampling conditions to be encountered. Variation in efficiency of

sampling from area to area biases among-area comparisons.

Page 26: IUFRO SPDC WORKSHOP: RESEARCH METHODS

YELLOWSTONE GRIZZLY BEARS

TRADITIONAL BARREL TRAP NOVEL TRAIL CAM “CAPTURE”

Page 27: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

7. If the area to be sampled has a large-scale environmental pattern, break the

area up into relatively homogeneous subareas and allocate samples to each in

proportion to the size of the subarea. If it is an estimate of total abundance

over the entire area that is desired, make the allocation proportional to the

number of organisms in the subarea.

Page 28: IUFRO SPDC WORKSHOP: RESEARCH METHODS

BLOCKING

Page 29: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

8. Verify that your sample unit size is appropriate to the size, densities, and

spatial distributions of the organisms you are sampling. Then estimate the

number of replicate samples required to obtain the precision you want.

Page 30: IUFRO SPDC WORKSHOP: RESEARCH METHODS

SIZE FACTORS

Page 31: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

9. Test your data to determine whether the ERROR (residual) variation is

homogeneous, normally distributed, and independent of the mean. If it is not,

as is frequently [occasionally] encountered in field data, then (a) appropriately

transform the data, (b) use a distribution-free [robust] statistical procedure, (c)

use a sequential sampling design, (d) test against simulate H0 data; [or (e) use

a mixed modeling approach].

Page 32: IUFRO SPDC WORKSHOP: RESEARCH METHODS

GREEN’S 10 PRINCIPLES

10. Having chosen the best [most powerful] statistical method to test your

hypothesis, stick with the result. An unexpected or undesired result is NOT a

valid reason for rejecting the method and hunting for a “better” one.

Page 33: IUFRO SPDC WORKSHOP: RESEARCH METHODS

WHAT IS A STATISTICAL HYPOTHESIS?

• A statistical hypothesis is an assumption about a population parameter

• This assumption may or may not be true

• Hypothesis testing refers to the formal procedures used by statisticians to fail to

reject or reject statistical hypotheses

Page 34: IUFRO SPDC WORKSHOP: RESEARCH METHODS

RESEARCH HYPOTHESES

• Statistical hypotheses are about facts,

measurements

• Mean tail length of cat in the USA =

mean tail length of cats in Europe

• Research hypothesis is about how

nature is or how nature works

• Cats evolved tails because tails have

a survival advantage; specifically,

when cats fall from high places, tails

help them right themselves, so that

they land on their feet

Page 35: IUFRO SPDC WORKSHOP: RESEARCH METHODS

STATISTICAL HYPOTHESES

• Null Hypothesis

• There is no differences between all of the means

• µ1 = µ2 = µ3

• Alternative Hypothesis

• At least one mean is different

• µ1 µ2 = µ3

• µ1 = µ2 µ3

• µ1 µ2 µ3

• Specific Hypothesis

• General Hypothesis

Page 36: IUFRO SPDC WORKSHOP: RESEARCH METHODS

STATISTICAL TEST

• We construct our “Test” statistic assuming the Null hypothesis is true

• If the Null hypothesis is true, the test statistic should be 0 (no difference)

• Most likely (and hopefully) our test statistic will be >> 0

• Because we have a sample we have sampling error

• We learned from Ting-Ru that the sampling error causes differences in our

estimates of the mean

Page 37: IUFRO SPDC WORKSHOP: RESEARCH METHODS

STATISTICAL TEST

• So we could obtain a test statistic >> 0, because of sampling error

• Therefore, we assess, given the variability in our population, what is the

probability that a difference as large as we have observed, is due to sampling error

• If that probability is small, then we assume the difference is not due to sample

error, but due to our treatment, and we conclude that we have significant

treatment effects

Page 38: IUFRO SPDC WORKSHOP: RESEARCH METHODS

TAKE AWAYS

• Research has a Design Phase and an Analysis Phase

• Predictions from Research Hypotheses are tested using Statistical Hypotheses

• Skipping steps to save time may cost you more time

• Unexpected results may reflect bad research design, bad hypotheses, or novel discoveries

• Controlling the first 2 lead to the 3rd

Page 39: IUFRO SPDC WORKSHOP: RESEARCH METHODS

REFLECTIVE WRITING #XX

• What logistical blocks do you have in your study? Have you missed or avoided any of

Green’s Principles?