october 16, 2002experimental design1 principles of experimental design october 16, 2002 mark conaway

October 16, 2002 Experimental Design 1

Principles of Experimental Design

October 16, 2002Mark Conaway


Use examples to illustrate principles

• Reference:Maughan et al. (1996) Effects of Ingested Fluids on Exercise Capacity and on Cardiovascular and Metabolic Responses to Prolonged Exercise in Man. Experimental Physiology, 81, 847-859.

• From paper summary– “The present study examined the effects of

ingestion of water and two dilute glucose-electrolyte drinks on exercise performance and ….”


Process of Experimental Design

• What’s the research question?– effect on exercise capacity...

• What treatments to study? – control group (no liquid intake) vs water vs 2

types of dilute glucose-electrolyte solutions• What are the levels of the treatments?

– Paper describes exact composition of solutions• How to measure the outcome of interest

– Exercise capacity: time to exhaustion on stationary cycle


Entire Process of Experimental Design

• Process of design relies heavily on researchers’ knowledge of the field, though statistical principles can help

– Do we need a “no liquid” control group?– Is “time to exhaustion” a valid measure of

exercise capacity?


Statistical DOE: Allocate treatments to experimental material to...

• Remove systematic biases in the evaluation of the effects of the treatments – “unbiased estimates of treatment effects”

• Provide as much information as possible about the treatments from an experiment of this size– “precision”


Statistical DOE

• Remove bias, obtain maximum precision, keeping in mind – simplicity/feasibility of design– natural variation in experimental units– generalizability


Focus on comparative experiments

• Treatments can be allocated to the experimental units by the experimenter

• Other types of studies also have these as goals but:– Methods for achieving goals (unbiased estimates,

precision) in comparative experiments rely on having treatments under control of experimenter


Back to example

• 4 treatments– no water (N)– water (W)– isotonic glucose-electrolyte(I)– hypotonic glucose-electrolyte (H)

• Outcome: time to exhaustion on bike• Pool of subjects available for study


Design 1: subjects select treatment

• Does this method of allocation achieve the goals?

• Possible that this method induces biases in comparisons of treatments– e.g. Would “naturally” better athletes choose

electrolytes?– e.g. Would more competitive athletes choose

electrolytes?


Design 1A: Investigators assign treatments

• “Systematically”– Everyone on Monday gets assigned no water– Tuesday subjects get water only...

• “Nonsystematically”:– Whatever I grab out of the cooler...

• Again possible that this method induces biases in comparisons of treatments


What are the sources of the biases?

• Key point: Bias in evaluating treatments due to allocating different treatments to different types of subjects– e.g., “better” riders get electrolyte– so differences between treatments mixed up with

differences between riders• To have unbiased estimates of effects of

treatment, need to have “comparable groups”


Randomization is key to having comparable groups

• Assign treatments at random– Note: Draw distinction between “random” and

“non-systematic”• Randomization is key element for removing

bias• In principle, creates comparable groups even

on factors not considered by the investigator


Completely randomized design

• Randomly assign treatments to subjects– Generally assign treatments to equal numbers of

subjects• Does this give us the most information

(precision) about the treatments?• Get precise estimates by comparing

treatments on units that are as similar as possible.


Randomized block designs (RBD)General

• Group units into subgroups (blocks) such that units within blocks are more homogeneous than in the group as a whole

• Randomly assign treatments to units within subgroups (blocks)


Randomized block designs in exercise example

• Do an initial “fitness screen” - let subjects ride bike (with water?) until exhaustion.

• Arrange subjects in order of increasing times (fitness)

• F1, F2, F3, F4 F5, F6,F7,F8 F9,F10,F11,F12

Block 1 Block 2 Block 3


Randomized block designs in exercise example

• Randomly assign treatments to units within

F1, F2, F3, F4 F5, F6,F7,F8 F9,F10,F11,F12

I H N W W N I H H W I N

Block 3Block 2Block 1


Advantages of RBD

• If variable used to create blocks is highly related to outcome, generally get much more precision than a CRD without doing a larger experiment

• Essentially guarantees that treatments will be compared on groups of subjects that are comparable on initial level of fitness


Disadvantages of RBD

• Now require 2 assessments per subject if block in this way

• Note: Could use some other measure of initial fitness that doesn’t require an initial assessment on the bike


Can take idea further

• Could group by more than one variable• Each blocking variable

– Adds complexity– Might not increase precision if grouping

variable is not sufficiently related to outcome


Repeated measures designs/Cross-over trials

• Natural extension of idea in RBD: want to compare treatments on units that are as similar as possible

• Subjects receive every treatment • Most common is ``two-period, two-treatment''

– Subjects are randomly assigned to receive either• A in period 1, B in period 2 or• B in period 1, A in period 2


Repeated measures designsCross-over Designs

• Important assumption: No carry-over effects– effect of treatment received in each period

is not affected by treatment received in previous periods.

• To minimize possibility of carry-over effects– ‘`wash-out'' time between the periods in

which treatments are received.


Cross-over designs: Example

• Cross-over was done in actual experiment

• Each of 12 subjects observed under each condition

• Randomize order. • One week period between

observations.


Cross-over designs: Example

• Illustrates the importance of– ``wash-out period'' and – randomizing/balancing the order that

treatments are applied.


In general, which design?• Is the natural variability within a subject likely

to be small relative to the natural variability across subjects?– More similarity within individuals or between

individuals?• Are there likely to be carry-over effects?• Are there likely to be ``drop-outs''?• Is a cross-over design feasible?


Which design?• No definitive statistical answer to the

question.• Answer depends on knowledge of

– experimental material and – the treatments to be studied

october 16, 2002experimental design1 principles of experimental design october 16, 2002 mark conaway

Documents