beginner’s guide to randomisation
Post on 23-Feb-2016
130 Views
Preview:
DESCRIPTION
TRANSCRIPT
Randomisation S1
Beginner’s guide to randomisation
Research supported by TLRI
Randomisation S2
•Experiments & the Randomisation Test:
• The difference between two medians
–Watch out for: • The ‘chance is acting alone’ explanation• How we assess the plausibility of the ‘chance
alone’ explanation – (test for ‘chance alone’)
Randomisation S3
Randomisation
iNZightVIT
What does ‘chance alone’ look like?
Randomisation S4
Did she brush her teeth?
1. She has brushed her teeth.
2. The toothbrush is dry.
3. The-toothbrush-is-dry would be highly unlikely if she had
brushed her teeth.
4. Therefore, it’s a fairly safe bet she has not brushed her teeth.
I have evidence that she has not brushed her teeth.
1. Formulate statement to test.
2. Data (information at hand).
3. Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2.?
4. Review the statement in 1. in light of 3. together with the data in 2.
Randomisation S5
Did she brush her teeth (2)?
1. She has brushed her teeth.
2. The toothbrush is wet.
3. The-toothbrush-is-wet would
NOT be surprising if she had brushed her teeth.
4. Therefore, she could have brushed her teeth.
I have no evidence that she has NOT brushed her teeth.
1. Formulate statement to test.
2. Data (information at hand).
3. Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2.?
4. Review the statement in 1. in light of 3. together with the data in 2. Or she
could have just run the brush under the tap.
Randomisation S6
The Walking Babies Experiment
Treatment Exercise Control
Does a special exercise programme lower walking age?Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315
10 male infants (& parents) were randomly assigned to one of two treatment groups.
Randomisation S7
The Walking Babies Experiment
Treatment Age (months) Exercise 9 9.5 9.75 10 11Control 13.25 11.5 12 13.5 11.5
Does a special exercise programme lower walking age?Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315
10 male infants (& parents) were randomly assigned to one of two treatment groups.
First walked without support:
Randomisation S8
The Walking Babies Experiment
Age (months)9 11 1210 13 14
Does it appear that these data provide evidence that the treatment is effective?Exercise
Control
Yes.
Randomisation S9
looking through a window with ripples in the glass
“What I see …is not quite the way it really is”
Looking at the world using data is like
Randomisation S10
The Walking Babies Experiment
Age (months)9 11 1210 13 14
Is it possible that these babies’ walking ages have nothing to do with whether they undertook the exercises or not, . . .
Exercise
Control
Randomisation S11
The Walking Babies Experiment
Age (months)9 11 1210 13 14
. . . i.e., it doesn’t matter which group they were randomly assigned to, they would still have the same walking age, . . .
Exercise
Control
Randomisation S12
The Walking Babies Experiment
Age (months)9 11 1210 13 14
. . . and so the observed difference pattern is purely and simply the result of the luck-of-the-draw as to which babies just happened by chance to be assigned to which group and nothing else?
Exercise
Control
Yes.Under this scenario we say: “Chance is acting alone”.
Randomisation S13
The Walking Babies Experiment
Possible explanation:One possible explanation for the observed difference between these two groups:
Chance is acting alone (the exercise has no effect)
Randomisation S14
The Walking Babies Experiment
Is the ‘chance alone’ explanation simply not plausible?• Would our observed difference be unlikely when chance
is acting alone? • How do we determine whether an observed
difference is unlikely when chance is acting alone?Answer: See what’s likely and what’s unlikely when chance is acting alone.
Possible explanation:One possible explanation for the observed difference between these two groups:
(the exercise has no effect) Chance is acting alone
Randomisation S15
Randomisation S16
2.25
Randomisation S17
Randomisation S18
Re-randomisation distribution of differences (under chance alone)
Is chance alone likely to generate differences as big as our difference?
Our observed difference = 2.25 months
Hardly ever, they’re almost always smaller.Tail proportion:
roughly __%
Randomisation S20
The Walking BabiesExperiment
Possible explanation:One possible explanation for the observed difference between these two groups:Chance is acting alone (the exercise has no effect)
Our tail proportion of roughly __% means: • Roughly __ times out-of-a-1000 times we get a
median difference of 2.25 months or more, when chance is acting alone.
• Under chance alone, it would be highly unlikely to get a difference equal to or bigger than our observed difference of 2.25 months.
Randomisation S21
The Walking BabiesExperiment
Possible explanation:One possible explanation for the observed difference between these two groups:Chance is acting alone (the exercise has no effect)
Our tail proportion of roughly __% means: • Roughly __ times out-of-a-1000 times we get a
median difference of 2.25 months or more, when chance is acting alone.
• Under chance alone, it would be highly unlikely to get a difference equal to or bigger than our observed difference of 2.25 months.
An observed difference of 2.25 months or greater is highly unlikely when chance is acting alone . . .
therefore, it probably isn’t. It’s a fairly safe bet chance is not acting alone.
Randomisation S22
The Walking BabiesExperiment
• We can rule out ‘chance is acting alone’ as a plausible explanation for the difference between the two groups.
• We have evidence against ‘chance is acting alone’
• We have evidence that chance is not acting alone
Possible explanation:One possible explanation for the observed difference between these two groups:Chance is acting alone (the exercise has no effect)
Randomisation S23
The Walking BabiesExperiment
• We can rule out ‘chance is acting alone’ as a plausible explanation for the difference between the two groups.
• We have evidence against ‘chance is acting alone’
• We have evidence that chance is not acting alone
Possible explanation:One possible explanation for the observed difference between these two groups:Chance is acting alone (the exercise has no effect)
If chance is not acting alone, then what else is also acting to help produce the observed difference?
Remember: Random assignment to 2 groups & each group receives different treatment.
Randomisation S24
Conclusion:Because the male infants (& parents) were randomly assigned to the groups, we may claim that the exercise was effective in lowering the walking age.
Because these subjects in this experiment were volunteers (not randomly selected), then we would need to consider carefully as to which wider group(s) this conclusion may apply.
The Walking BabiesExperiment
Randomisation S25
Did she brush her teeth?
1. She has brushed her teeth.
2. The toothbrush is dry.
3. The-toothbrush-is-dry would be highly unlikely if she had brushed her teeth.
4. Therefore, it’s a fairly safe bet she has not brushed her
teeth. I have evidence that she has
not brushed her teeth.
Steps
1. Statement to test.
2. Collect data (information).
3. Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2.?
4. Review the statement in 1. in light of 3. together with the data in 2.
Randomisation S26
Is the exercise programme effective?Steps
1. Statement to test.
2. Collect data.
3. Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2. or more?
4. Review the statement in 1. in light of 3. together with the data in 2.
1. Chance is acting alone. (The exercise has no effect.)
2. Diff between medians = 2.25 mths.
3. A difference of 2.25 months or more is highly unlikely when chance is acting alone.(Tail prop = roughly __%)
4. Therefore, it’s a fairly safe bet chance is not acting alone. We have evidence against ‘chance is acting alone’.
Randomisation S27
Steps
1. Statement to test.
2. Collect data.
3. Consider 1. and the data:If 1. is true, then what are the chances of getting data like that in 2. or more?
4. Review the statement in 1. in light of 3. together with the data in 2.
1. Chance is acting alone. (The exercise has no effect.)
2. diff medians= 2.25 mths.
3. A diff of 2.25 mths or more is highly unlikely when chance is acting alone.
4. Therefore, it’s a fairly safe bet chance is not acting alone. We have evidence against chance is acting alone.
Is the exercise programme effective?
Randomisation S28
Questions…
Randomisation S29
Two types of Inference
There are two types of inference1. Sample-to-population
eg x = 172cm so the population mean is about 172cm.
2. Experiment-to-causationeg The treatment was effective
Randomisation S30
When the tail proportion is small (less than 10%).:• the observed difference would be unlikely when
chance is acting alone . . . therefore, it’s a fairly safe bet chance is not acting alone.
• we have evidence against ‘chance-is-acting-alone’
• we have evidence that chance is not acting alone
Guidelines for assessing ‘Chance alone’
Randomisation S31
When the tail proportion is large (10% or more ) then:
• the observed difference is not unusual when chance is acting alone, therefore chance could be acting alone
• we have NO evidence against ‘chance is acting alone’
• EITHER chance could be acting alone OR something else as well as ‘chance’ COULD also be acting.
-- we do NOT have ENOUGH INFORMATION to MAKE A CALL as to which one.
Guidelines for assessing ‘Chance alone’
top related