Hypothesis TestingHypothesis Testing

… understanding statistical claims

Doc Martin is at it again!After his line of Brain Pills was shown to be no better than a placebo, Doc Martin decided to devote his business genius to marketing his new line of ski wax – “Wax 2 the Max”.In a recent competition between conventional and the new space-age W2tMax, when applied to the skis of 12 cross country skiers, 9 of the winners were using the Wax-2-the-Max skis!

Wax-2-the-Max!Wax-2-the-Max!Try it .Try it .

My wax is My wax is proven to work!proven to work!

How can we test this claim?

Either it works or it “don’t”

• The null hypothesis:– H0: The wax makes no difference

• The alternate hypothesis:– Ha: The wax improves performance

If we accept the null hypothesis, then it would mean that winning the 9 (or more) races occurred merely by chance. How likely is this?

Wax-2-the-Max

• You either win or don’t – what kind of distribution does this suggest?

• What is the probability of this happening by chance?

B(12,0.5)

P = P(9)+P(10)+P(11)+P(12)P = 0.0537+0.0161+0.0029+0.0002P = 0.0729

What’s this mean?

Bad news for Doc Martin!• The null hypothesis occurs with a

probability of about 7.3%• If the null hypothesis occurs with 5% or

lower chance then you would reject H0 and conclude that the alternate hypothesis is statistically significant

• If the null hypothesis occurs with 1% or lower chance then Ha is strongly significant

Rats! Foiled again!

Null Hypothesis and Statistical Significance• It is usual to create and test a null hypothesis

which essentially asks “how likely is the effect that we are testing” due to chance alone. For example:– In testing the claim that my ski wax provided a

significant advantage we tested the null hypothesis that is did not and the effect I claimed could be explained as a result of chance.

– A probability threshold called the statistical statistical significance levelsignificance level is set to decide to accept or reject the null hypothesis.

Significance Levels…

• Symbol denotes the significance level. An of 5% or 0.05 means that events have a 1/20 chance of occurring by chance

• US Supreme Court sets statistical significance at 2 or 3 away from the mean:– 2 = 0.0223– 3 = 0.0013

Stats 300 Causes Stress!

• An un-named student (whose initials are Carl) claims that Stats 300 causes stress. To prove this he measured the blood pressure of a SRS of 100 subjects at King’s between the ages of 18 and 36 and found a mean systolic blood pressure of 122 with a standard deviation if 12. He then took the blood pressure of the entire class and found a blood pressure of 128 and assumes the same standard deviation of 12 for each reading. Does this evidence support Carl’s claim?

• We are making the assumption that Carl’s original SRS was normally distributed as is the Stats 300 class– Null Hypothesis The Stats 300 class has the same

blood pressure as the SRS, ie: No effect on stress. :

H0: = 122

– The Alternative Hypothesis:

Ha: > 122

• We will test at the significance level by first

128 1223.122

1239

Xz

n

So … what’s this mean?

A one-sided alternative

• A blood pressure of 128 is 3.122 above the mean. Either:

– A) H0 is true and we just got 128 by chance

– B) H0 is false – STATS 300 really does cause stress!

• So how likely is A)?P(Z >= 3.122) implies

that this only occurs

with a probability of

p = 1 – 0.9991

= 0.0009! H0 is false!!

(Another way of thinking about this is that 99.91% of the readings expected would be less than 128 – getting this by chance is pretty unlikely!)

• There are three possible scenarios for the alternative hypothesis:

• H: > o

• H: < o

• H: ≠ o

One-Sided and Two-Sided Alternatives

One-sided

Two-sided

One-sided and two-sided alternative hypotheses have slightly different probability formulae

Probability Formulae…

• H: > o : P-value for H0 is P(Z ≥ z)

• H: < o : P-value for H0 is P(Z ≤ z)

• H: ≠ o : P-value for H0 is 2P(Z ≥ |z|)

Closer look … example 6.13

• Make the null hypothesis: “sample contains 0.86% of the active ingredient” or H0: = 0.86

• Alternative is H: ≠ 0.86

• P = 2P(Z ≥ |z|)

It could be more or lessSo this is a two-sidedcase

z = 4.99

The probability of the null hypothesis is less than 2P(Z≥4.99) = 2(1-1)=0!