AP StatisticsAP StatisticsChapter 11 NotesChapter 11 Notes

Significance Test & Significance Test & HypothesisHypothesis

Significance test: a formal procedure Significance test: a formal procedure for comparing observed data with a for comparing observed data with a hypothesis whose truth we want to hypothesis whose truth we want to assess.assess.

Hypothesis: a statement about a Hypothesis: a statement about a population parameter.population parameter.

Null (HNull (Hoo) and Alternative ) and Alternative (H(Haa) Hypotheses) Hypotheses

The null hypothesis is the statement being The null hypothesis is the statement being tested in a significance test.tested in a significance test. Usually a statement of “no effect”, “no Usually a statement of “no effect”, “no

difference”, or no change from historical values.difference”, or no change from historical values. The significance test is designed to assess the The significance test is designed to assess the

strength of evidence strength of evidence againstagainst the null hypothesis. the null hypothesis. The alternative hypothesis is the claim The alternative hypothesis is the claim

about the population that we are trying to about the population that we are trying to find evidence find evidence forfor..

Example: One-sided testExample: One-sided test Administrators suspect that the weight of Administrators suspect that the weight of

the high school male students is the high school male students is increasing. They take an SRS of male increasing. They take an SRS of male seniors and weigh them. A large study seniors and weigh them. A large study conducted years ago found that the conducted years ago found that the average male senior weighed 163 lbs.average male senior weighed 163 lbs. What are the null and alternative hypotheses?What are the null and alternative hypotheses? HHoo: : μμ = 163 lbs. = 163 lbs. HHaa: : μμ > 163 lbs. > 163 lbs.

Example: Two-sided testExample: Two-sided test How well do students like block How well do students like block

scheduling? Students were given scheduling? Students were given satisfaction surveys about the traditional satisfaction surveys about the traditional and block schedules and the block score and block schedules and the block score was subtracted from the traditional score.was subtracted from the traditional score. What are the null and alternative hypotheses?What are the null and alternative hypotheses? HHoo: : μμ = 0 = 0 HHaa: : μμ ≠ 0 ≠ 0

*You must pick the type of test you want to *You must pick the type of test you want to do do beforebefore you look at the data.* you look at the data.*

Be sure to define the parameter.Be sure to define the parameter.

Conditions for Conditions for Significance TestsSignificance Tests

SRSSRS Normality (of the sampling distribution)Normality (of the sampling distribution)

For means:For means: 1. population is Normal or1. population is Normal or 2. Central Limit Theorem (n > 30) or2. Central Limit Theorem (n > 30) or 3. sample data is free from outliers or strong 3. sample data is free from outliers or strong

skewskew For proportions:For proportions:

np np >> 10, n(1 - p) 10, n(1 - p) >> 10 10 Independence (N Independence (N >> 10n) 10n)

Test StatisticTest Statistic Compares the parameter stated in HCompares the parameter stated in Hoo with with

the estimate obtained from the sample.the estimate obtained from the sample. Estimates that are far from the parameter Estimates that are far from the parameter

give evidence against Hgive evidence against Hoo..

For now we’ll us the z-test. For now we’ll us the z-test.

P-ValueP-Value Assuming that HAssuming that H00 is true, the is true, the

probablility that the observed outcome probablility that the observed outcome (or a more extreme outcome) would (or a more extreme outcome) would occur is called the occur is called the p-valuep-value of the test. of the test. Small p-value = strong evidence against HSmall p-value = strong evidence against H00..

How small does the p-value need to be?How small does the p-value need to be? We compare it with a significance level (We compare it with a significance level (αα – –

level) chosen beforehand.level) chosen beforehand. Most commonly Most commonly αα = .05 = .05

P-value continuedP-value continued If the p-value is as small or smaller than If the p-value is as small or smaller than αα, then , then

the data are “statistically significant at level the data are “statistically significant at level αα”.”. Ex: Ex: αα = .05 = .05 If the p-value is < .05, then there is less than a 5% If the p-value is < .05, then there is less than a 5%

chance of obtaining this particular sample chance of obtaining this particular sample estimate if Hestimate if H00 is true. is true. Therefore we reject the null hypothesis.Therefore we reject the null hypothesis.

If the p-value is > .05, our result is not that If the p-value is > .05, our result is not that unlikely to occur.unlikely to occur. Therefore we fail to reject the null hypothesis.Therefore we fail to reject the null hypothesis.

If done by hand, the p-value must be doubled when If done by hand, the p-value must be doubled when performing a 2-sided test. The calculator will already performing a 2-sided test. The calculator will already display this doubled p-value if you choose the 2-sided display this doubled p-value if you choose the 2-sided option.option.

Confidence vs. Confidence vs. SignificanceSignificance

Performing a level Performing a level αα 2-sided 2-sided significance test is the same as significance test is the same as performing a 1 – performing a 1 – αα confidence interval confidence interval and seeing if and seeing if μμ00 falls outside of the falls outside of the interval.interval.

e.g. If a 99% CI estimated a mean to be e.g. If a 99% CI estimated a mean to be (4.27, 5.12), then a significance test (4.27, 5.12), then a significance test testing the null hypothesis Htesting the null hypothesis H00:: µ = 4 µ = 4 would be significant at would be significant at αα = .01. = .01.

Reminders about Reminders about Significance TestsSignificance Tests

1. Don’t place too much importance on 1. Don’t place too much importance on “statistically significant”.“statistically significant”. Smaller p-value = stronger evidence against Smaller p-value = stronger evidence against

HH00 2.Statistical significance is not the same as 2.Statistical significance is not the same as

practical importance.practical importance. 3. Don’t automatically use a test…examine 3. Don’t automatically use a test…examine

the data and check the conditions.the data and check the conditions. 4. Statistical inference is not valid for 4. Statistical inference is not valid for

badly-produced data.badly-produced data.

Mistakes in significance Mistakes in significance testingtesting

Type I error:Type I error: Reject HReject H00 when H when H00 is actually true. is actually true.

Type II Error:Type II Error: Fail to reject HFail to reject H00 when H when H00 is actually is actually

false.false.

Errors ContinuedErrors Continued

Errors continuedErrors continued The significance level The significance level αα is the is the

probability of making a Type I error.probability of making a Type I error. Power: The probability that a fixed level Power: The probability that a fixed level

αα significance test will reject H significance test will reject H00 when a when a particular alternative value of the particular alternative value of the parameter is true.parameter is true.

Ways to increase the power.Ways to increase the power. Increase Increase αα Decrease Decrease σσ Increase nIncrease n