Power of a testPower of a test

The powerpower of a test (against a specific alternative value)• Is the probability that the test

will reject the null hypothesis when the alternative is true

• In practice, we carry out the test in hope of showing that the null hypothesis is false, so high power is important

H0 True

H0

FalseReject

Fail to reject

Type I Correct

Correct

Type II

Power

Suppose H0 is true – what if we decide to

fail to reject it?

Suppose H0 is false – what if we decide to

reject it?

Suppose H0 is true – what if we decide to reject

it?

Suppose H0 is false – what if

we decide to fail to reject it?

We correctly We correctly reject a false Hreject a false H00!!

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use = .05.

What are the hypotheses?H0: p = .7

Ha: p < .7

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use = .05.

Find p and p.p = .7p = .0458

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use = .05.

What is the probability of committing a Type I error? = .05

.7

H0: p = .7Ha: p < .7 = .05For what values of the sample proportion would you reject the null hypothesis?

Invnorm(.05,.7,.0458) =.625

= .05

p?

So if we get p-

hat=.625 or less, we

would reject H0.

H0: p = .7Ha: p < .7

We reject H0 and decide that p<.7.

Suppose that pa is 0.6.

What is the probability of committing a Type II error?

Where did this

number come from?

I selected a number that

was less than .7

.7

.6

= .05Reject

What is a type II error?

= ?

How can we find this area?What is the standard deviation of this curve?

Normalcdf(.625,∞,.6,.0458) =.293

failing to reject H0 when the alternative is true

What is the power of the test?

Power = 1 - .293= .707

.7

.6

= .05

=.293

What is the definition of

power?The probability that the test correctly rejects H0

Power = ?

Power - the probability

that the test correctly

rejects H0, if p = .6, is .707

Is power a conditional probability?

Suppose we select .55 as the alternative proportion (p).

a)What is the probability of the type II error?

b) What is the power of the test?

= normalcdf(.625,∞, .55,.0458) = .051

.7

.6

.55

= .05

Power = 1 - .051= .949

What happened to the power of the

test when the difference |p0 – pa| is

increased?

Suppose we select .65 as the alternative proportion (p).

a) What is the probability of the type II error?

b) What is the power of the test?

Power = 1 - .707= .293

.7

.6

.65

= .05

= normalcdf(.625,∞, .65,.0458) = .707

What happened to the power

when the difference |p0-pa|

is decreased?

Power

Suppose that we change alpha to 10%.

Using pa = .6, what would happen to the probability of a type II error and the power of the test?

.7

.6

= .05 = .1

Power

The probability of the type II error () decrease and power increased, BUT BUT the probability of a type I error also also increasedincreased.

= .1836

Power = .8164

What happens to What happens to , , , & , & power when the sample size power when the sample size

is increased?is increased?Reject H0Fail to Reject H0

p0

pa

P(type II) decreases

when n increases

Power

Power increases when n

increases

p0

pa

Reject H0Fail to Reject H0

Power = 1 -

Recap:What affects the power of a test?

As |p0 – pa| increases, power increases

As increases, power increases

As n increases, power increases

Facts:Facts:• The researcher is free to determine the

value of .• The experimenter cannot control , since it

is dependent on the alternate value.• The ideal situation is to have as small as

possible and power close to 1. (Power > .8)• As increases, powerpower increases. (But also

the chance of a type I error has increased!)• Best way to increase power, without

increasing , is to increase the sample sizesample size

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. In a test, vaccinated people were exposed to the flu. Is this claim too high?

Identify the decision:

a) You decide that the proportion of vaccinated people who do not get the flu is less than 70% when it really is not.

Type I Error

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. In a test, vaccinated people were exposed to the flu. Is this claim too high?

Identify the decision:

b) You decide that the proportion of vaccinated people who do not get the flu is less than 70% when it really is.

Correct –

Power!!