monday, november 11 statistical power

Monday, November 11

Statistical Power

Monday, November 12

Statistical Power

•It teaches you about the importance of effect size, (gamma).

Monday, November 12

Statistical Power

•It teaches you about the importance of effect size, (gamma).•It helps put the risk of Type I error, (alpha) into perspective.

Monday, November 12

Statistical Power

•It teaches you about the importance of effect size, (gamma).•It helps put the risk of Type I error, (alpha) into perspective.

•It helps you appreciate the value of the sample size, N.

Monday, November 12

Statistical Power

•It teaches you about the importance of effect size, (gamma).•It helps put the risk of Type I error, (alpha) into perspective.

•It helps you appreciate the value of the sample size, N.•It simply makes you a better person.

Monday, November 12

Statistical Power

•It teaches you about the importance of effect size, (gamma).•It helps put the risk of Type I error, (alpha) into perspective.

•It helps you appreciate the value of the sample size, N.•It simply makes you a better person.

= x f (N)

= x f (N)

Let’s sample from HSB universe of N=300 where we know the population values.

What causes you to decide on a value of under H1?

What causes you to decide on a value of under H1?

The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”!

What causes you to decide on a value of under H1?

The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”!

Would you pay attention to Stanley Kaplan raising SAT’s by 10 points? 20 points? 30 points? …

What causes you to decide on a value of under H1?

The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”!

Would you pay attention to Stanley Kaplan raising SAT’s by 10 points? 20 points? 30 points? …

So you set the value of alternative hypothesis at a point thatyou care about -- a value of practical significance.

What causes you to decide on a value of under H1?

The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”!

Would you pay attention to Stanley Kaplan raising SAT’s by 10 points? 20 points? 30 points? …

So you set the value of alternative hypothesis at a point thatyou care about -- a value of practical significance.

Power analysis says, if the effect is of that magnitude, whatis the risk that I will fail to detect it, by failing to reject thenull hypothesis.

If I really care about a 5-point difference, then this is bad news.

“Reality”

H0 True H0 False

Dec

isio

n Reject H0

Don’t Reject H0 Yeah!

Yeah!

Yeah!

Type I Error

“Reality”

H0 True H0 False

Dec

isio

n Reject H0

Don’t Reject H0 Yeah!

Yeah!

Yeah!

Type I Error Yeah!

Type II Error

1 -

The ability to avoid Type II error(fail to reject H0 that should be rejected).

Ordinarily, one is well advised to take the largest samplethat is practical and then determine if this sample has adequatepower for detecting a difference large enough to be of interest.

Researchers often strive for power 80 with = .05. More often,however, one finds that power is low even for detecting differenceslarge enough to be of practical importance.

Problem

You develop a new measure of social efficacy for adolescentgirls, with 24 items on a 3-point scale. The scale seems tohave = 18, and = 16.

You are asked to evaluate a new program to promote socialefficacy in adolescent girls, and want to use your scale. You sample 16, but alas find that the sample mean of 22 does not allow you to reject the null hypothesis at =.05.

You’re really really frustrated because you think that a 4-pointgain is meaningful. What should your next steps be?

= x f (N) = N 1/2

= 4/16 = .25N = 16

= 1.0

= x f (N) = N 1/2

= 4/16 = .25N = 16

= 1.0

What would it take for power = .80?

N = ( / )2

N = (2.8 / .25)2 = 125.44

What can you do to increase power?

•Increase n

What can you do to increase power?

•Increase n

•Decrease measurement error

What can you do to increase power?

•Increase n

•Decrease measurement error

•Increase , say, from .05 to .10 (or fiddle with tails*)

What can you do to increase power?

•Increase n

•Decrease measurement error

•Increase , say, from .05 to .10 (or fiddle with tails*)

*not advised

What can you do to increase power?

•Increase n

•Decrease measurement error

•Increase , say, from .05 to .10 (or fiddle with tails*)

•Increase the magnitude of the effect

*not advised

As sample size increases, the magnitude of the sampling error decreases; at a certainpoint, there are diminishing returns of increasing sample size to decrease sampling error.

You are a better person because now you appreciate this better!