hypotheses psy 201: statistics in psychology omnibus keep ...gfrancis/classes/psy201/l34.pdf · psy...

PSY 201: Statistics in PsychologyLecture 34

Power for Analysis of VarianceKeep it simple!

Greg Francis

Purdue University

Fall 2019

Greg Francis (Purdue University) PSY 201: Statistics in Psychology Fall 2019 1 / 33

HYPOTHESES

The null for an ANOVA is an omnibus hypothesis. It suppose nodi↵erence between any population means

H0 : µi = µj 8 i , j

the alternative is the complement

H0 : µi 6= µj for some i , j

To compute power, we have to provide the standard deviation, ↵, n’s,and specific values for the means


POWER CALCULATOR

For other power calculators, it was kind of easy to identify how poweris a↵ected by the specific alternative:

bigger di↵erences (between population means, proportions, orcorrelations) leads to more power

That is also true for ANOVA, but it can be more complicated becausethere are multiple means


POWER CALCULATOR

Consider a situation with K = 4means (one di↵erent from theothers):

We estimate the power to be0.76792


POWER CALCULATOR

Consider a situation with K = 8means (one di↵erent from theothers):

We estimate the power to be0.70688.

Power is a↵ected by the ratio ofthe variability between groupmeans and the variability withineach group (� = 1). If just onemean is di↵erent from theothers, this ratio decreases as Kgets bigger


POWER CALCULATOR

Consider a situation with K = 8means (four di↵erent from theothers):



POWER CALCULATOR

Consider a situation with K = 4means (two di↵erent from theothers):


Thus, it is not just that powerdecreases as K increases. Itdepends on the values of themeans


POWER CALCULATOR

Consider a situation with K = 4means (every mean is di↵erentfrom the others):


The biggest and smallest meansdi↵er by 0.75, just like previouscases, but that alone does notdetermine power


TRUST THE MATH

With su�cient experience, you can learn to recognize what types ofsituations produce large (or small) power

Until you get that experience, rely on the calculator (even after youget the experience you need the calculator to do the actualcomputations)

It is still the case that larger samples lead to higher power.


EXAMPLE

There are lots of mnemonic tricks to try to improve your memory.They really do work!

To compare these tricks we can use a standard memory test (Nairne,Pandeirada & Thompson, 2008):

A subject is shown a word and asked to do some kind of task. This isrepeated for 30 words.

At the end of the experiment, the subject is asked to recall as manywords as possible. Usually, this is a surprise memory task.

For each subject, we compute the proportion of recalled words.

We are interested in the mean value of the proportion across subjects.

We can compare how well di↵erent tasks influence memory.


TASKS

Pleasantness: Rate the pleasantness of the word on a scale from 1 to5.

Imagery: Rate how easy it is to form a mental image of the word on ascale from 1 to 5.

Self-reference: Rate how easily the word brings to mind an importantpersonal experience on a scale from 1 to 5.

Generation: Words are partially scrambled; unscramble and then ratethe pleasantness of the word on a scale from 1 to 5. (e.g., “iktten”)

Survival: Rate the relevance of the word for survival if you arestranded in the grasslands of a foreign land, on a scale from 1 to 5.

Intentional learning: Try to remember the words for a future memorytest.

Di↵erent subjects are assigned to di↵erent conditions


ORIGINAL RESULTS

Nairne, Pandeirada & Thompson (2008) found a big advantage forsurvival processing compared to the other methods. ni = 50 for eachgroup

F5,294 = 4.41, p = 0.00178, MSW = 0.019


NEW METHOD

Suppose that you want to further explore these kinds of memorytricks. You think that the survival processing method does wellbecause it gets subjects to be really be engaged in thinking about theword. You come up with a new method

Vacation: Rate the relevance of the word for enjoyment while onvacation at a fancy resort, on a scale from 1 to 5.

You expect that the vacation task will do about the same as thesurvival task

You worry that other details of the experiment may change the overalllevel of performance for all tasks, so you decide to repeat the fullstudy, with the addition of your new, Vacation, task. So there will beseven groups.

How do you plan an appropriate sample size?


SPECIFIC MEANS

As the values for the specific means, we can use the sample meansfound in the original study

We get them from the figure

For the Vacation task, we expect performance to be the same as theSurvival task

For the standard deviation, we can use the square root of MSW

� =p

MSW =p0.019 = 0.1378


POWER FOR ANOVA

Power is quite high (0.999) if weuse ni = 50, as in the originalstudy


POWER FOR ANOVA

If we accept power of 0.9,n = 25 subjects in each sampleis su�cient


POWER FOR ANOVA

If we accept power of 0.8,n = 20 subjects in each sampleis su�cient


CONTRASTS

However, just a significant ANOVA is not enough for what we arestudying

We want to show that the Vacation task is better than most of theother tasks (not including the Survival task)

We also want to show that the Survival task is better than most ofthe other tasks (not including the Vacation task)

For our hypothesis test, we will set up two contrasts to test Vacationand Survival against the other tasks:

We need to include those contrasts in the power analysis (moresubjects)


CONTRASTS


NULL

You might also want to demonstrate that memory performance is thesame for the Survival and Vacation tasks (after all, your idea is thatboth tasks are engaging, so they should have similar performance)

Unfortunately, hypothesis testing cannot show that two groups haveequal means (that would be proving the null hypothesis)

Thus, we cannot set a sample size so that we are sure the Survivaland Vacation tasks are equally e↵ective for improving memory


ANOTHER EXAMPLE

Bushman (2018) investigated the “weapons e↵ect”: the merepresence of weapons can increase aggression

Subjects were assigned to view a set of images of one type:

Criminals, Soldiers, Police in military gear, Police in regular gear,Olympians with guns, Police in plain clothes

Afterwards, complete a word fragment task:

I C H O EI K II M U E RI C T

each fragment can be completed to form an aggressive ornon-aggressive word

Count how many aggressive words are formed: measure of aggressivethoughts


EXAMPLE IMAGES

on human targets prime aggressive thoughts, whereas weaponsused on inanimate targets do not prime aggressive thoughts.

This research also has practical implications. There are plentyof guns in the world, both in the real world and in the virtual world.In the United States, for example, there are about 90 guns forevery 100 citizens (MacInnis, 2007). Although the United Statesis only about 4% of the world’s population, U.S. civilians possessabout 31% of the world’s guns. Guns are even more common inthe mass media. For example, acts of gun violence in PG-13 films(for ages 13þ) has more than doubled since the rating was intro-duced in 1985 (Bushman, Jamieson, Weitz, & Romer, 2013), andthis upward trend continues in more recent years (Romer, Jamie-son, & Jamieson, 2017). Frequent exposure to guns in the realworld and virtual world can produce a weapons effect.

The primary limitation of the present research is that neitherexperiment included a measure of aggressive behavior, whichis difficult to measure in online experiments. Future researchshould test whether the findings obtained in these onlineexperiments replicate in field and laboratory experiments.

In discussions of gun violence, the weapons effect is oftenoverlooked. As Leonard Berkowitz noted, the trigger of a guncan also pull the finger, producing a weapons effect. Theoreti-cally, weapons increase aggression because they prime aggres-sive thoughts in memory. This research shows that the merepresence of a gun can prime or activate aggressive thoughtsin memory, even if a “good guy” is holding the gun.

Appendix

One sample image for each type of image—criminals withguns, soldiers in military gear with guns, police officers in mil-itary gear with guns, police officers in regular gear with guns,Olympians carrying guns, and police officers in plainclothes

without guns, respectively. To enhance generalizability, therewere eight photos for each type of image in each experiment.

Bushman 731





Appendix



Bushman 731





Appendix



Bushman 731

Acknowledgment

I would like to thank Kelly Dillon and Kevin Collier for their assis-

tance with this research. I would also like to thank Pablo Brinol for

suggesting the Olympians with guns condition in Experiment 2.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to

the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, author-

ship, and/or publication of this article.

References

Anderson, C. A., Benjamin, A. J., Jr., & Bartholow, B. D. (1998).

Does the gun pull the trigger? Automatic priming effects of

weapon pictures and weapon names. Psychological Science, 9 ,

308–314. doi:10.1111/1467-9280.00061

Anderson, C. A., Carnagey, N. L., & Eubanks, J. (2003). Exposure to

violent media: The effects of songs with violent lyrics on aggres-

sive thoughts and feelings. Journal of Personality and Social

Psychology, 84, 960–971. doi:10.1037/0022-3514.84.5.960

Anderson, C. A., Carnagey, N. L., Flanagan, M., Benjamin, A. J.,

Eubanks, J., & Valentine, J. C. (2004). Violent video games: Spe-

cific effects of violent content on aggressive thoughts and beha-

vior. Advances in Experimental Social Psychology, 36 , 199–249.

doi:10.1016/S0065-2601(04)36004-1

Bartholow, B. D., Anderson, C. A., Carnagey, N. L., & Benjamin, A.

J. (2005). Individual differences in knowledge structures and prim-

ing: The weapons priming effect in hunters and nonhunters. Jour-

nal of Experimental Social Psychology, 41, 48–60. doi:10.1016/j.

jesp.2004.05.005

Benjamin, A. J., Jr., Kepes, S., & Bushman, B. J. (in press). Effects of

weapons on aggressive thoughts, angry feelings, hostile appraisals,

and aggressive behavior: A meta-analytic review of the weapons

effect literature. Personality and Social Psychology Review.

Berkowitz, L. (1990). On the formation and regulation of anger and

aggression: A cognitive–neoassociationistic analysis. American

Psychologist, 45, 494–503. doi:10.1037/0003-066X.45.4.494

Berkowitz, L., & LePage, A. (1967). Weapons as aggression-eliciting

stimuli. Journal of Personality and Social Psychology, 7 , 202–207.

doi:10.1037/h0025008

Bushman, B. J., Jamieson, P. E., Weitz, I., & Romer, D. (2013). Gun

violence trends in movies. Pediatrics, 132 , 1014–1018. doi:10.

1542/peds.2013-1600

Carlson, M., Marcus-Newhall, A., & Miller, N. (1990). Effects of

situational aggression cues: A quantitative review. Journal of Per-

sonality and Social Psychology, 58, 622–633. doi:10.1037/0022-

3514.58.4.622

CBS News. (2012, December 21). NRA: ‘Only way to stop a bad guy

with a gun is with a good guy with a gun.’ Retrieved from http://

washington.cbslocal.com/2012/12/21/nra-only-way-to-stop-a-

bad-guy-with-a-gun-is-with-a-good-guy-with-a-gun/

Cohen, J. (1988). Statistical power analysis for the behavioral

sciences (2nd ed.). New York, NY: Academic Press.

732 Social Psychological and Personality Science 9(6)

Acknowledgment







Funding



References




308–314. doi:10.1111/1467-9280.00061




Psychology, 84, 960–971. doi:10.1037/0022-3514.84.5.960





doi:10.1016/S0065-2601(04)36004-1





jesp.2004.05.005










doi:10.1037/h0025008



1542/peds.2013-1600




3514.58.4.622








Acknowledgment







Funding



References




308–314. doi:10.1111/1467-9280.00061




Psychology, 84, 960–971. doi:10.1037/0022-3514.84.5.960





doi:10.1016/S0065-2601(04)36004-1





jesp.2004.05.005










doi:10.1037/h0025008



1542/peds.2013-1600




3514.58.4.622









DATA

Roughly n = 100 for each image set


MANY TESTS

Conclusions are based on many contrastsI Significant ANOVA (some di↵erence across image types)I Contrast between people with guns vs. plainclothes police (no guns):

Weapon is importantI Contrast between Olympians vs. Others: Person must intend to hurt

othersI Contrast between people with guns vs. Olympians: Weapon must be to

hurt people

Conclusion: only guns intended to shoot human targets primeaggressive thoughts


REPLICATION STUDY

Suppose you want to replicatethis study. To estimate poweryou use the means and standarddeviation of the original finding.You want to see what happens ifyou use a similar sample size asthe original study, n = 100, foreach sample

We enter the information in theANOVA Power calculator


REPLICATION STUDY

We set up each of the contrast tests in the ANOVA Power calculator:


REPLICATION STUDY

When we hit the “Calculate power” button, we get:

Each test has around a 65% chance of rejecting its H0, but theprobability of all tests rejecting the H0 for one set of samples is onlyaround 40%.


ADJUSTING ↵

Bushman (2018) was concerned about multiple tests increasing TypeI error, so he set ↵ = 0.025 for the second contrast


ADJUSTING ↵

When we hit the “Calculate power” button, we get:

The power of the second contrast drops a bit. The other powerestimates change, but that is just a side e↵ect of the calculations. Wecould increase the number of iterations to avoid these changes.

The power for all tests drops from 40% to around 33%


SAMPLE SIZE

How big a sample size do we need to have 80% power?

n = 223, which means a total of 6⇥ 223 = 1338 subjects

The power values would be distributed across the tests as:


SIMPLE IS BETTER

If your conclusion depends on many hypothesis tests producingsignificant results, you should design your study to take into accountall of those tests

Adding tests always lowers power

Complicated experiments require much larger samples than simpleexperiments

Lots of studies that are published are woefully underpowered becausethey do not consider these details of experimental design


CONCLUSIONS

power for ANOVA

power for contrasts

simple is better


NEXT TIME

Dependent ANOVA

Contrasts

Leverage relationships


hypotheses psy 201: statistics in psychology omnibus keep ...gfrancis/classes/psy201/l34.pdf · psy...

Documents