ANOVA: PART II

Last week

Introduced to a new test: One-Way ANOVA

ANOVA’s are used to minimize family-wise error: If the ANOVA is statistically significant:

One or more groups are significantly different in your test Follow-up testing required to determine which groups

If the ANOVA is not statistically significant: All groups are statistically similar (can’t be confident

enough that any differences aren’t simply random sampling error)

No follow-up testing is needed, all will also be non-significant

These ‘rules’ are true for all ANOVA’s

Tonight… We’ll finish testing for ‘group differences’

with: Factorial ANOVA

Any ANOVA including more than one IV Introduce interactions

‘Hint at’: ANCOVA

ANOVA’s including a co-variate Repeated Measures ANOVA

ANOVA’s for dependent, paired, or related groups

Something to think about…

One-Way ANOVA’s ‘look’ very similarly to t-tests They compare individual groups, one by one Compare means between the levels of one variable

Factorial ANOVA’s, ANCOVA’s, and RM ANOVA’s are actually more closely related to multiple linear regression All of these tests use the General Linear Model

Y = mX + b We will be comparing groups, but we’ll be doing it

using the General Linear Model behind the scenes This should make more sense later in the lecture…

Factorial ANOVA example

Example dataset contains patients that were diagnosed with breast cancer Key Variables:

Survival Time in Months – Survival time after diagnosis/treatment (our dependent variable)

Tumor Size – Subjects grouped by initial tumor size (< 2 cm, 2-5 cm, or > 5 cm, our independent variable)

We want to know if Tumor Size effects the cancer Survival Time

Picture of our analysis

If we ran a One-way ANOVA, it would look something like this:

Will compare mean survival time between each of the three groups and determine if they are significantly different

Tumor Size

< 2 cmMean

Survival1

2-5 cmMean

Survival2

> 5 cmMean

Survival3

However…

This is a good analytical plan – but in reality more variables influence cancer survival than just tumor size We would like to examine several different IV’s

For example, lymph nodes are essentially a means for cancer to travel around the body. If the cancer spreads, survival time decreases Lymph node involvement may also influence survival time

Instead of 1 – we could run 2 one-way ANOVA’s…

New Plan

ANOVA #1:

ANOVA #2:

Tumor Size

< 2 cmMean

Survival1

2-5 cmMean

Survival2

> 5 cmMean

Survival3

Lymph Node Involvement?

NoMean

Survival1

YesMean

Survival2

This will tell us if Tumor Size impacts survival

This will tell us Lymph Node involvement impacts survival

But…

Now our two tests examine how two independent variables influence our dependent variable but…

…what if Tumor Size and Lymph Node involvement are related to each other? Remember collinearity? This is a similar concept

Called Interaction in ANOVA (more on this later) By running two separate tests we are ignoring

this potential interaction of the variables

Factorial ANOVA’s Running a factorial ANOVA allows us to run one

ANOVA that: 1) Uses two (or more) independent variables 2) Examines the potential for interaction between our

IV’s

Again, notice here that Factorial ANOVA’s are very similar to multiple regression We’re using two or more independent variables and

one dependent variable The only difference is the Factorial ANOVA tests for

group differences while the Regression predicts

Terminology In Factorial ANOVA’s the independent variables are

called ‘main effects’ Tumor Size is a main effect Lymph Node involvement is a main effect This is just the lingo, think of it as ‘main’ variable

Factorial ANOVA’s are usually described by how many levels each main effect has How many levels does our Tumor Size variable have? How many levels does our Lymph Node variable have? We have a 3 x 2 Factorial ANOVA

New Analysis: 6 Groups

Instead of two tests – everything is tested at once Let’s run this model in SPSS…

Tumor Size

< 2 cm 2-5 cm > 5cm

Lymph Node

Involvement?

NoMean

Survival1Mean

Survival2Mean

Survival3

YesMean

Survival4Mean

Survival5Mean

Survival6

Main Effect #1

Main Effect #2

Two-Way ANOVAOr

3 X 2 Factorial ANOVA

Factorial ANOVA use the General Linear Model tab The “Univariate” description refers to the number of dependent variables you have – not

independent Again you can see a clear separation of One-Way ANOVA’s (that are like t-tests) and all the

other ANOVA’s (that are like multiple regression). SPSS puts them under different options.

Dependent Variable is Survival Time

Fixed Factors = IV’s Tumor Size Lymph Node

Post-Hoc Tests same as before

Options similar (can get means, SD, for each group, etc…)

For Post-Hoc options now I can select all of my independent variables (not just 1)

Use the “Plots” option every time you run a Factorial ANOVA – we’ll discuss why later

It really doesn’t matter which IV you enter first, but I tend to “Separate Lines” by the IV with the fewest levels, in this case Lymph Nodes

Estimated Marginal Means

Factorial ANOVA’s have another new addition under the ‘Options’ button The Post-Hoc tab will use Pairwise Comparisions to

compare the levels of our independent variables (Tumor Size and Lymph Node) and look for differences

However, the Estimated Marginal Means are required when you want to make comparisons using BOTH of your two main effects at the same time Lets use our picture…

What Post-Hoc Tests do…

Tumor Size

< 2 cmMean

Survival1

2-5 cmMean

Survival2

> 5 cmMean

Survival3

Post-Hoc tests will compare Tumor Size between ONLY these three groups (ignoring the effect of Lymph Node Involvement)

And the post-hoc test will compare Lymph Nodes, ignoring Tumor Size

Lymph Node Involvement?

NoMean

Survival1

YesMean

Survival2

That is NOT the only thing we are interested in!

Estimated Marginal Means vs Post-Hoc

In other words: Post-Hoc tests: Test for group difference one

independent variable at a time Adequate for One-Way ANOVA’s Like a t-test

Estimated Marginal Means: Test for group differences while considering all variables in your model More appropriate for Factorial ANOVA’s

How do they Work? This option will ‘adjust’ the group means of Tumor

Size for Lymph Node Involvement If everyone had the same amount of lymph node

involvement – what effect would Tumor Size have?

These are ‘estimated’ means because they do NOT exist in real life – SPSS is saying, “What if…?”

Tumor Size

< 2 cm 2-5 cm > 5cm

Lymph Node

Involvement?

NoMean

Survival1Mean

Survival2Mean

Survival3

YesMean

Survival4Mean

Survival5Mean

Survival6

X1

X2

X3

Estimated Marginal Means

These are ‘estimated’ means because they do NOT exist in real life – SPSS is saying, “What if…?”

Has SPSS lost it’s mind? How can we ‘adjust’ for another variable? How can we ‘estimate’ how an independent

variable will effect the dependent variable?

SPSS is using the General Linear Model to ‘predict’

The General Linear Model in ANOVA

What SPSS is really doing: Survival Time = TumorSizeX1 + LymphX2 + b Estimated Marginal Means function by assuming

a constant value for one of the X’s – in this case it is Lymph Node Involvement: Survival Time1 = (<2cm)X1 + (Constant)X2 + b Survival Time 2 = (2-5cm)X1 + (Constant)X2 + b Survival Time 3 = (>5cm)X1 + (Constant)X2 + b

The constant for the IV is similar to the mean lymph node involvement for the sample

Estimated Marginal Means This option will ‘adjust’ the group means

of Tumor Size for Lymph Node Involvement If everyone had the same amount of lymph

node involvement – what effect would Tumor Size have?

Tumor Size

< 2 cm 2-5 cm > 5cm

Lymph Node

Involvement?

NoMean

Survival1Mean

Survival2Mean

Survival3

YesMean

Survival4Mean

Survival5Mean

Survival6

X1

X2

X3

Estimated Survival Time for everyone with a Tumor Size < 2cm (adjusting for Lymph Node Involvement)

Initial Questions on Estimated Marginal Means?

Move over all factors into the box on the right Display Estimated

Means for… Check the box that

says “Compare Main Effects” This will generate

p-values for the groups – adjusted for the other variables

Factorial Outputs

Once you run the test the Output file can be overwhelming since there is so much to look at. Move through one output box at a time

For this 1 analysis, I requested: Descriptive statistics A Means plot Estimated Marginal Means for Tumor Size,

Lymph Node Involvement, and our Interaction Post-Hoc tests and pair-wise comparisons …And the actual ANOVA

Ready for the results…?

Sample Sizes

You want to try and have all groups be equal in size, but it’s almost impossible to do! How many subjects in our study?

826 + 274 + 42= 1142= 871+ 271

Descriptives for all 6 groups

Main Effect #1:Tumor Size

Main Effect #2Lymph Node?

ANOVA Results

Take note of df, F, and Significance for each Main Effect (we have 2) and each Interaction (we have 1)

Also, you will report the Error (or ‘overall’) df = 1136

Initial interpretation: Survival Time is significantly impacted by Tumor Size

p < 0.001 Survival Time is not significantly impacted by Lymph Node Involvement

p = 0.207 Survival Time is significantly impacted by the interaction of Tumor Size and Lymph Node Involvement

p = 0.016

Next Step?Estimated Marginal Means to determine WHERE the

differences are!

EMM for Tumor Size Estimated Marginal Means/Pairwise

Comparisons for Tumor Size (adjusting for Lymph Node)

Looks just like a Post-Hoc test – except it’s based on the Estimated Marginal Means!

EMM for Tumor Size Interpretation:

Those with Tumors <= 2 cm tended to survive longer than those in the 2-5 cm group (p = 0.001) and > 5 cm group (p = 0.006). There was not a significant difference between the 2-5 cm and > 5 cm group (p = 0.254).

?

Do I need to look at the Estimated Marginal Means for Lymph Node involvement? Was the Main Effect of Lymph Node

statistically significant? NO! But you can verify that for yourself:

Interactions

Aside from the Main Effects, we did have a statistically significant interaction we need to consider

Interactions: A combination of Main Effects. Indicates that

two IV’s are related to each other Similar to: collinearity, confounders, etc…

If you have a significant interaction: At least one of your Main Effects is influenced

by another main effect

Interactions Think of it this way…

When considered alone: As tumor size increased survival time decreased Lymph node involvement had no effect on survival time

An statistically significant interaction means… The effect that tumor size has on survival time is

MODIFIED by lymph node involvement Interactions also known as ‘effect modification’

The two variables interact to cause a different effect on survival

The effect of tumor size is dependent on lymph node involvement!

Interactions

When you have a statistically significant interaction, you have to ignore any statistically significant main effects Why? Because the interaction ‘overrides’ them So, in writing you would report the significant

main effect, but realize the interaction is MORE important

This is why we requested the means plot earlier. A plot of the means (including both the interaction terms) makes it easier to see…

If there is no interaction, these lines would be parallel

The effect of increasing tumor size would be the same for both lines

You can see clearly that Lymph Node does have an effect – but only for those with tumors larger than 2 cm

No difference

Big difference

Big difference

Interactions

In general, if the lines on your plot are NOT parallel, you probably have a significant interaction

Without effect modification/interactions, you would expect the variables to have the same impact no matter what their combination

Examples…

Example: Interaction

Figure A

0

2

4

6

8

10

12

LowPressure

ModeratePressure

HighPressure

Low Anxiety

High Anxiety

Example: NO Interaction

Figure B

0

2

4

6

8

10

LowPressure

ModeratePressure

HighPressure

Low Anxiety

High Anxiety

Example: Interaction

Figure C

0

2

4

6

8

10

12

LowPressure

ModeratePressure

HighPressure

Low Anxiety

High Anxiety

The interaction is significant…and we can see it on the plot…what now…?

Significant Interactions

In truth, it’s much easier if you do NOT find significant interactions Report only the statistically significant main

effects Use the Post-Hoc tests or Estimated Marginal Means

to see which groups are different Ignore the non-significant main effects Ignore the non-significant interaction

When you do find a significant interaction, you must now treat each group separately (in our case, consider all 6 groups)

Estimated Marginal Means Interaction Table When you request it, you will get a

pairwise comparison table for the interaction using the estimated marginal means You have to use the estimated marginal

means here because we have ‘combined’ two variables

This table will tell you which groups are significantly different – but you have to use the 95% confidence intervals!

When 95% confidence intervals for 2 groups do NOT overlap, they are significantly different (p < 0.05)

Which groups are statistically different here? Compare the two groups WITHIN each category of Tumor Size

Estimated Marginal Means Interaction

Reporting Your Findings A 3 x 2 Factorial ANOVA revealed a statistically significant

main effect for Tumor Size (F(2, 1136) = 7.719, p < 0.001). In general, as tumor size increased, survival time decreased*. The main effect of Lymph Node Involvement was not statistically significant (F(1, 1136) = 1.592, p = 0.207). However, the Tumor Size x Lymph Node interaction was statistically significant (F(2, 1136) = 4.139, p = 0.016. For individuals with tumors <2cm, lymph node involvement made no difference. For those in the 2-5cm and >5cm group, lymph node involvement further decreased survival time*.

*You could list the means/SD for each group and reveal the results of the pairwise comparisons. But most of the time this information is place in a table (with 95% CI’s).

Checklist for Factorial ANOVA’s

Look at group descriptives and sample size Examine the ANOVA output box (Tests of Between

Subjects Effects) to see df, F ratio’s, and p-values for the Main Effects and Interactions(s) If you have significant Main Effects

Use Estimated Marginal Means + Pair-wise Comparisons to see which groups are different (adjusting for the other variables in your model)

If you have a significant Interaction: Ignore the Main effects! The interaction is what is REALLY going on! Use the means Plot to see how your Main Effects work together Use the estimated marginal means interaction table and the 95%

confidence intervals to see which groups are different

Questions on the Checklist?

Factorial ANOVA’s Our example here includes a 3-level variable and a

2-level variable. As you add levels – or variables – these things will

really grow quickly Example: Besides tumor size and lymph node

involvement, progesterone receptor status (2-level; positive or negative) might also be relevant to cancer survival. I’ll add it to our ANOVA: DV = Survival Time IV = Tumor Size, Lymph Node Involvement,

Progesterone Receptor Status What kind of Factorial ANOVA is this now?

3 x 2 x 2 ANOVA

Note my NEW main effects and interactionS What would I need to do now?

QUESTIONS ON FACTORIAL ANOVA’S?

Upcoming…

In-class activity Homework:

Cronk complete 6.6 Cronk READ 6.7 – you are not required to DO this

section, just check it out Holcomb Exercises 51 and 52 (on Factorial ANOVA)

We’ll change things up next week by talking about 2 statistical concepts (rather than ‘new’ tests) Power and Effect Size ANCOVA