anova, continued

ANOVA, ContinuedANOVA, Continued

PSY440

July 1, 2008

Quick Review of 1-way ANOVA

• When do you use one-way ANOVA?• What are the components of the F Ratio?• How do you calculate the degrees of freedom for a one-

way ANOVA?• What are the null and alternative hypotheses in a typical

one-way ANOVA?• What are the assumptions of ANOVA?• Why can’t you test a “one-tailed” hypothesis with

ANOVA?• Questions before we move on?

The structural model and ANOVA• The structural model is all about deviations

Score

(X)

Group mean

(M)

Grand mean

(GM)

Score’s deviation

from group mean

(X-M)

Group’s mean’s

deviation from

grand mean

(M-GM)

Score’s deviation from grand mean

(X-GM)

SStotal =SSWithin +SSBetween

X −GM( )∑ 2= X−M( )∑ 2

+ M −GM( )∑ 2

1 factor ANOVA

Null hypothesis: H0: all the groups are equal

XBXA XC

XA = XB = XCAlternative hypotheses

HA: not all the groups are equal

XA ≠ XB ≠ XC XA ≠ XB = XC

XA = XB ≠ XC XA = XC ≠ XB

The ANOVA tests this one!!

Planned Comparisons

• Simple comparisons

• Complex comparisons

• Bonferroni procedure– Use more stringent significance level for each

comparison

Which follow-up test?

• Planned comparisons– A set of specific comparisons that you “planned” to do in advance

of conducting the overall ANOVA – don’t actually need to calculate the “omnibus” ANOVA F statistic in order to test planned comparisons.

– General rule of thumb, don’t exceed the number of conditions that you have (or even stick with one fewer), & make sure comparisons are orthogonal (more on this in a minute)

• Post-hoc tests– A set of comparisons that you decided to examine only after you

find a significant (reject H0) ANOVA

Planned Comparisons

• Different types– Simple comparisons - testing two groups– Complex comparisons - testing combined groups– Bonferroni procedure

• Use more stringent significance level for each comparison

• Basic procedure:– Within-groups population variance estimate (denominator)– Between-groups population variance estimate of the two groups of

interest (numerator)– Figure F in usual way

Post-hoc tests

• Generally, you are testing all of the possible comparisons (rather than just a specific few)– Different types

• Tukey’s HSD test (use if testing only pairwise comparisons)

• Scheffe test (use if testing more complex comparisons)

• Others (Fisher’s LSD, Neuman-Keuls test, Duncan test)

– Generally they differ with respect to how conservative they are.

Planned Comparisons & Post-Hoc Tests as ContrastsPlanned Comparisons & Post-Hoc Tests as Contrasts

• A contrast is basically a way of assigning numeric values to your grouping variable in a manner that allows you to test a specific difference between two means (or between one mean and a weighted average of two or more other means).

Contrasts for follow-up tests

Think of the formula for independent samples t-tests:

€

t =(X A − X B ) − (μA − μB )

sX A −X B

Typically, we are testing the null hypothesis that µA- µB= 0, so the numerator reduces to the difference between the two sample means.This is a simple contrast comparing two groups. The contrast is anarray of multipliers defining a linear combination of the means. In this case, the array is (1,-1). The mean of sample A is multiplied by 1, and the mean of sample B is multiplied by -1, and the two productsare added together, forming the numerator of the t statistic.


In the case of one-way ANOVA, contrasts can be used in a similar way to test specific statements about the equality or inequality of particular means in the analysis.

In a one-way ANOVA with three groups, contrasts such as: (0,1,-1) , (1,0,-1), or (1,-1,0) define linear combinations of means that test whether a specified pair of means is equal, while ignoring the third mean.

The null hypothesis is that the linear combination of means is equal to zero (similar to independent-samples t-test).


Contrasts can also define linear combinations of means to test more complex hypotheses (such as mean one is equal to the average of mean two and mean three).

For example: (1, -.5, -.5)

The weights must sum to 0, because the null hypothesis is always that the linear combination of means defined by the contrast is equal to 0.


Follow-up tests are not always independent of each other. The hypothesis that A>B=C is not independent of the hypothesis that A>B>C, because both include the inequality A>B.

For planned comparisons, contrasts tested should be independent.

Independence of contrasts can be tested by summing the cross-products of the elements (see next slide)


Consider two contrasts testing first the pairwise comparison between means a and c, and second whether mean b is equal to the average of means and c:

(1,0,-1) (.5,-1,.5)Sum of cross-products = (1*.5) + (0*-1) + (-1*.5) =.5+0+(-.5)=0, so these are independent.Consider contrasts testing one pairwise comparison (b vs. c) and one

comparison between average of means a and b vs. mean c:(.5,.5,-1) (0,1,-1)Sum of cross-products = (.5*0) + (.5*1) + (-1*-1) =0+.5+1=1.5, so

these contrasts are not independent (both are comparing b and c).


SPSS will let you specify contrasts for planned comparisons that are not independent, but this is not recommended practice.

If you want to test several dependent contrasts, you should use a post-hoc correction such as Scheffe.

If you want to test all pairwise comparisons, you can use Tukey’s HSD correction.

Fixed vs. Random Factors in ANOVAFixed vs. Random Factors in ANOVA• One-way ANOVAs can use grouping variables that are fixed or

random.

– Fixed: All levels of the variable of interest are represented by the variable (e.g., treatment and control, male and female).

– Random: The grouping variable represents a random selection of levels of that variable, sampled from a population of levels (e.g., observers).

– For one-way ANOVA, the math is the same either way, but the logic of the test is a little different. (Testing either that means are equal or that the between group variance is 0)

ANOVA in SPSS

• Let’s see how to do a between groups 1-factor ANOVA in SPSS (and the other tests too)

• Analyze=>Compare Means=>One-Way ANOVA

• Your grouping variable is the “factor” and your continuous (outcome) variable goes in the “dependent list” box.

• Specify contrasts for planned comparisons• Specify any post-hoc tests you want to run• Under “options,” you can request descriptive statistics

(e.g., to see group means)

Within groups (repeated measures) ANOVA

• Basics of within groups ANOVA– Repeated measures

– Matched samples

• Computations• Within groups ANOVA in SPSS

Example

• Suppose that you want to compare three brand name pain relievers. – Give each person a drug, wait 15 minutes, then ask them to keep

their hand in a bucket of cold water as long as they can. The next day, repeat (with a different drug)

• Dependent variable: time in ice water

• Independent variable: 4 levels, within groups– Placebo

– Drug A

– Drug B

– Drug C

Statistical analysis follows design

• The 1 factor within groups ANOVA:

– One group

– Repeated measures

– More than 2 scores per subject


• The 1 factor within groups ANOVA:

– One group

– Repeated measures

– More than 2 scores per subject

– More than 2 groups

– Matched samples

– Matched groups

- OR -

Within-subjects ANOVA

Placebo Drug A Drug B Drug C

3 4 6 7

0 3 3 6

2 1 4 5

0 1 3 4

0 1 4 3

XA =2.0

SSA =8.0

XB =4.0

SSB =6.0

XC =5.0

SSC =10.0

XP =1.0

SSP =8.0

XBXA XCXP

• n = 5 participants• Each participates

in every condition (4 of these)

Within-subjects ANOVA

– Step 2: Set your decision criteria

– Step 3: Collect your data

– Step 4: Compute your test statistics • Compute your estimated variances (2 steps of partitioning used)

• Compute your F-ratio

• Compute your degrees of freedom (there are even more now)

– Step 5: Make a decision about your null hypothesis

• Hypothesis testing: a five step program– Step 1: State your hypotheses

Step 4: Computing the F-ratio

• Analyzing the sources of variance– Describe the total variance in the dependent measure

• Why are these scores different? • Sources of variability

– Between groups

– Within groups

XBXA XCXP

• Individual differences

• Left over variance (error)Because we use the same people in each condition, we can figure out

how much of the variability comes from the individuals and remove it

from the analysis

Because we use the same people in each condition, we can figure out

how much of the variability comes from the individuals and remove it

from the analysis

Partitioning the varianceTotal variance

Stage 1

Between groups variance Within groups variance


Stage 1


Stage 2Between subjects variance Error variance


Stage 1



1) Treatment effect2) Error or chance

(without individual differences)

1) Individual differences

2) Other error

1) Other error (without individual differences)


Because we use the same people in each condition, none of this

variability comes from having different people in different

conditions

Because we use the same people in each condition, none of this

variability comes from having different people in different

conditions

• The F ratio– Ratio of the between-groups variance estimate to the

population error variance estimate

Step 4: Computing the F-ratio

Observed variance

Variance from chanceF-ratio = =

MSBetween

MSError


Stage 1



1) Treatment effect2) Error or chance

(without individual differences)


2) Other error

1) Other error (without individual differences)


F =MSBetween

MSError


Stage 1


SSTotal = X−GM( )∑ 2

dfTotal =N −1

SSWithin = SSeach group∑dfWithin = dfeach group∑

SSBetween = n X−GM( )∑ 2

dfbetween =#groups−1

Partitioning the variancePlacebo Drug A Drug B Drug C

3 4 6 7

0 3 3 6

2 1 4 5

0 1 3 4

0 1 4 3

XA =2.0SSA =8.0

XB =4.0SSB =6.0

XC =5.0SSC =10.0

XP =1.0SSP =8.0

GM =3.0

SSTotal = X−GM( )∑ 2= 3−3( )2 + ...+ 3−3( )2

dfTotal =N −1=20 −1=19SSTotal =82


dfbetween =#groups−1=4 −1=3

=5 1 − 3( )2

+ 5 2 − 3( )2

+ 5 4 − 3( )2

+ 5 5 − 3( )2

=50.0

SSWithin = SSeach group∑ =SSP +SSA +SSB +SSC =32

dfWithin = dfeach group∑ =4 + 4 + 4 + 4 =16


Stage 1




dfTotal =N −1




PWhat is ?


3 4 6 7

0 3 3 6

2 1 4 5

0 1 3 4

0 1 4 3

XA =2.0

SSA =8.0

XB =4.0

SSB =6.0

XC =5.0

SSC =10.0

XP =1.0

SSP =8.0

The average score for each person

GM =3.0

Between subjects variance

SSBetweenSubs = ngroups P −GM( )∑ 2


3 4 6 7

0 3 3 6

2 1 4 5

0 1 3 4

0 1 4 3

XA =2.0

SSA =8.0

XB =4.0

SSB =6.0

XC =5.0

SSC =10.0

XP =1.0

SSP =8.0SSBetweenSubs = ngroups P −GM( )∑ 2

dfbetweenSubs =nsubjects −1=5−1=4

PWhat is ?

The average score for each person

P20

4 =5.012

4 =3.012

4 =3.08

4 =2.08

4 =2.0

GM =3.0

=4 5 − 3( )2

+ 4 3 − 3( )2

+ 4 3 − 3( )2

+

4 2 −3( )2 + 4 2 −3( )2

=24

Between subjects variance


Stage 1




dfTotal =N −1




dfbetweenSubs =nsubjects −1



3 4 6 7

0 3 3 6

2 1 4 5

0 1 3 4

0 1 4 3

XA =2.0

SSA =8.0

XB =4.0

SSB =6.0

XC =5.0

SSC =10.0

XP =1.0

SSP =8.0SSError =SSWithin −SSBetweenSubsGM =3.0

Error variance

SSError =32 −24 =8

dferror = Nscores −ngroups( )− nsubjects −1( )

dferror = 20 −4( )− 5 −1( ) =12


Stage 1




dfTotal =N −1




SSError =SSWithin −SSBetweenSubs

dfbetweenSubs =nsubjects −1


dferror = Nscores −ngroups( )− nsubjects −1( )

Partitioning the variance

Mean Squares (Variance)

SSBetween =50.0

dfbetween =3

MSBetween =50.03

=16.67 MSError =812

=0.67

Between groups variance Error variance

SSError =8

dferror =12

Now we return to variance. But, we call it Means Square (MS)

Now we return to variance. But, we call it Means Square (MS)

Recall:Recall:

variance =SSdf


Stage 1



dfTotal =19SSTotal =82

SSWithin =32dfWithin =16

SSBetween =50dfbetween =3

SSError =8dferror =12

SSBetweenSubs =24

dfbetweenSubs =4

MSBetween =6.0

MSError =0.67

Within-subjects ANOVA• The F table

– Need two df’s• dfbetween (numerator)

• dferror (denominator)

– Values in the table correspond to critical F’s

• Reject the H0 if your computed value is greater than or equal to the critical F

– Separate tables for 0.05 & 0.01

Do we reject or failto reject the H0?

Do we reject or failto reject the H0?

– From the table (assuming 0.05) with 3 and 12 degrees of freedom the critical F = 3.89.

– So we reject H0 and conclude that not all groups are the same

F =MSBetween

MSError

=16.67

0.67= 24.89

Within-subjects ANOVA in SPSS

– Setting up the file– Running the analysis– Looking at the output


• Setting up the file:– Each person has one line of data, with each of the

“conditions” represented as different variables.– Our chocolate chip cookie data set is a good

example. Each type of cookie (jewel, oatmeal, and chips ahoy) is a different “condition” that everyone in the sample experienced.

– We created three “sets” of similar variables, with data from each person entered into all three sets of variable fields.


• Running the analysis:– Analyze=>General Linear Model=>Repeated

Measures– Define your within-subjects factor (give it a name

and specify number of levels - this is the number of variables it will be based on, then click on define and select variables for each level of the factor).

– Can request descriptives and contrasts (though the contrasts are defined in a different manner)


• Interpreting the output– Output is complex, and generally set up to

accommodate much more complex designs than one-way repeated measures ANOVA, so for current purposes much can be ignored.

– Scroll down to where it says: Tests of Within-Subjects Effects & find between group and error sums of squares, df, F, and Sig. for “sphericity assumed.”

Factorial ANOVA

• Basics of factorial ANOVA– Interpretations

• Main effects

• Interactions

– Computations

– Assumptions, effect sizes, and power

– Other Factorial Designs• More than two factors

• Within factorial ANOVAs


• The factorial (between groups) ANOVA:

– More than two groups

– Independent groups

– More than one Independent variable

Factorial experiments

• Two or more factors– Factors - independent variables– Levels - the levels of your independent variables

• 2 x 3 design means two independent variables, one with 2 levels and one with 3 levels

• “condition” or “groups” is calculated by multiplying the levels, so a 2x3 design has 6 different conditions

B1 B2 B3

A1

A2

Factorial experiments

• Two or more factors (cont.)– Main effects - the effects of your independent variables

ignoring (collapsed across) the other independent variables

– Interaction effects - how your independent variables affect each other

• Example: 2x2 design, factors A and B

• Interaction:– At A1, B1 is bigger than B2

– At A2, B1 and B2 don’t differ

Results

• So there are lots of different potential outcomes:• A = main effect of factor A• B = main effect of factor B• AB = interaction of A and B

• With 2 factors there are 8 basic possible patterns of results:

5) A & B6) A & AB7) B & AB8) A & B & AB

1) No effects at all2) A only3) B only4) AB only

2 x 2 factorial design

Condition

mean

A1B1

Condition

mean

A2B1

Condition

mean

A1B2

Condition

mean

A2B2

A1 A2

B2

B1

Marginal means

B1 mean

B2 mean

A1 mean A2 mean

Main effect of B

Main effect of A

Interaction of ABWhat’s the effect of A at B1?What’s the effect of A at B2?

Main effect of AMain effect of BInteraction of A x B

A

B

A1 A2

B1

B2

Main Effect of A

Main Effect of B

60

45

45

30

6030

6030

A

A1 A2

Dependent

Vari

able

B1

B2

√X

X

Examples of outcomes


A

B

A1 A2

B1

B2

Main Effect of A

Main Effect of B

45

60

30

45

3030

6060

A

A1 A2

Dependent

Vari

able

B1

B2

√X

X



A

B

A1 A2

B1

B2

Main Effect of A

Main Effect of B

45

45

45

45

6030

3060

A

A1 A2

Dependent

Vari

able

B1

B2

√X

X



A

B

A1 A2

B1

B2

Main Effect of A

Main Effect of B

45

45

30

30

3030

6030

A

A1 A2

Dependent

Vari

able

B1

B2

√

√

√


Factorial Designs

• Benefits of factorial ANOVA (over doing separate 1-way ANOVA experiments)– Interaction effects

– One should always consider the interaction effects before trying to interpret the main effects

– Adding factors decreases the variability– Because you’re controlling more of the variables that influence

the dependent variable– This increases the power of the statistical tests

Basic Logic of the Two-Way ANOVA

• Same basic math as we used before, but now there are additional ways to partition the variance

• The three F ratios– Main effect of Factor A (rows)

– Main effect of Factor B (columns)

– Interaction effect of Factors A and B


Stage 1


Stage 2Factor A variance Factor B variance Interaction variance

Figuring a Two-Way ANOVA

• Sums of squares

SSWithin = SSgroups∑

SSA = nAgroups(X∑

Agroups

−GM )2

SSAB =SSBetween − SSA +SSB( )

SSBetweenGroups = n(XAllGroups∑ −GM )2

SSTotal = (X∑ −GM )2

SSB = nBgroups(X∑

Bgroups

−GM )2


• Degrees of freedom

dfA =NA −1

dfB =NB −1

dfAB =NConditions −dfA −dfB −1

dfWithin = dfeach group∑Number of levels of A

Number of levels of A

Number of levels of B

Number of levels of B


• Means squares (estimated variances)

MSA =SSA

dfA

MSB =SSB

dfB

MSAB =SSAB

dfAB

MSWithin =SSWithin

dfeach group


• F-ratios

FA =MSA

MSWithin

FB =MSB

MSWithin

FAB =MSAB

MSWithin


• ANOVA table for two-way ANOVA

ExampleFactor B: Arousal Level

LowB1

MediumB2

HighB3

FactorA:

Task

Difficulty

A1

Easy

3

1

1

6

4

2

5

9

7

7

0

0

4

3

1

A2

Difficult

3

0

0

2

0

3

8

3

3

3

0

0

0

5

0

XA1B1 =3

SSA1B1 =18

nA1B1 =5

XA2 B1 =1 XA2 B2 =4

XA1B2 =6 XA1B3 =9

XA2 B3 =1

SSA1B2 =28 SSA1B3 =26

SSA2 B1 =8 SSA2 B2 =20 SSA2 B3 =20

nA1B2 =5 nA1B3 =5

nA2 B1 =5 nA2 B2 =5 nA2 B3 =5


LowB1

MediumB2

HighB3

FactorA:

Task

Difficulty

A1

Easy

3

1

1

6

4

2

5

9

7

7

0

0

4

3

1

A2

Difficult

3

0

0

2

0

3

8

3

3

3

0

0

0

5

0

XA1B1 =3

SSA1B1 =18

nA1B1 =5

XA2 B1 =1 XA2 B2 =4

XA1B2 =6 XA1B3 =9

XA2 B3 =1

SSA1B2 =28 SSA1B3 =26

SSA2 B1 =8 SSA2 B2 =20 SSA2 B3 =20

nA1B2 =5 nA1B3 =5

nA2 B1 =5 nA2 B2 =5 nA2 B3 =5

GM =4N =30

SSTotal =360

SSBetweenGroups = n(XAllGroups∑ −GM )2

=5(3 − 4)2 + 5(6 − 4)2 + 5(9 − 4)2 +5(1−4)2 + 5(4 −4)2 + 5(1−4)2

=240

SSWithin = SSgroups∑=18 + 28 + 26 + 8 + 20 + 20

=120


LowB1

MediumB2

HighB3

FactorA:

Task

Difficulty

A1

Easy

3

1

1

6

4

2

5

9

7

7

0

0

4

3

1

A2

Difficult

3

0

0

2

0

3

8

3

3

3

0

0

0

5

0

XA1B1 =3

SSA1B1 =18

nA1B1 =5

XA2 B1 =1 XA2 B2 =4

XA1B2 =6 XA1B3 =9

XA2 B3 =1

SSA1B2 =28 SSA1B3 =26

SSA2 B1 =8 SSA2 B2 =20 SSA2 B3 =20

nA1B2 =5 nA1B3 =5

nA2 B1 =5 nA2 B2 =5 nA2 B3 =5

GM =4N =30

SSTotal =360

SSA = nAgroups(X∑

Agroups

−GM )2

=15 6 − 4( )2

+ 15 2 − 4( )2

=120

SSB = nBgroups(X∑

Bgroups

−GM )2

=10 2 − 4( )2

+10 5 − 4( )2

+10 5 − 4( )2

=60


LowB1

MediumB2

HighB3

FactorA:

Task

Difficulty

A1

Easy

3

1

1

6

4

2

5

9

7

7

0

0

4

3

1

A2

Difficult

3

0

0

2

0

3

8

3

3

3

0

0

0

5

0

XA1B1 =3

SSA1B1 =18

nA1B1 =5

XA2 B1 =1 XA2 B2 =4

XA1B2 =6 XA1B3 =9

XA2 B3 =1

SSA1B2 =28 SSA1B3 =26

SSA2 B1 =8 SSA2 B2 =20 SSA2 B3 =20

nA1B2 =5 nA1B3 =5

nA2 B1 =5 nA2 B2 =5 nA2 B3 =5

GM =4N =30

SSTotal =360

SSBetweenGroups =240

SSA =120

SSB =60

SSAB =SSBetween − SSA +SSB( )=240 − 120 + 60( )

=60

Example

Source SS df MS FBetween

A

B

AB

120

60

60

1

2

2

120

30

30

24.0

6.0

6.0

Within

Total

120

360

24 5

√

√

√

Assumptions in Two-Way ANOVA

• Populations follow a normal curve• Populations have equal variances• Assumptions apply to the populations that go with

each cell

Extensions & Special Cases of Factorial ANOVA

• Three-way and higher ANOVA designs

• Repeated measures ANOVA

• Mixed factorial ANOCA

Factorial ANOVA in Research Articles

A two-factor ANOVA yielded a significant main effect of voice, F(2, 245) = 26.30, p < .001. As expected, participants responded less favorably in the low voice condition (M = 2.93) than in the high voice condition (M = 3.58). The mean rating in the control condition (M = 3.34) fell between these two extremes. Of greater importance, the interaction between culture and voice was also significant, F(2, 245) = 4.11, p < .02.

Repeated Measures & Mixed Factorial ANOVA

• Basics of repeated measures factorial ANOVA– Using SPSS

• Basics of mixed factorial ANOVA– Using SPSS

• Similar to the between groups factorial ANOVA– Main effects and interactions– Multiple sources for the error terms (different

denominators for each main effect)

Example

• Suppose that you are interested in how sleep deprivation impacts performance. You test 5 people on two tasks (motor and math) over the course of time without sleep (24 hrs, 36 hrs, and 48 hrs). Dependent variable is number of errors in the tasks.– Both factors are manipulated as within subject

variables– Need to conduct a within groups factorial

ANOVA

Example

Factor B: Hours awake24B1

36B2

48B3

Factor A:

Task

A1

Motor

0

1

0

4

0

0

3

1

5

1

6

5

5

9

5

A2

Math

1

1

0

3

1

1

2

1

2

3

4

6

6

4

4

Example

Source SS df MS F A

Error (A)

1.20

13.13

1

4

1.20

3.28

0.37

B

Error (B)

AB

Error (AB)

104.60

6.10

2.60

8.10

2

8

2

8

52.30

0.76

1.30

1.01

69.00

1.29

Example

• It has been suggested that pupil size increases during emotional arousal. A researcher presents people with different types of stimuli (designed to elicit different emotions). The researcher examines whether similar effects are demonstrated by men and women.– Type of stimuli was manipulated within subjects

– Sex is a between subjects variable

– Need to conduct a mixed factorial ANOVA

Example

Factor B: StimulusNeutral

B1

PleasantB2

AversiveB3

FactorA:

Sex

A1

Men

4

3

2

3

3

8

6

5

3

8

3

3

2

6

1

A2

Women

3

2

4

1

3

6

4

6

7

5

2

1

6

3

2

Example

Source SS df MS FBetween

A

Error (A)

0.83

20.00

1

8

0.83

2.50

0.33

Within

B

AB

Error (B)

58.10

0.07

39.20

2

2

16

29.00

0.03

2.45

11.85

0.01

anova, continued

Documents