General Linear Model 2

Intro to ANOVA

Questions ANOVA makes assumptions about error for

significance tests. What are the assumptions?

What might happen (why would it be a problem) if the assumption of {normality, equality of error, independence of error} turned out to be false?

What is an expected mean square? Why is it important?

Why do we use the F test to decide whether means are equal in ANOVA?

Questions (2) Correctly interpret ANOVA summary tables. Find correct values of critical F from tabled

values for a given test. Suppose someone has worked out that a one-

way ANOVA with 6 levels has a power of .80 for the overall F test. What does this mean?

Describe (make up) a concrete example of a one-way ANOVA where it makes sense to use an overall F test. Explain why ANOVA (not t, chi-square or something else) is the best method for the analysis.

New Distributions

• So far, the normal (z) and its short, fat relative, the t distribution.

• The normal has two children, chi-square ( ) and F.

• Chi-square is made of the sum of v squared deviations from the unit normal. It essentially show the sampling distribution of the variance.

• F is the ratio of two chi-squares.

2

ANOVA Assumptions

• Recall we can partition total SS into between (treatment) and within (error) SS. No assumptions needed.

• To conduct tests about population effects, have to make assumptions:

1. Within cells (treatments) error is normal.

2. Homogeneity of error variance.

3. Independent errors.

Assumptions

• Normality – sampling distribution of means,variances; not bad if N is large; e.g. reaction time

• Homogeneity – pooled estimate of population value. Where are means different? Assumed equal error for each. E.g., ceiling effects in training.

• Independence – sampling distribution again; e.g., cheating on exam, nesting (schools, labs)

Mean Square Between Groups

• Mean square = SS/df = = variance estimate.

• MS between =

• E(MS between) =

• If there is no treatment effect, MS between = error variance.

• If there is a treatment effect, MS between is bigger than error variance.

vv

2)(

1J

betweenSS (J treatments)

1

2

2

J

nj

jj

e

Mean Square Within Groups

• MS within =

• E(MS within) =• Expected mean square for error is .

Expected mean square for treatment is same plus treatment effect: .

• When there is no treatment effect, between and within estimate same thing.

JN

withinSS

(N is total sample size and J is number of groups.)

2e

2e

1

2

2

J

nj

jj

e

Review

ANOVA makes assumptions about error for significance tests. What are the assumptions?

What might happen (why would it be a problem) if the assumption of {normality, equality of error, independence of error} turned out to be false?

What is an expected mean square? Why is it important?

The F Test (1)

• Suppose

• The null is equivalent to:

• If the null is true, then

,0:0 jH for all j

,0:1 jH for some j

),1(~ JNJFwitinMS

betweenMS

The ratio of the two variance estimates will be distributed as F with J-1 and N-J degrees of freedom.

j...21

The F Test (2) ),1(~ JNJFwitinMS

betweenMS

This is a big deal because we can use variance estimates to test the hypothesis that any number of population means are equal. Equality of means is same as testing population treatment effect(s).

For a treatment effect to be detected, F must be larger than 1. F is one-tailed in the tables which show upper tail values of F

given the two df.

6543210Obtained F (2 and 10 df)

1.0

0.8

0.6

0.4

0.2

0.0

p va

lue

(sig

nific

ance

)

No Effect (n.s.) Signifcant

(Alpha = .05)

F Table – Critical ValuesNumerator df: dfB

dfW 1 2 3 4 5

5 5%

1%

6.61

16.3

5.79

13.3

5.41

12.1

5.19

11.4

5.05

11.0

10 5%

1%

4.96

10.0

4.10

7.56

3.71

6.55

3.48

5.99

3.33

5.64

12 5%

1%

4.75

9.33

3.89

6.94

3.49

5.95

3.26

5.41

3.11

5.06

14 5%

1%

4.60

8.86

3.74

6.51

3.34

5.56

3.11

5.04

2.96

4.70

Review

Why do we use the F test to decide whether means are equal in ANOVA?

Suppose we have an ANOVA design with 3 cells and 5 people per cell. What is the critical value of F at alpha = .05?

Calculating F – 1 Way ANOVA

Sums of squares (squared deviations from the mean) tell the story of variance. The simple ANOVA designs have 3 sums of squares.

2)( XXSS ijtot

2)( jijW XXSS

2)( XXnSS jjB

WBTOT SSSSSS

The total sum of squares comes from the distance of all the scores from the grand mean. This is the total; it’s all you have.

The within-group or within-cell sum of squares comes from the distance of the observations to the cell means. This indicates error.

The between-cells or between-groups sum of squares tells of the distance of the cell means from the grand mean. This indicates IV effects.

Computational Example: Caffeine on Test ScoresG1: Control G2: Mild G3: Jolt

Test Scores

75 80 70

77 82 72

79 84 74

81 86 76

83 88 78

Means

79 84 74

SDs (N-1)

3.16 3.16 3.16

G1 75 79 16

Control 77 79 4

M=79 79 79 0

SD=3.16 81 79 4

83 79 16

G2 80 79 1

M=84 82 79 9

SD=3.16 84 79 25

86 79 49

88 79 81

G3 70 79 81

M=74 72 79 49

SD=3.16 74 79 25

76 79 9

78 79 1

Sum 370

Total Sum of Squares

2)( XXSS ijtot

G1 75 79 16

Control 77 79 4

M=79 79 79 0

SD=3.16 81 79 4

83 79 16

G2 80 84 16

M=84 82 84 4

SD=3.16 84 84 0

86 84 4

88 84 16

G3 70 74 16

M=74 72 74 4

SD=3.16 74 74 0

76 74 4

78 74 16

Sum 120

Within Sum of Squares

2)( jijW XXSS ijXjX 2)( jij XX

G1 79 79 0

Control 79 79 0

M=79 79 79 0

SD=3.16 79 79 0

79 79 0

G2 84 79 25

M=84 84 79 25

SD=3.16 84 79 25

84 79 25

84 79 25

G3 74 79 25

M=74 74 79 25

SD=3.16 74 79 25

74 79 25

74 79 25

Sum 250

Between Sum of Squares

2)( XXnSS jjB jX X 2)( XX j

Source SS df MS F

Between Groups

250 J-1=

3-1=2

SS/df

250/2=

125 =MSB

F = MSB/MSW = 125/10

=12.5

Within Groups

120 N-J=

15-3=12

120/12 = 10 =

MSW

Total 370 N-1=

15-1=14

ANOVA Source (Summary) Table

89.3)12,2,05.( F

ANOVA Summary

• Calculate SS (total, between, within)

• Each SS has associated df to calculate MS

• F is ratio of MSb to MSw

• Compare obtained F (12.5) to critical value (3.89). Significant if obtained F is larger than critical.

• One-tailed test makes sense for F.

Review

• Suppose we have 4 groups and 10 people per group. We find that SSB = 60 and SSW = 40. Construct an ANOVA summary table and test for significance of the overall effect.

ANOVA Descriptive Stats

• Because SStot = SSb+SSw we can figure proportion of total variance due to treatment.

• Proportion of total variance due to treatment is:

• R2= SSb/SStot.

• Varies from 0 (no effect) to 1 (no error).• Sample value is biased (too large).

Estimating Power

• Power for what? For one-way ANOVA, power usually means for the overall F, i.e., at least 1 group mean is different from the others.

• Howell uses noncentral F for sample size calculation.

2

2 /)('

e

j k

n'

Where k is the number of treatment goups; n is sample size per group. Variance of error is MSE in the population (variance of DV within cells). Mu(j) are treatment means; mu is grand mean.2'

2

n

SAS Power calculation

• SAS will compute sample size requirements for a given scenario.

• You input the expected means and a common (within cell) standard deviation, (along with alpha and desired power) and it will tell you the sample size you need.

SAS Inputrun;

*********************************************************** Power computation example from Howell, 2010, p. 350.* Note the standard deviation is the square root of the* provided MSE: sqrt(240.35) = ~ 15.5.**********************************************************;proc power ; onewayanova groupmeans = 34 | 50.8 | 60.33 | 48.5 | 38.1 stddev = 15.5 alpha = 0.05 npergroup = . power = .8;

SAS OutputThe POWER Procedure Overall F Test for One-Way ANOVA

Fixed Scenario Elements Method Exact Alpha 0.05 Group Means 34 50.8 60.33 48.5 38.1 Standard Deviation 15.5 Nominal Power 0.8 Computed N Per Group Actual N Per

Power Group 0.831 8

Review

Suppose someone has worked out that a one-way ANOVA with 6 levels has a power of .80 for the overall F test. What does this mean?

Describe (make up) a concrete example of a one-way ANOVA where it makes sense to use an overall F test. Explain why ANOVA (not t, chi-square or something else) is the best method for the analysis.