7/29/2019 ANOVA Assumptions

1/25

The Assumptions ofANOVA

Dennis Monday

Gary Klein

Sunmi Lee

May 10, 2005

7/29/2019 ANOVA Assumptions

2/25

Major Assumptions of Analysis of

Variance The Assumptions

Independence

Normally distributed

Homogeneity of variances

Our Purpose Examine these assumptions

Provide various tests for these assumptions Theory

Sample SAS code (SAS, Version 8.2)

Consequences when these assumptions are not met

Remedial measures

7/29/2019 ANOVA Assumptions

3/25

Normality

Why normal? ANOVA is anAnalysis of Variance

Analysis of two variances, more specifically, the ratio of

two variances Statistical inference is based on the F distribution

which is given by the ratio of two chi-squareddistributions

No surprise that each variance in the ANOVA ratio comefrom a parent normal distribution

Calculations can always be derived no matterwhat the distribution is. Calculations arealgebraic properties separating sums of squares.Normality is only needed for statistical inference.

7/29/2019 ANOVA Assumptions

4/25

Normality

Tests Wide variety of tests we can perform to test if the

data follows a normal distribution.

Mardia (1980) provides an extensive list for both

the univariate and multivariate cases,categorizing them into two types Properties of normal distribution, more specifically, the

first four moments of the normal distribution

Shapiro-Wilks W (compares the ratio of the standard

deviation to the variance multiplied by a constant to one) Goodness-of-fit tests,

Kolmogorov-Smirnov D

Cramer-von Mises W2

Anderson-Darling A2

7/29/2019 ANOVA Assumptions

5/25

Normality

Tests

procunivariate data=temp normal plot;

var expvar;

run;

procunivariate data=temp normal plot;

var normvar;

run;

Tests for Normality

Test --Statistic--- -----p Value------

Shapiro-Wilk W 0.731203 Pr < W D W-Sq A-Sq D >0.1500

Cramer-von Mises W-Sq 0.03225 Pr > W-Sq >0.2500

Anderson-Darling A-Sq 0.224264 Pr > A-Sq >0.2500

Normal Probability Plot2.3+ ++ *

| ++*| +**| +**| ****| ***| **+| **| ***| **+| ***

0.1+ ***| **| ***| ***| **| +***| +**| +**| ****| ++| +*

-2.1+*+++----+----+----+----+----+----+----+----+----+----+

-2 -1 0 +1 +2

Stem Leaf # Boxplot22 1 1 |20 7 1 |18 90 2 |16 047 3 |14 6779 4 |12 469002 6 |10 2368 4 |8 005546 6 +-----+6 228880077 9 | |4 5233446 7 | |2 3458447 7 *-----*0 366904459 9 | + |-0 52871 5 | |-2 884318651 9 | |-4 98619 5 +-----+-6 60 2 |-8 98557220 8 |-10 963 3 |-12 584 3 |-14 853 3 |-16 0 1 |-18 4 1 |-20 8 1 |

----+----+----+----+Multiply Stem.Leaf by 10**-1

Normal Probability Plot

8.25+

| *

|

|

| *

|

| *

| +

4.25+ ** ++++

| ** +++

| *+++

| +++*

| ++****

| ++++ **

| ++++*****

| ++******

0.25+* * ******************

+----+----+----+----+----+----+----+----+----+----+

Stem Leaf # Boxplot8 0 1 *7766 1 1 *55 2 1 *4 5 1 04 4 1 03 588 3 03 3 1 02 59 2 |2 00112234 8 |1 56688 5 |1 00011122223444 14 +--+--+0 55555566667777778999999 23 *-----*0 000011111111111112222222233333334444444 39 +-----+----+----+----+----+----+----+----+----

7/29/2019 ANOVA Assumptions

6/25

Consequences of Non-Normality

F-test is very robust against non-normal data,especially in a fixed-effects model

Large sample size will approximate normality by

Central Limit Theorem (recommended samplesize > 50)

Simulations have shown unequal sample sizesbetween treatment groups magnify any departurefrom normality

A large deviation from normality leads tohypothesis test conclusions that are too liberaland a decrease in power and efficiency

7/29/2019 ANOVA Assumptions

7/25

Remedial Measures for Non-

Normality Data transformation

Be aware - transformations may lead to afundamental change in the relationship between

the dependent and the independent variable andis not always recommended.

Dont use the standard F-test. Modified F-tests

Adjust the degrees of freedom

Rank F-test (capitalizes the F-tests robustness)

Randomization test on the F-ratio

Other non-parametric test if distribution is unknown

Make up our own test using a likelihood ratio ifdistribution is known

7/29/2019 ANOVA Assumptions

8/25

Independence

Independent observations No correlation between error terms

No correlation between independent variables and error

Positively correlated data inflates

standard error The estimation of the treatment means are more

accurate than the standard error shows.

7/29/2019 ANOVA Assumptions

9/25

Independence Tests

If we have some notion of how the data wascollected, we can check if there exists any

autocorrelation.

The Durbin-Watson statistic looks at the

correlation of each value and the value before it

Data must be sorted in correct order for meaningful

results

For example, samples collected at the same time wouldbe ordered by time if we suspect results could depend

on time

7/29/2019 ANOVA Assumptions

10/25

Independence Tests

procglmdata=temp;class trt;

model y = trt / p;

output out=out_ds r=resid_var;

run;

quit;

data out_ds;

set out_ds;

time = _n_;run;

procgplot data=out_ds;

plot resid_var * time;

run;

quit;

procglmdata=temp;class trt;

model y = trt / p;

output out=out_ds r=resid_var;

run;

quit;

data out_ds;

set out_ds;

time = _n_;run;

procgplot data=out_ds;

plot resid_var * time;

run;

quit;

First Order Autocorrelation

0.00479029

Durbin-Watson D

1.96904290

First Order Autocorrelation

0.90931

Durbin-Watson D

0.12405

7/29/2019 ANOVA Assumptions

11/25

Remedial Measures for Dependent

Data First defense against dependent data is proper

study design and randomization Designs could be implemented that takes correlation

into account, e.g., crossover design Look for environmental factors unaccounted for

Add covariates to the model if they are causingcorrelation, e.g., quantified learning curves

If no underlying factors can be found attributed to

the autocorrelation Use a different model, e.g., random effects model

Transform the independent variables using thecorrelation coefficient

7/29/2019 ANOVA Assumptions

12/25

Homogeneity of Variances

Eisenhart (1947) describes the problem ofunequal variances as follows the ANOVA model is based on the proportion of the

mean squares of the factors and the residual meansquares

The residual mean square is the unbiased estimator of2, the variance of a single observation

The between treatment mean squares takes into accountnot only the differences between observations, 2,justlike the residual mean squares, but also the variance

between treatments If there was non-constant variance among treatments,

we can replace the residual mean square with someoverall variance, a2, and a treatment variance, t2,which is some weighted version of a2

The neatness of ANOVA is lost

7/29/2019 ANOVA Assumptions

13/25

Homogeneity of Variances

The omnibus (overall) F-test is very robust

against heterogeneity of variances,

especially with fixed effects and equal

sample sizes.

Tests for treatment differences like t-tests

and contrasts are severely affected,

resulting in inferences that may be tooliberal or conservative.

7/29/2019 ANOVA Assumptions

14/25

Tests for Homogeneity of Variances

Levenes Test

computes a one-way-anova on the absolute value (orsometimes the square) of the residuals, |yiji| with t-1, Nt degrees of freedom

Considered robust to departures of normality, but tooconservative

Brown-Forsythe Test a slight modification of Levenes test, where the median is

substituted for the mean (Kuehl (2000) refers to it as theLevene (med) Test)

The Fmax Test

Proportion of the largest variance of the treatment groups

to the smallest and compares it to a critical value table Tabachnik and Fidell (2001) use the Fmax ratio more as a

rule of thumb rather than using a table of critical values.

Fmax ratio is no greater than 10

Sample sizes of groups are approximately equal (ratio ofsmallest to largest is no greater than 4)

No matter how the Fmax test is used, normality must beassumed.

7/29/2019 ANOVA Assumptions

15/25

Tests for Homogeneity of Variances

procglmdata=temp;

class trt;

model y = trt;

means trt / hovtest=levene hovtest=bf;

run;

quit;

procglmdata=temp;

class trt;

model y = trt;

means trt / hovtest=levene hovtest=bf;

run;

quit;

Homogeneous Variances

The GLM Procedure

Levene's Test for Homogeneity of Y Variance

ANOVA of Squared Deviations from Group Means

Sum of Mean

Source DF Squares Square F Value Pr > F

TRT 1 10.2533 10.2533 0.60 0.4389

Error 98 1663.5 16.9747

Brown and Forsythe's Test for Homogeneity of Y Variance

ANOVA of Absolute Deviations from Group Medians

Sum of Mean

Source DF Squares Square F Value Pr > F

TRT 1 0.7087 0.7087 0.56 0.4570

Error 98 124.6 1.2710

Heterogenous Variances

The GLM Procedure

Levene's Test for Homogeneity of y Variance

ANOVA of Squared Deviations from Group Means

Sum of Mean

Source DF Squares Square F Value Pr > F

trt 1 10459.1 10459.1 36.71

7/29/2019 ANOVA Assumptions

16/25

Tests for Homogeneity of Variances

SAS (as far as I know) does not have a procedure

to obtain Fmax (but easy to calculate)

More importantly:

VARIANCE TESTS ARE ONLY FOR ONE-WAY

ANOVA

WARNING: Homogeneity of variance testing and Welch's

ANOVA are only available for unweighted one-way

models.

7/29/2019 ANOVA Assumptions

17/25

Tests for Homogeneity of Variances

(Randomized Complete Block Design and/or

Factorial Design)

In a CRD, the variance of each treatment

group is checked for homogeneity

In factorial/RCBD, each cells varianceshould be checked

H0: ij

2= ij

2, For all i,j where i i, j j

7/29/2019 ANOVA Assumptions

18/25

Tests for Homogeneity of Variances

(Randomized Complete Block Design and/or

Factorial Design)

Approach 1 Code each row/column to its own

group

Run HOVTESTS as before

Approach 2 Recall Levenes Test and Brown-

Forsythe Test are ANOVAs based on

residuals

Find residual for each observation

Run ANOVA

data newgroup;

set oldgroup;

if block = 1 and treat = 1 then newgroup = 1;

if block = 1 and treat = 2 then newgroup = 2;

if block = 2 and treat = 1 then newgroup = 3;

if block = 2 and treat = 2 then newgroup = 4;

if block = 3 and treat = 1 then newgroup = 5;

if block = 3 and treat = 2 then newgroup = 6;

run;

procglm data=newgroup;

class newgroup;

model y = newgroup;

means newgroup / hovtest=levene hovtest=bf;

run;

quit;

procsort data=oldgroup; by treat block; run;

procmeans data=oldgroup noprint; by treat block;

var y;

output out=stats mean=mean median=median;

run;

data newgroup;

merge oldgroup stats;

by treat block;

resid = abs(mean - y);

if block = 1 and treat = 1 then newgroup = 1;

run;

procglm data=newgroup;

class newgroup;

model resid = newgroup;

run; quit;

7/29/2019 ANOVA Assumptions

19/25

Tests for Homogeneity of Variances

(Repeated-Measures Design) Recall the repeated-measures set-up:

Treatment

a1 a2 a3

s1 s1 s1

s2 s2 s2

s3 s3 s3

s4 s4 s4

7/29/2019 ANOVA Assumptions

20/25

Tests for Homogeneity of Variances

(Repeated-Measures Design) As there is only one score per cell, the variance

of each cell cannot be computed. Instead, fourassumptions need to be tested/satisfied

Compound Symmetry Homogeneity of variance in each column

a12= a2

2 =a32

Homogeneity of covariance between columns

a1a2 =a2a3

= a3a1

No A x S Interaction (Additivity) Sphericity

Variance of difference scores between pairs are equal

Ya1-Ya2 = Ya1-Ya3

= Ya2-Ya3

7/29/2019 ANOVA Assumptions

21/25

Tests for Homogeneity of Variances

(Repeated-Measures Design) Usually, testing sphericity will suffice

Sphericity can be tested using the Mauchly test inSAS

procglm data=temp;

class sub;

model a1 a2 a3 = sub / nouni;

repeated as 3 (123) polynomial / summary printe;

run; quit;

Sphericity Tests

Mauchly's

Variables DF Criterion Chi-Square Pr > ChiSq

Transformed Variates 2 Det = 0 6.01 .056

Orthogonal Components 2 Det = 0 6.03 .062

7/29/2019 ANOVA Assumptions

22/25

Tests for Homogeneity of Variances

(Latin-Squares/Split-Plot Design) If there is only one score per cell, homogeneity of

variances needs to be shown for the marginals of

each column and each row

Each factor for a latin-square

Whole plots and subplots for split-plot

If there are repititions, homogeneity is to be

shown within each cell like RCBD

If there are repeated-measures, follow guidelinesfor sphericity, compound symmetry and additivity

as well

7/29/2019 ANOVA Assumptions

23/25

Remedial Measures for

Heterogeneous Variances Studies that do not involve repeated measures

If normality is not violated, a weighted ANOVA is suggested(e.g., Welchs ANOVA)

If normality is violated, the data transformation necessary to

normalize data will usually stabilize variances as well If variances are still not homogeneous, non-ANOVA tests

might be your option

Studies with repeated measures For violations of sphericity

modify the degrees of freedom have been suggested. Greenhouse-Geisser

Huynh and Feldt

Only do specific comparisons (sphericity does not apply sinceonly two groups sphericity implies more than two)

MANOVA

Use an MLE procedure to specify variance-covariance matrix

7/29/2019 ANOVA Assumptions

24/25

Other Concerns

Outliers and influential points

Data should always be checked for influential

points that might bias statistical inference Use scatterplots of residuals

Statistical tests using regression to detect outliers

DFBETAS

Cooks D

7/29/2019 ANOVA Assumptions

25/25

References Casella, G. and Berger, R. (2002). Statistical Inference. United States: Duxbury.

Cochran, W. G. (1947). Some Consequences When the Assumptions for the Analysis ofVariances are not Satisfied. Biometrics. Vol. 3, 22-38.

Eisenhart, C. (1947). The Assumptions Underlying the Analysis of Variance. Biometrics.Vol. 3, 1-21.

Ito, P. K. (1980). Robustness of ANOVA and MANOVA Test Procedures. Handbook ofStatistics 1: Analysis of Variance (P. R. Krishnaiah, ed.), 199-236. Amsterdam: North-Holland.

Kaskey, G., et al. (1980). Transformations to Normality. Handbook of Statistics 1: Analysisof Variance (P. R. Krishnaiah, ed.), 321-341. Amsterdam: North-Holland.

Kuehl, R. (2000). Design of Experiments: Statistical Principles of Research Design andAnalysis, 2nd edition. United States: Duxbury.

Kutner, M. H., et al. (2005). Applied Linear Statistical Models, 5th edition. New York:McGraw-Hill.

Mardia, K. V. (1980). Tests of Univariate and Multivariate Normality. Handbook of Statistics1: Analysis of Variance (P. R. Krishnaiah, ed.), 279-320. Amsterdam: North-Holland.

Tabachnik, B. and Fidell, L. (2001). Computer-Assisted Research Design and Analysis.Boston: Allyn & Bacon.