1

Model Adequacy Checking in the ANOVAText reference, Section 3-4, pg. 75

• Checking assumptions is important

• Have we fit the right model?

• Normality

• Independence

• Constant variance

Later we will talk about what to do if some of these assumptions are violated

2

Model Adequacy Checking in the ANOVA

• Residuals (see text, Sec. 3-4, pg. 75)

• Statistical software usually generates the residuals

• Residual plots are very useful

• Normal probability plot of residuals

.

ˆij ij ij

ij i

e y y

y y

Violations of the basic assumptions and model adequacy can be investigated by examination of residuals

3

Other Important Residual Plots

4

Other Important Residual Plots

• A tendency of positive/negative residuals indicates correlation – violation of independence assumption

• Special attention is needed on uneven spread on the two ends in residuals versus time plot

• Peak discharge vs. fitted value (Ex. 3-5, page 81) – a violation of independence or constant variance assumptions

Figure 3-7 Plot of residuals versus fitted value (Ex. 3-5)

5

Outliers• When the residual is much larger/smaller than any of the others

• First check if it is a human mistake in recording /calculation

• If it cannot be rejected based on reasonable non-statistical grounds, it must be specially studied

• At least two analyses (with and without the outlier) can be made

• Procedure to detect outliers

• Visual examination

• Using standardized residuals

• If eij: N(0,2), then dij: N(0,). Therefore,

68% of dij should fall within the limits ±1;

95% of dij should fall within the limits ±2;

all dij should fall within the limits ±3;

A residual bigger than 3 or 4 standard deviations from zero is a potential outlier.

E

ijij

MS

ed

6

Post-ANOVA Comparison of Means

• The analysis of variance tests the hypothesis of equal treatment means

• Assume that residual analysis is satisfactory• If that hypothesis is rejected, we don’t know which specific

means are different • Determining which specific means differ following an ANOVA

is called the multiple comparisons problem• There are lots of ways to do this…see text, Section 3-5, pg. 87• We will use pairwise t-tests on means…sometimes called

Fisher’s Least Significant Difference (or Fisher’s LSD) Method

7

Contrasts

Used for multiple comparisons

A contrast is a linear combination of parameters

with

The hypotheses are

Hypothesis testing can be done using a t-test:

The null hypothesis would be rejected if |t > t/2,N-a

a

iic1

0:

0:

11

10

a

ii

a

ii

cH

cH

01

a

ic

a

iE

a

ii

cn

MS

yct

1

2

1.

0

8

Contrasts (cont.)

Hypothesis testing can be done using an F-test:

The null hypothesis would be rejected if F > F,1,N-a

The 100(1-) percent confidence interval on the contrast is

a

iE

a

ii

o

cn

MS

yc

tF

1

2

2

1.

20

01

a

iic

a

iE

aN

a

ii

a

ii

a

iE

aN

a

ii cn

MStyccc

n

MStyc

1

2,2/

1.

11

2,2/

1.

9

Comparing Pairs of Treatment Means

Comparing all pairs of a treatment means <=> testing Ho: i = j for all i j.

Tukey’s testTwo means are significantly different if the absolute value of their

sample differences exceeds

for equal sample sizes

Overall significance level is

for unequal sample sizes

Overall significance level is at most

n

MSfaqT E),(

)11

(2

),(

jiE nn

MSfaq

T

10

100(1-) percent confidence intervals (Tukey’s test)

for equal sample sizes

for unequal sample sizes

n

MSfaqyy

n

MSfaqyy E

jijiE

ji ),(),( ....

)11

(2

),(

)11

(2

),(

..

..

jiEji

jiji

Eji

nnMS

faqyy

nnMS

faqyy

11

Example: etch rate experiment

= 0.05 f = 16 (degrees of freedom for error)

q0.05(4,16) = 4.05 (Table VII),

09.335

70.33305.4)16,4(05.005.0

n

MSqT E

2.551.1 y 4.587.2 y 4.625.3 y 0.707.4 y

*20.36.2.1 yy

*20.74.3.1 yy

*8.155.4.1 yy

*0.38.3.2 yy

*6.119.4.2 yy

*6.81.4.3 yy

12

The Fisher Least Significant Difference (LSD) MethodTwo means are significantly different if

where for equal sample sizes

for unequal sample sizes

LSDyy ji

)11

(,2/ji

EaN nnMStLSD

n

MStLSD E

aN

2,2/

For Example 3-1 49.24

5

)70.333(2120.2

216,025.0

n

MStLSD E

*20.36.2.1 yy

*20.74.3.1 yy

*8.155.4.1 yy

*0.38.3.2 yy

*6.119.4.2 yy

*6.81.4.3 yy

13

Duncan’s Multiple Range Test (page 100)The Newman-Keuls Test (page 102)

Are they the same?Which one to use?

• Opinions differ• LSD method and Duncan’s multiple range test are the most powerful ones• Tukey method controls the overall error rate

14

Design-Expert Output

Treatment Means (Adjusted, If Necessary)Estimated StandardMean Error

1-160 551.20 8.17 2-180 587.40 8.17 3-200 625.40 8.17 4-220 707.00 8.17

Mean Standard t for H0Treatment Difference DF Error Coeff=0 Prob > |t| 1 vs 2 -36.20 1 11.55 -3.13 0.0064 1 vs 3 -74.20 1 11.55 -6.42 <0.0001 1 vs 4 -155.80 1 11.55 -13.49 < 0.0001 2 vs 3 -38.00 1 11.55 -3.29 0.0046 2 vs 4 -119.60 1 11.55 -10.35 < 0.0001 3 vs 4 -81.60 1 11.55 -7.06 < 0.0001

15

For the Case of Quantitative Factors, a Regression Model is often Useful

Quantitative factors: ones whose levels can be associated with points on a numerical scale

Qualitative factors: Whose levels cannot be arranged in order of magnitude

The experimenter is often interested in developing an interpolation equation for the response variable in the experiment – empirical model

Approximations:

y = 0 + 1 x + 2 x2 + (a quadratic model)

y = 0 + 1 x + 2 x2 + 3 x3 + (a cubic model)

The method of least squares can be used to estimate the parameters.

Balance between “goodness-of-fit” and “generality”