Introduction to the Analysis of Variance
Basic Concepts, Section 12.1 - 12.2
One-Way ANOVA, Section 12.3
PP 7 2
ANOVA Overview
Test for a difference among several means from independently drawn samples The extension of the two sample t-test for means to three or more samples
requires the analysis of variance Consider the negative income tax experiment in New Jersey
Tested whether was a difference in hours of work between the control and the treatment group
In this experiment income was supplemented by different amounts The benefit guarantee level ranged from 50 to 125% of the poverty level
Consider then three groups of income The control group, the first treatment group that received 50% of the poverty level and a
second treatment group that received 75% of the poverty level The null hypothesis is that the mean annual hours over three years is
the same for each group H0: 1 = 2 = 3 H1: at least one of the population means differs from the others
PP 7 3
ANOVA Overview
Could compare the three population means by evaluating all possible pairs of sample means using the two sample t-test Compare
Group 1 to group 2 Group 1 to group 3 Group 2 to group 3
For a total of three groups the number of tests required is (3 pick 2)
Evaluated as 3!/(2!1!) If number of groups = 10 there would be 45 different pair-wise
t-tests (10 pick 2)
PP 7 4
ANOVA Overview
Pair-wise t-tests are likely to lead to an incorrect conclusion Suppose that the three population means are in fact equal and
that we conduct all three pair-wise tests Assume that the tests are independent and set the significance
level at 0.05 for each one By the multiplication rule, the probability of failing to reject a null
hypothesis of no difference in all three instances would be P(fail to reject in all three tests) = (1 - 0.05)3 = (0.95)3 = 0.857
(probability of “accepting” all three) Consequently, the probability of rejecting the null hypothesis in at
least one of the tests is P(reject in at least one test) = 1 - 0.857 = 0.143
Since we know that the null hypothesis is true in each case, 0.143 is the probability of committing a type I error
PP 7 5
ANOVA Overview
Need a testing procedure in which the overall probability of committing a Type I error is equal to some predetermined level of alpha One-way analysis of variance is such a technique
An experiment is a study designed for the purpose of examining the effect that one variable (the independent variable) has on the value of another variable (the dependent variable)
PP 7 6
ANOVA Overview
Negative income tax experiment was a designed experiment Families were assigned to different treatment groups and
given money (or not given money) by the Labor Dept Intervention by researcher
Hours of work were observed for the next three years Economists often work with observational studies
rather than actual experiments For example, we might study families from the Current
Population Survey or the Census Observe level of income and hours of work for each family
and try to relate the two variables
PP 7 7
ANOVA Overview
In NIT example, hours worked is the dependent variable
What influences the hours of work? There are three groups of families, distinguished by the
amount of income they received from the government Think of the income received as the independent variable
Income received will influence hours of work The independent variable is also called the factor or
treatment effect Here we have an experiment in which we try to determine if
various levels of a given factor (income) might have different effects on hours of work
PP 7 8
Variation between and within Groups Looking at the data
There are three different levels of the factor income The values of the hours worked for the different families are grouped by
the factor level We observe the group means
1X 2X 3XGroup Mean
Factor: Income Supplement
Level j groups, j=1, 2, …t
i rows, i=1, 2, …n 1 2 3Measurements: x11 x12 x13
Hours Worked x21 x22 x23
for different families .. .. .. xn1 xn2 xnt
PP 7 9
Two Sources of Variation
Variation between groups reflects the effect of the factor levels, of the treatment Variation between groups is seen by looking at the three
group means If there are large differences in the group means
Suggest that the differences in income supplements has an effect on average hours worked
Variation within groups represents random error from sampling Values within a sample will vary chance
ANOVA uses these two kinds of variation to test for whether the factor has an effect on the dependent variable
PP 7 10
The Model and Assumptions
One-way analysis of variance Examines populations that are classified by one
characteristic In our example, the characteristic is the amount of income
supplement the family receives There are three levels of that factor, or three groups
If we had only two samples instead of t samples, one-way ANOVA is equivalent to the two sample equal variance t-test for independent samples
PP 7 11
Assumptions
The samples have been independently selected
The population variances are equal Not usually tested
The dependent variable follows a normal distribution in the populations
PP 7 12
Online Homework - Chapter 12 Intro to ANOVA CengageNOW ninth assignment: Chapter 12
Intro to ANOVA
PP 7 13
Procedure
Remember each population represents a level of a factor
The hypotheses are H0: 1 = 2 = …. = t
H1: Not all the means are equal The null hypothesis would be
Supported if we observed small differences from one sample mean to the next
Rejected if at least some of the differences in sample means were large
PP 7 14
Procedure
We need a precise measure of the discrepancies among the sample means
A possible choice is the variance of the sample means The basic idea of ANOVA is to express a measure
of the total variation in a data set as a sum of two components
Variation within groups and variation between groups If the variation within groups is small relative to the
variation between the group means Suggests that the population means are in fact different
PP 7 15
Problem – Are There Any Differences in Detergents? Consumer Report is testing the cleansing
action of three leading detergents Cleansing action is the dependent variable The different detergents represent the treatment There are three levels of the factor because there
are three detergents
PP 7 16
Problem – Are There Any Differences in Detergents? There are 15 swatches of dirty cloth We select at random 5 swatches to be
washed by each of the detergents After the swatches are cleaned, rate each on
the basis of 0 to 100 Let the level of significance be 0.01
PP 7 17
Problem – Are There Any Differences in Detergents?
Detergent (factor)
A (1) B (2) C (3)
77 72 76
81 58 85
71 74 82
76 66 80
80 70 77
771 X 682 X 803 X
What is the value of x23?
= X23What is x51?
= x51
PP 7 18
Problem – Are There Any Differences in Detergents? Consider all 15 observations as one data set
for the moment Calculate the total variation in the pooled
data set Then break the total variation into two
component Variation within groups Variation between groups
PP 7 19
Total Variation
Total variation
Grand mean
2)( t
jij
n
i
xxSSTj
Where xij is the ith observation in the jth sample
j = 1, 2,….t samples or groups or levels of the factor
i = 1, 2, … nj observations in a group
N
x
grandmeanx
t
jij
n
i
j
PP 7 20
Total Variation
Grand mean is the mean of all the pooled observations
Capital N represents the total number of observations when the data are pooled
Not necessary for each sample (group) to have the same number of observations
321 nnnnNt
jj
PP 7 21
Summation Notation - Grand Mean When we work with
double summation signs, evaluate the inner summation sign first
535251
232221
131211
321
......
)(
xxx
xxx
xxx
xxxx ii
n
ii
t
jij
n
i
jj
77+ 72+ 76+
81+ 68+ 85+
71+ 74+ 82+
76+ 66+ 80+
80+ 70+ 77 = 1135
75151135 x
PP 7 22
Total Sum of Squares
The total sum of squares can be found next, SST
666)7577()7570()7580(
)7585()7558()7581(
)7576()7572()7577(
222
222
222
2)( t
jij
n
i
xxSSTj
PP 7 23
SST = SSTR + SSE
SST is divided into the variation between groups and the variation within groups (not variance)
SST = SSTR + SSE SSTR = Variation between groups (Treatment) SSE = Variation within groups (Error)
SST = SSB + SSW
PP 7 24
SSTR – Treatment Sum of Squares -Variation between Groups SSTR
2. )( xxnSSTR j
t
jj
j
n
iij
jn
xx
j
.
The dot means that the average is carried out across the index i. We select a particular group, j, and then find the average of all the observations within that group.
390)7580(5)7568(5)7577(5 222
PP 7 25
SSE - Error Sum of Squares - Variation within Groups SSE
Note that SST = SSTR + SSE 666 = 390 + 276 Can solve for two of the
three and find the remaining Sum of Squares (SS) by subtraction
jn
i
j
t
jij xxSSE 2
. )(
276)8077()6870()7780(
)8085()6858()7781(
)8076()6872()7777(
222
222
222
PP 7 26
SST = SSTR + SSE
Examine two different variances One based on the SSTR The other based on the SSE
Remember that a variance is computed by dividing the sum of squared deviations by the appropriate degrees of freedom
Do the same here Create Variances
Also called Mean Squared Deviations
PP 7 27
Mean Square Deviations
Mean Square Deviation for Treatment
1t
SSTrMSTR
where t = the number of groups (We use up one degree of freedom in estimating the grand mean.)
tN
SSEMSE
where N = the total number of observations across all groups (Each group mean is estimated by the sample observations anduses up one degree of freedom.)
PP 7 28
Rationale of the Test
The variance within groups, MSE, measures Variability of the values around the mean of each group Random variation of values within groups
The variance between groups, MSTR, measures Random variation of values within groups Also measures differences from one group to another
If there is no real difference from group to group, the variance between groups should be close to the variance within groups MSTR MSE
Ratio is close to 1 However, if there is a difference between groups, then
MSTR > MSE
PP 7 29
ANOVA - Test Statistic
Test Statistic
If the null hypothesis is true and we draw a large number of samples from the populations and calculate the test statistic repeatedly The sampling distribution of the test statistic follows the F
distribution with t - 1 and N - t degrees of freedom “Most” of the F values will be close to 1
MSE
MSTR= Ft-1,N-t
PP 7 30
ANOVA - Test Statistic
Even when the null hypothesis is true, arithmetically, the SSTR > SSE So the test takes place in
the upper tail of the distribution
Place all of the level of significance in the upper tail
Sampling Distribution of MSE
MSTR
= F t – 1, N - tMSE
MSTR
⍺ reject
PP 7 31
ANOVA – Test Statistic
Find critical value F⍺
The decision rule is If test statistic
),1(,),1( tNttNt FcvF
reject the H0
= F t – 1, N - tMSE
MSTR
Sampling Distribution of MSE
MSTR
reject
PP 7 32
Problem - ANOVA
Calculate the MSTR
Calculate the MSE
Calculate the F test
1952
390
1
t
SSTRMSTR
2312
276
tN
SSEMSE
48.823
19512,2 F
F 2,12
Sampling Distribution of MSE
MSTR
0.01reject
Do not reject
df2/df1 1 2 3 4 51 4052.181 4999.5 5403.352 5624.583 5763.652 98.503 99 99.166 99.249 99.2993 34.116 30.817 29.457 28.71 28.2374 21.198 18 16.694 15.977 15.5225 16.258 13.274 12.06 11.392 10.967
6 13.745 10.925 9.78 9.148 8.7467 12.246 9.547 8.451 7.847 7.468 11.259 8.649 7.591 7.006 6.6329 10.561 8.022 6.992 6.422 6.05710 10.044 7.559 6.552 5.994 5.636
11 9.646 7.206 6.217 5.668 5.31612 9.33 6.927 5.953 5.412 5.06413 9.074 6.701 5.739 5.205 4.86214 8.862 6.515 5.564 5.035 4.69515 8.683 6.359 5.417 4.893 4.556
F Table ⍺ = 0.01
PP 7 34
Problem - ANOVA
Find critical value at = 0.01
Reject H0, some of the means differ significantly
Some of the detergents clean better than others
93.601,.12,2 F
0.01
6.93 8.48 F 2,12
reject
PP 7 35
ANOVA Table
Source of Variation
Sum of Squares
Degrees of Freedom
Mean Square
F
Between Groups=Treatment
SSTR t-1 SSTR/(t-1) MSTR/MSE
Within Groups=Error
SSE N-t SSE/(N-t)
Total SST N-1
PP 7 36
Completed ANOVA Table
Source of Variation
Sum of Squares
Degrees of Freedom
Mean Square
F p - value
Between Groups
390 2 195 8.48 0.0051
Within Groups
276 12 23
Total 666 14
PP 7 37
ANOVA p - value
Computer output provides the probability of observing an F test statistic as large as 8.48 if the H0 is true This p-value is 0.0051
To find the p-value, in a cell within a Microsoft Excel spreadsheet, type =FDIST(Test value, t-1, N-t) =FDIST(8.48,2,12) = .0051
Setting our level of significance at 0.01, .0051 < 0.01 Reject the null hypothesis
PP 7 38
Multiple Comparison Procedures What happens if we reject the null hypothesis?
Conclude that the population means are not all equal Do not know whether all of the means are different from one
another or if only some of them are different Want to conduct additional tests to find out where the differences
lie Number of multiple comparison tests available, each with
advantages and disadvantages Simple approach is to perform a series of two sample t-tests
This increases the probability of committing a Type I error Avoid this problem by reducing the individual levels to ensure
that the overall level of significance is kept at a predetermined level
PP 7 39
ANOVA Assumption - Homogeneity of Variances Bartlett’s Test for Homogeneity of Variances
Most common method used to test whether the population variances are equal Test is powerful
Can discern that the null hypothesis is false Badly affected by non-normal populations
ANOVA is robust Robust means that the validity of a test is not seriously affected by moderate
deviations from the underlying assumptions Anova operates well even with considerable heterogeneity of
variances, as long as nj are equal or nearly equal ANOVA is also robust with respect to the assumption of the
underlying populations’ normality, especially as n increases
PP 7 40
Online Homework - Chapter 12 ANOVA CengageNOW tenth assignment: Chapter 12
ANOVA CengageNOW eleventh assignment: Chapter
12: Overview of ANOVA
PP 7 41
Multiple Comparison Technique: Bonferroni Correction The significance level for each of the individual
comparisons depends on the number of pair-wise tests being conducted
In our problem, we set = 0.01 and we have (3 pick 2) = 3 pair-wise comparisons
To set the overall probability of committing a Type I error at 0.01 we should use
for the significance level for an individual comparison
003.3
01.0*
PP 7 42
Bonferroni Correction
Instead of pooling the data from only two samples to estimate the common variance, pool all t samples
Degrees of freedom are N – t The test statistic is
)]/1()/1[( 212
21
nnS
xxt
p
df
PP 7 43
Bonferroni Correction
The sample variances are
The pooled variance is
S1 = 3.937S2 = 6.325S3 = 3.674
00.2312
)50.13(4)00.40(4)50.15(42
pS
PP 7 44
Bonferroni Correction: Group 1&2, Group 1&3, Group 2&3 Perform three t –tests
967.2033.3
9
)5/15/1(23
687712
t p-value = .0118, do not reject at = .003
989.0033.3
807713
t p-value = .171, do not reject at
= .003
956.3033.3
806823
t
p-value = .0019, reject at = 0.003. There is a significant difference between detergent 2 and 3.
PP 7 45
P-values from Excel
Using Excel’s statistical function =TDIST(x,df,tails)
=TDIST(2.967,12,2) =TDIST(-.989,12,2) =TDIST(-3.956,12,2)