Download - Two-Way Balanced Independent Samples ANOVA
Two-Way Balanced Independent Samples
ANOVAComputations
ContrastsConfidence Intervals
Partitioning the SStotal
• The total SS is divided into two sources– Cells or Model SS– Error SS– The model is
ijkjkkjijk eY
Partitioning the SScells
• The cells SS is divided into three sources– SSA, representing the main effect of factor A– SSB, representing the main effect of factor B– SSAxB, representing the A x B interaction
• These sources will be orthogonal if the design is balanced (equal sample sizes)– They sum to SScells– Otherwise the analysis gets rather complicated.
Gender x Smoking History
• Cell n = 10, Y2 = 145,140
SMOKING HISTORY GENDER never < 1m 1 m - 2 y 2 y - 7 y 7 y - 12 y marginal Male 300 200 220 250 280 1,250 (25) Female 600 300 350 450 500 2,200 (44) marginal 900 (45) 500 (25) 570 (28.5) 700 (35) 780 (39) 3,450
025,119100
3450)( 22
NYCM
115,26025,119140,1452 CMYSSTotal
Computing Treatment SS
Square and then sum group totals, divide by the number of scores that went into each total, then subtract the CM.
SScells and SSerror
SSerror is then SStotal minus SSCells = 26,115 ‑ 15,405 =
10,710.
SMOKING HISTORY GENDER never < 1m 1 m - 2 y 2 y - 7 y 7 y - 12 y marginal Male 300 200 220 250 280 1,250 (25) Female 600 300 350 450 500 2,200 (44) marginal 900 (45) 500 (25) 570 (28.5) 700 (35) 780 (39) 3,450
cellsSS
405,15
025,11910
500450350300600280250220200300 2222222222
SSgender and SSsmoke
SMOKING HISTORY GENDER never < 1m 1 m - 2 y 2 y - 7 y 7 y - 12 y marginal Male 300 200 220 250 280 1,250 (25) Female 600 300 350 450 500 2,200 (44) marginal 900 (45) 500 (25) 570 (28.5) 700 (35) 780 (39) 3,450
025,9025,11950
200,2250,1 22
GenderSS
140,5
025,11920
780700570500900 22222
SmokeSS
SSSmoke x Gender
SSinteraction = SSCells SSGender SSSmoke =
15,405 ‑ 9,025 ‑ 5,140 = 1,240.
Degrees of Freedom
• dftotal = N - 1• dfA = a - 1• dfB = b - 1• dfAxB = (a - 1)(b -1)• dferror = N - ab
Source TableSource SS df MS F p A-gender 9025 1 9025 75.84 <.001 B-smoking history 5140 4 1285 10.80 <.001 AxB interaction 1240 4 310 2.61 .041 Error 10,710 90 119 Total 26,115 99
Simple Main Effects of Gender
SS Gender, never smoked
SS Gender, stopped < 1m
SS Gender, stopped 1m ‑ 2y
SMOKING HISTORY GENDER never < 1m 1 m - 2 y 2 y - 7 y 7 y - 12 y marginal Male 300 200 220 250 280 1,250 (25) Female 600 300 350 450 500 2,200 (44) marginal 900 (45) 500 (25) 570 (28.5) 700 (35) 780 (39) 3,450
500,420
90010
600300 222
50020
50010
300200 222
84520
57010
350220 222
Simple Main Effects of Gender
SS Gender, stopped 2y - 7y
SS Gender, stopped 7y ‑ 12 y
SMOKING HISTORY GENDER never < 1m 1 m - 2 y 2 y - 7 y 7 y - 12 y marginal Male 300 200 220 250 280 1,250 (25) Female 600 300 350 450 500 2,200 (44) marginal 900 (45) 500 (25) 570 (28.5) 700 (35) 780 (39) 3,450
000,220
70010
450250 222
420,220
78010
500280 222
Simple Main Effects of Gender
• MS = SS / df; F = MSeffect / MSE• MSE from omnibus model = 119 on 90
df Smoking History SS Gender at never < 1m 1 m - 2 y 2 y - 7 y 7 y - 12 y F(1, 90) 37.82 4.20 7.10 16.81 20.34 p <.001 .043 .009 <.001 <.001
Interaction Plot
Simple Main Effects of Smoking
• SS Smoking history for men
• SS Smoking history for women
• Smoking history had a significant simple main effect for women, F(4, 90) = 11.97, p < .001, but not for men, F(4, 90) = 1.43, p =.23.
68050250,1
10280250220200300 222222
700,550200,2
10500450350300600 222222
Multiple Comparisons Involving A Simple Main Effect• Smoking had a significant simple main
effect for women.• There are 5 smoking groups.• We could make 10 pairwise
comparisons.• Instead, we shall make only 4
comparisons.• We compare each group of ex-smokers
with those who never smoked.
Female Ex-Smokersvs. Never Smokers
• There is a special procedure to compare each treatment mean with a control group mean (Dunnett).
• I’ll use a Bonferroni procedure instead.
• The denominator for each t will be:
.0125.405. pc
8785.4)10/110/1(119
• See Obtaining p values with SPSS
Never Smoked vs Quit 8785.4
)90( ji MMt
p Significant?
< 1 m (60-30) / 4.8785=6.149 < .001 yes 1 m - 2 y (60-35) / 4.8785=5.125 < .001 yes 2 y - 7 y (60-45) / 4.8785=3.075 .0028 yes 7 y - 12 y (60-50) / 4.8785=2.050 .0433 no
Multiple Comparisons Involving a Main Effect
• Usually done only if the main effect is significant and not involved in any significant interaction.
• For pedagogical purposes, I shall make pairwise comparisons among the marginal means for smoking.
• Here I use Bonferroni, usually I would use REGWQ.
Bonferroni Tests, Main Effect of Smoking
• c = 10, so adj. criterion = .05 / 10 = .005.• n’s are 20: 20 scores went into each mean.
L e v e l i v s j t = S i g n i f i c a n t ? 1 v s 2
80.54496.320)20/120/1(119/)2545(
y e s
1 v s 3 ( 4 5 - 2 8 . 5 ) / 3 . 4 4 9 6 = 4 . 7 8 y e s 1 v s 4 ( 4 5 - 3 5 ) / 3 . 4 4 9 6 = 2 . 9 0 y e s 1 v s 5 ( 4 5 - 3 9 ) / 3 . 4 4 9 6 = 1 . 7 4 n o 2 v s 3 ( 2 8 . 5 - 2 5 ) / 3 . 4 4 9 6 = 1 . 0 1 n o 2 v s 4 ( 3 5 - 2 5 ) / 3 . 4 4 9 6 = 2 . 9 0 y e s 2 v s 5 ( 3 9 - 2 5 ) / 3 . 4 4 9 6 = 4 . 0 6 y e s 3 v s 4 ( 3 5 - 2 8 . 5 ) / 3 . 4 4 9 6 = 1 . 8 8 n o 3 v s 5 ( 3 9 - 2 8 . 5 ) / 3 . 4 4 9 6 = 3 . 0 4 y e s 4 v s 5 ( 3 9 - 3 5 ) / 3 . 4 4 9 6 = 1 . 1 6 n o
Smoking History Mean
< 1 m 25.0A
1m - 2y 28.5AB
2y - 7y 35.0BC
7y - 12y 39.0CD
Never 45.0D
Means sharing a superscript do not differ from one another at the .05 level.
Results of Bonferroni Test
Interaction Contrasts 2 x 2• Coefficients must be doubly centered• Sum to zero in every row and every column• Consider a 2 x 2, for which there is only one
interaction contrast.
• (A1B1 + A2B2) – (A1B2 + A2B1)– One diagonal versus the other.
Rearranging terms,• (A1B1 - A2B1) – (A1B2 - A2B2)
– Effect of A at B1 versus effect of A at B2
• (A1B1 - A1B2) – (A2B1 - A2B2)– Effect of B at A1 versus effect of B at A2
Coefficients B1 B2
A1 1 -1A2 -1 1
SAS Code
• AB cells (level of A, level of B) are 11, 12, 21, 22
• Proc GLM; Class Cell; Model Y=Cell;• CONTRAST 'A x B' Cell 1 -1 -1 1;• This will produce the interaction SS.
2 x 3 Two Interaction df
• Simple main effect of combined B1B2 versus B3 at A1 contrasted with simple main effect of B1B2 versus B3 at A2,, or (next slide)
A x B12 vs 3
B1 B2 B3
A1 1 1 -2 A2 -1 -1 2
• Simple main effect of A (A1 versus A2) at combined B1B2 contrasted with simple main effect of A at B3
• That is, the A x B interaction with levels 1 and 2 of B combined.
A x B12 vs 3
B1 B2 B3
A1 1 1 -2 A2 -1 -1 2
• Simple main effect of A at B1 contrasted with simple main effect of A at B2
• Simple main effect of B12 (B1 versus B2 ignoring B3) at A1 contrasted with same effect at A2
• That is, the A x B interaction ignoring level 3 of B.
• Proc GLM; Class Cell; Model Y=Cell;• CONTRAST 'B12vs3' Cell 1 1 -2 -1 -1 2;
CONTRAST 'B1vs2' Cell 1 -1 0 -1 1 0;Contrast DF Contrast
SSMean
SquareF Value Pr > F
A x B12vs3 1 644.03 644.033 91.83 <.0001
A x B1vs2 1 152.10 152.10 21.69 <.0001
SSAxB = 644.03 + 152.10 = 769.13
Standardized Contrasts• Give the contrast in standard deviation units• Should the standardizer (the denominator of
estimated d) include or exclude variance due to other factors in the design?
• Kline argues that the standardizer should include all variance in the outcome variable.
• I generally agree.
Therapy x Sex of Patient, 2 x 2• You want to estimate d for the main effect of
therapy. • Should the denominator of estimated d
include variance due to sex?• The MSE excludes variance due to sex, but• In the population, sex may account for some
of the variance in the outcome variable.• The SQRT(MSE) would under-estimate the
population standard deviation.
• Kline says one should include in the standardizer variance due to any factor that is naturally variable in the population (like sex or type of therapy).
• But is the distribution of the factor in the research the same as it is in the population?
• If not, then the factor may account for more or less variance in the research than in the population.
Obtaining a Pooled Standardizer
• You decide to include in the standardizer, for the effect of therapy, variance due to all other effects.
• Could pool the SSwithin-cells, SSsex, and SSinteraction to form an appropriate standardizer.
• Or just drop Sex and (Therapy x Sex) from the model, run the ANOVA, and use the SQRT of the resulting MSE as the standardizer.
Simple Main Effects• Effect of therapy in men vs. in women.• Should the standardizer be computed
within-sex? If so, that for men might differ from that for women.
• Do you want to estimate d in a single-sex population or in a way that the estimate for men can be compared to that for women without considering differences in the denominators?
(Semipartial) Eta-Squared• 2 = SSEffect SSTotal
• Using our smoking history data,• For the interaction,
• For gender,
• For smoking history, 197.115,26
140,52
346.115,26025,92
047.115,26
12402
CI.90 Eta-Squared• Compute the F that would be obtained
were all other effects added to the error term.
• For gender,
dfF 98 1, on 752.5139.174
025.9
39.174199
025,9115,26
GenderTotal
GenderTotal
GxSHSHE
GxSHSHE
dfdfSSSS
dfdfdfSSSSSSMSE
CI.90 Eta-Squared• Use that F with my Conf-Interval-R2-
Regr.sas• 90% CI [.22, .45] for gender• [.005, .15] for smoking• [.000, .17] for the interaction• Yikes, 0 in the CI for a significant effect!• The MSE in the ANOVA excluded
variance due to other effects, that for the CI did not.
Partial Eta-Squared• The value of η2 can be affected by the
number and magnitude of other effects contributing to variance in the outcome variable.
• For example, if our data were only from women, SSTotal would not include SSGender and SSInteraction.
• This would increase η2.• Partial eta-squared estimates what the
effect would be if the other effects were all zero.
Partial Eta-Squared
• For the interaction,
• For gender,
• For smoking history, 324.710,10140,5
140,52
p
457.710,10025,9
025,92
p
104.710,10240,1
240,12
p
ErrorEffect
Effectp SSSS
SS
2
CI.90 on Partial Eta-Squared
• If you use the source table F-ratios and df with my Conf-Interval-R2-Regr.sas, it will return confidence intervals on partial eta-squared.
• Gender: [.33, .55]• Smoking: [.17, .41]• Interaction [.002, .18] note that it
excludes 0
Partial 2 and F For Interaction
104.
710,10240,1240,12
EGxSH
GxSHp SSSS
SS
61.2
90710,104240,1
EE
GxSHGxSH
dfSSdfSSF
Notice that the denominator of both includes error but excludes the effects of Gender and Smoking History.
Semi-Partial 2
047.710,10140,5025,9240,1
240,1
2
ESHGGxSH
GxSH
SSSSSSSSSS
Notice that, unlike partial 2 and F, the denominator of semi-partial 2 includes the effects of Gender and Smoking History.
Omega-Squared• For the interaction,
• For gender,
• For smoking history, 18.234,26
)119(4140,52
34.234,26
)119(1025,92
03.234,26
764119115,26
)119(4240,12
SAS EFFECTSIZE• PROC GLM; CLASS Age Condition;• MODEL Items=Age|Condition /
EFFECTSIZE alpha=0.1;• This will give you eta-squared, partial
eta-squared, omega-squared, and confidence intervals for each.
2 or Partial 2 ?• I generally prefer 2
• Kline says you should exclude an effect from standardizer only if it does not exist in the natural population.
• Values of partial 2 can sum to greater than 100%. Can one really account for more than all of the variance in the outcome variable?
25.10025
*
2
ErrorBABA
Effect
SSSSSSSSSS
For every effect,
For every effect,
50.50252
ErrorEffect
Effectp SSSS
SSThese sum to 150%
2 for Simple Main Effects
• For the women, SStotal = 11,055• and SSsmoking = 5,700• 2 = 5,700/11,055 = .52• To construct confidence interval, need
compute an F using data from women only.
• The SSE is 11,055 (total) – 5,700 (smoking) = 5,355.
• 90% CI [.29, .60]• For the men, 2 = .11, 90% CI [0, .20]
97.1145/53554/700,5 45) (4, F
Assumptions• Normality within each cell• Homogeneity of variance across cells
Advantages of Factorial ANOVA
• Economy -- study the effects of two factors for (almost) the price of one.
• Power -- removing from the error term the effects of Factor B and the interaction gives a more powerful test of Factor A.
• Interaction -- see if effect of A varies across levels of B.
One-Way ANOVAConsider the partitioning of the sums of squares illustrated to the right.SSB = 15 and SSE = 85. Suppose there are two levels of B (an experimental manipulation) and a total of 20 cases.
Treatment Not Significant
MSB = 15, MSE = 85/18 = 4.722. The F(1, 18) = 15/4.72 = 3.176, p = .092. Woe to us, the effect of our experimental treatment has fallen short of statistical significance.
Sex Not Included in the Model
• Now suppose that the subjects here consist of both men and women and that the sexes differ on the dependent variable.
• Since sex is not included in the model, variance due to sex is error variance, as is variance due to any interaction between sex and the experimental treatment.
Add Sex to the Model
Let us see what happens if we include sex and the interaction in the model. SSSex = 25, SSB = 15, SSSex*B = 10, and SSE = 50. Notice that the SSE has been reduced by removing from it the effects of sex and the interaction.
Enhancement of PowerThe MSB is still 15, but the MSE is now 50/16 = 3.125 and the F(1, 16) = 15/3.125 = 4.80, p = .044. Notice that excluding the variance due to sex and the interaction has reduced the error variance enough that now the main effect of the experimental treatment is significant.
Presenting the ResultsParticipants were given a test of their ability to detect the
scent of a chemical thought to have pheromonal properties in humans. Each participant had been classified into one of five groups based on his or her smoking history. A 2 x 5, Gender x Smoking History, ANOVA was employed, using a .05 criterion of statistical significance and a MSE of 119 for all effects tested. There were significant main effects of gender, F(1, 90) = 75.84, p < .001, 2 = .346, 90% CI [.22, .45], and smoking history, F(4, 90) = 10.80, p < .001, 2 = .197, 90% CI [.005, .15], as well as a significant interaction between gender and smoking history, F(4, 90) = 2.61, p = .041, 2 = .047, 90% CI [.00, .17],. As shown in Table 1, women were better able to detect this scent than were men, and smoking reduced ability to detect the scent, with recovery of function being greater the longer the period since the participant had last smoked.
Table 1. Mean ability to detect the scent. Smoking History Gender < 1 m 1 m -2 y 2 y - 7 y 7 y - 12 y never marginal Male 20 22 25 28 30 25 Female 30 35 45 50 60 44 Marginal 25 28 35 39 45
The significant interaction was further investigated with tests of the simple main effect of smoking history. For the men, the effect of smoking history fell short of statistical significance, F(4, 90) = 1.43, p = .23, 2 = .113, 90% CI [.00, .20]. For the women, smoking history had a significant effect on ability to detect the scent, F(4, 90) = 11.97, p < .001, 2 = .516, 90% CI [.29, .60]. This significant simple main effect was followed by a set of four contrasts. Each group of female ex-smokers was compared with the group of women who had never smoked. The Bonferroni inequality was employed to cap the familywise error rate at .05 for this family of four comparisons. It was found that the women who had never smoked had a significantly better ability to detect the scent than did women who had quit smoking one month to seven years earlier, but the difference between those who never smoked and those who had stopped smoking more than seven years ago was too small to be statistically significant.
Interaction Plot