multivariate analysis of variance
TRANSCRIPT
Multivariate analysis of varianceMultivariate analysis of variance (MANOVA) is a generalized form of univariate analysis
of variance (ANOVA). It is used when there are two or more dependent variables. It helps to
answer : 1. do changes in the independent variable(s) have significant effects on the
dependent variables; 2. what are the interactions among the dependent variables and 3.
among the independent variables.[1]
Where sums of squares appear in univariate analysis of variance, in multivariate analysis of
variance certain positive-definite matrices appear. The diagonal entries are the same kinds
of sums of squares that appear in univariate ANOVA. The off-diagonal entries are
corresponding sums of products. Under normality assumptions about error distributions, the
counterpart of the sum of squares due to error has a Wishart distribution.
Analogous to ANOVA, MANOVA is based on the product of model variance
matrix, Σmodel and inverse of the error variance matrix, , or . The
hypothesis that Σmodel = Σresidual implies that the product A∼I[2] . Invariance considerations imply
the MANOVA statistic should be a measure of magnitude of the singular value
decomposition of this matrix product, but there is no unique choice owing to the multi-
dimensional nature of the alternative hypothesis.
The most common[3][4] statistics are summaries based on the roots (or eigenvalues) λp of
the A matrix:
Samuel Stanley Wilks '
ΛWilks =∏ (1 / (1 + λp))
1...p
distributed as lambda (Λ)
the Pillai-M. S. Bartlett trace,
ΛPillai = ∑ (1 / (1 + λp))
1...p
the Lawley-Hotelling trace,
ΛLH = ∑ (λp)
1...p
Roy's greatest root (also called Roy's largest root), ΛRoy = maxp(λp)
Discussion continues over the merits of each, though the greatest root leads only to a
bound on significance which is not generally of practical interest. A further complication is
that the distribution of these statistics under the null hypothesis is not straightforward and
can only be approximated except in a few low-dimensional cases. The best-
known approximation for Wilks' lambda was derived by C. R. Rao.
In the case of two groups, all the statistics are equivalent and the test reduces to Hotelling's
T-square.
[edit]References
1. ̂ Stevens, J. P. (2002). Applied multivariate statistics for the social sciences. Mahwah, NJ:
Lawrence Erblaum.
2. ̂ Carey, Gregory. "Multivariate Analysis of Variance (MANOVA): I. Theory". Retrieved 2011-03-22.
3. ̂ Garson, G. David. "Multivariate GLM, MANOVA, and MANCOVA". Retrieved 2011-03-22.
4. ̂ UCLA: Academic Technology Services, Statistical Consulting Group.. "Stata Annotated Output --
MANOVA". Retrieved 2011-03-22.
[edit]
© Gregory Carey, 1998 MANOVA: I - 1
Multivariate Analysis of Variance (MANOVA): I. Theory Introduction
The purpose of a t test is to assess the likelihood that the means for two groups are sampled from the same sampling distribution of means. The purpose of an ANOVA is to test whether the means for two or more groups are taken from the same sampling distribution. The multivariate equivalent of the t test is Hotelling’s T 2
. Hotelling’s T2 tests whether the two vectors of means for the two groups are sampled from the same
sampling distribution. MANOVA is the multivariate analogue to Hotelling's T2
. The purpose of MANOVA is to test whether the vectors of means for the two or more groups are sampled from the same sampling distribution. Just as Hotelling's T2 will provide a measure of the likelihood of picking two random vectors of means out of the same hat, MANOVA gives a measure of the overall likelihood of picking two or more random vectors of means out of the same hat.
There are two major situations in which MANOVA is used. The first is when there are several correlated dependent variables, and the researcher desires a single, overall statistical test on this set of variables instead of performing multiple individual tests. The second, and in some cases, the more important purpose is to explore how independent variables influence some patterning of response on the dependent variables. Here, one literally uses an analogue of contrast codes on the dependent variables to test hypotheses about how the independent variables differentially predict the dependent variables.
MANOVA also has the same problems of multiple post hoc comparisons as ANOVA. An ANOVA gives one overall test of the equality of means for several groups for a single variable. The ANOVA will not tell you which groups differ from which other groups. (Of course, with the judicious use of a priori contrast coding, one can overcome this problem.) The MANOVA gives one overall test of the equality of mean vectors for several groups. But it cannot tell you which groups differ from which other groups on their mean vectors. (As with ANOVA, it is also possible to overcome this problem through the use of a priori contrast coding.) In addition, MANOVA will not tell you which variables are responsible for the differences in mean vectors. Again, it is possible to overcome this with proper contrast coding for the dependent variables.
In this handout, we will first explore the nature of multivariate sampling and then explore the logic behind MANOVA.
1. MANOVA: Multivariate Sampling
To understand MANOVA and multivariate sampling, let us first examine a
MANOVA design. Suppose a researcher in psychotherapy interested in the treatment
efficacy of depression randomly assigned clinic patients into four conditions:
(1) a placebo control group who received typical clinic psychotherapy and a placebo drug;
(2) a placebo cognitive therapy group who received the placebo medication and systematic
cognitive psychotherapy; (3) an active antidepressant medication group who received the© Gregory Carey, 1998 MANOVA: I – 2 typical clinic psychotherapy; and (4) an active medication group who received cognitive therapy. This is a (2 x 2) factorial design with medication (placebo versus drug) as one
factor and type of psychotherapy (clinic versus cognitive) as the second factor.
Studies such as this one typically collect a variety of measures before treatment, during treatment, and after treatment. To keep the example simple, we will focus only on three outcome measures, say, Beck
Depression Index scores (a self-rated depression inventory), Hamilton Rating Scale scores (a clinician rated depression inventory), and Symptom.
Checklist for Relatives (a rating scale that a relative completes on the patient--it was made up for this example). High scores on all these measures indicate more depression; low scores indicate normality. The data matrix would look like this:
Person Drug Psychotherapy BDI HRS SCR Sally placebo cognitive 12 9 6 Mortimer drug clinic 10 13 7
Miranda placebo clinic 16 12 4 . . . . . Waldo drug cognitive 8 3 2
For simplicity, assume the design is balanced with equal numbers of patients in all four conditions. A univariate ANOVA on any single outcome measure would contain three effects, a main effect for psychotherapy, a mean effect for medication, and an interaction between psychotherapy and medication. The MANOVA will also contain the same three effects. The univariate ANOVA main effect for psychotherapy tells whether the clinic versus the cognitive therapy groups have different means, irrespective of their medication.
The MANOVA main effect for psychotherapy tells whether the clinic versus the cognitive therapy group have different mean vectors irrespective of their medication; the vectors in this case are the (3 x 1) column vectors of (BDI, HRS, and SCR) means.
The univariate ANOVA for medication tells whether the placebo group has a different mean from the drug group irrespective of psychotherapy. The MANOVA main effect for medication tells whether the placebo group has a different mean vector from the drug group irrespective of psychotherapy. The univariate ANOVA interaction tells whether the four means for a single variable differ from the value predicted from knowledge of the main effects of psychotherapy and drug. The MANOVA interaction term tells whether the four mean vectors differ from the vector predicted from knowledge of the main effects of psychotherapy and drug. If you are coming to the impression that a MANOVA has all the properties as an ANOVA, you are correct. The only difference is that an ANOVA deals with a (1 x 1) mean vector for any group while a MANOVA deals with a (p x 1) vector for any group, p being the number of dependent variables, 3 in our example. Now let's think for a minute.
What is the variance-covariance matrix for a single variable? It is a (1 x 1) matrix that has only one element, the variance of the variable. What is the variance-covariance matrix for p variables? It is now a (p x p) matrix with the variances on the diagonal and the covariances© Gregory Carey, 1998 MANOVA: I – 3 on the off diagonals. The ANOVA partitions the (1 x 1) covariance matrix into a part due to error and a part due to hypotheses (the two main effects and the interaction term as described above). Or for our example, we can write Vt = Vp + Vm + V(p*m) + Ve (1.1) Equation (1.1) states that the total variability (Vt ) is the sum of the variability due to psychotherapy (Vp), the variability due to medication (Vm), the variability due to the interaction of psychotherapy with medication (V(p*m) ), and error variability (Ve ).
The MANOVA likewise partitions its (p x p) covariance matrix into a part due to error and a part due to hypotheses. Consequently, the MANOVA for our example will have a (3 x 3) covariance matrix for total
variability, a (3 x 3) covariance matrix due to psychotherapy, a (3 x 3) covariance matrix due to edication, a (3 x 3) covariance matrix due to the interaction of psychotherapy with medication, and a (3 x 3) covariance matrix for error. Or we can now write Vt = Vp + Vm + V(p*m) + Ve (1.2) where V now stands for the appropriate (3 x 3) matrix. Note how equation (1.2) equals (1.1) except that (1.2) is in matrix form. Actually, if we considered all the variances in a univariate ANOVA as (1 by 1) matrices and wrote equation (1.1) in matrix form, we would have equation (1.2).
Let's now interpret what these matrices in (1.2) mean. The Ve matrix will look like this BDI HRS CSR BDI Ve1 cov(e1,e2) cov(e1,e3) HRS cov(e2,e1) Ve2 cov (e2,e3) CSR cov(e3,e1) cov(e3,e2) Ve3
The diagonal elements are the error variances. They have the same meaning as the error variances in their univariate ANOVAs. In a univariate ANOVA the error variance is an average variability within the four groups. The error variance in MANOVA has the same meaning. That is, if we did a univariate ANOVA for the Beck Depression Inventory, the error variance would be the mean squares within groups. ve1 would be the same means squares within groups; it would literally be the same number as in the univariate analysis. The same would hold for the mean squares within groups for the HRS and the CSR. The only difference then is in the off diagonals. The off diagonal covariances for the error matrix must all be interpreted as within group covariances. That is, cov(e1,e2) tells us the extent to which individuals within a group who have high BDI scores also tend to have high HRS scores. Because this is a covariance matrix, the matrix can be scaled to correlations to ease inspection. For example, corr(e1,e2) = cov(e1,e2) (ve1ve2) -1/2 .© Gregory Carey, 1998 MANOVA: I – 4 I suggest that you always have this error covariance matrix and its correlation matrix printed and inspect the correlations. In theory, if you could measure a sufficient number of factors, covariates, etc. so that the only remaining variability is due to random noise, then these correlations should all go to 0. Inspection of these correlations is often a sobering experience because it demonstrates how far we have to go to be able to predict behavior.
What about the other matrices? In MANOVA, they will all have their analogues in the univariate ANOVA. For example, the variance in BDI due to psychotherapy calculated from a univariate ANOVA of the BDI would be the first diagonal element in the Vp matrix. The variance of HRS calculated from a univariate ANOVA is the second diagonal element in Vp. The variance in CSR due to the interaction between psychotherapy and drug as calculated from a univariate ANOVA will be the third diagonal element in V(p*m) .
The off diagonal elements are all covariances and should be interpreted as between group covariances. That is, in Vp, cov(1,2) = cov(BDI, HRS) tells us whether the psychotherapy group with the highest mean score on BDI also has the highest mean score on the HRS. If, for example, the cognitive therapy were more efficacious than the clinic therapy, then we should expect that all the covariances in Vp be large and positive. Again, cov(2,3) = cov(HRS, CSR) in the V(p*n) matrix has the following interpretation: if we control for the main effects of psychotherapy and medication, then do groups with high average scores on the Hamilton also tend to have high average scores on the relative's checklist?
In theory, if there were no main effect for psychotherapy on any of the measures, then all the elements of Vp will be 0. If there were no main effect for medication, then Vm will be all 0's. And if there were no
interaction, then all of V(p*m) would be 0's. It makes sense to have these matrices calculated and printed out so that you can inspect them. However, just as most programs for univariate ANOVAs do not give you the variance components, most computer programs for MANOVA do not give you the variance
component matrices. You can, however, calculate them by hand 2. Understanding MANOVA Understanding of MANOVA requires understanding of three basic principles.
They are: · An understanding of univariate ANOVA. · Remembering that mathematicians work very hard to be lazy. · A little bit of matrix algebra. Each of these topics will now be convered in detail. 2.1 Understanding Univariate ANOVA. Let us review univariate ANOVA by examining the expected results of a simple oneway ANOVA under the null hypothesis. The overall logic of ANOVA is to obtain two different estimates of the population variance; hence, the term analysis of variance.© Gregory Carey, 1998 MANOVA: I – 5 The first estimate is based on the variance within groups. The second estimate of the variance is based on the variance of the means of the groups. Let us examine the logic behind this by referring to an example of a oneway ANOVA. Suppose we select four groups of equal size and measure them on a single variable. Let n denote the sample size within each group. The null hypothesis assumes that the scores for individuals in each of the four groups are all sampled from a single normal distribution with mean m and variance s 2 . Thus, the expected value for the mean of the first group is m and the expected value of the variance of the first group is s 2 , the expected value for the mean of the second group is m and the expected value of the variance of the second group is s 2 , etc. The observed statistics and their expected values for this ANOVA are given in Table 1. Table 1. Observed and expected means and variances of four groups in a oneway ANOVA under the null hypothesis. Group 1 2 3 4 Sample Size: n n n n Means: Observed X 1 X 2 X 3 X 4 Expected m m m m Variances: Observed s 1 2 s 2 2 s 3 2 s 4 2 Expected s2s2s2s2 Now concentrate on the rows for the variances in Table 1. There are four observed variances, one for each group, and they all have expected values of s2. Instead of having four different estimates of s2, we can obtain a better overall estimate of s 2 by simply taking the average of the four estimates. That is, ) ss22=s12+ s22+ s32+ s424(2.1)The notation) ss22 is used to denote that this is the estimate of s2 derived from the observed variances. (Note that the only reason we could add up the four variances is because the four groups are independent. Were they dependent in some way, this could not be done.)Now examine the four means. A very famous theorem in statistics, the central limit theorem, states that when scores are normally distributed with mean m and variance s 2, then the the sampling distribution of means based on size n will be normally© Gregory Carey, 1998 MANOVA: I – 6 distributed with an overall mean of m and variance s2/n. Thus, the four means in Table 1 will be sampled from a single normal distribution with mean m and variance s2/n. A second estimate of s2 may now be obtained by going through three steps: (1) treat the four means as raw scores, (2) calculate the variance of the four means, and (3) multiplyingthe result by n. Let x denote the overall mean of the four means, then sx2= n (xi - x )2i=14å4 -1(2.2)where )sx2 denotes the estimate of s2 based on the means and not the variance of the means.
We now have two separate estimates of s 2, the first based on the within group variances ()ss22) and the second based on the means ()sx2 sx 2 ). If we take a ratio of the two estimates, we expect a value close to 1.0, or E)
s
x
2
)
s
s
2
2
æ
è
ç
ö
ø
÷ » 1. (2.3)
This is the logic of the simple oneway ANOVA, although it is most often
expressed in different terms. The estimate
)
s
s
2
2
from equation (2.1) is the mean squares
within groups. The estimate
)
s
x
2
from (2.2) is the mean squares between groups. And the
ratio in (2.3) is the F ratio expected under the null hypothesis. In order to generalize to
any number of groups, say g groups, we would perform the same step but substitute g in
place of 4 in Equation (2.2).
Now the above derivations all pertain to the null hypothesis. The alternative
hypothesis states that at least one of the four groups has been sampled from a different
normal distribution than the other three means. Here, for the sake of exposition, it is
assumed that scores in the four groups are sampled from different normal distributions
with respective means of m1, m2, m3, and m4, but with the same variance, s
2
.
Because the variances do not differ, the estimate derived from the observed group
variances in equation (2.1), or
)
s
s
2
2
, will remain a valid estimate of s
2
. However, the
variance derived from the means using equation (2.2) is no longer an estimate of s
2
. If we
performed the calculation on the right hand side of Equation (2.2), the expected results
would be
E n
( x
i - x )
2
i =1
4
å
4 -1
æ
è
ç
ö
ø
÷
÷
÷
= nsm
2
+
)
s
x
2
. (2.4)© Gregory Carey, 1998 MANOVA: I - 7
Consequently, the expectation of the F ratio becomes
E(F) =
nsm
2
+
)
s
x
2
)
s
s
2
2
=
)
s
x
2
)
s
s
2
2
+
nsm
2
)
s
s
2
2
»1 +
nsm
2
)
s
s
2
2
. (2.5)
Instead of having an expectation around 1.0 (which F has under the null hypothesis), the
expectation under the alternative hypothesis will be sometihing greater than 1.0. Hence,
the larger the F statistic, the more likely that the null hypothesis is false.
2.2 Mathematicians Are Lazy
The second step in understanding MANOVA is the recognition that
mathematicians are lazy. Mathematical indolence, as applied to MANOVA, is apparent
because mathematical statisticians do not bother to express the information in terms of
the estimates of variance that were outlined above. Instead, all the information is
expressed in terms of sums of squares and cross products. Although this may seem quite
foreign to the student, it does save steps in calculations--something that was important in
the past when computers were not available. We can explore this logic by once again
returning to a simple oneway ANOVA.
Recall that the two estimates of the population variance in a oneway ANOVA are
termed mean squares. Computationally, mean squares denote the quantity resulting from
dividing the sum of squares by its associated degrees of freedom. The between group
mean squares--which will now be called the hypothesis mean squares, is
MSh =
SSh
df
h
and the within group mean squares--which will now be called the error mean squares, is
MSe =
SSe
df
e
.
The F statistic for any hypothesis is defined as the mean squares for the
hypothesis divided by the mean squares for error. Using a little algebra, the F statistic
can be shown to be the product of two ratios. The first ratio is the degrees of freedom for
error divided by the degrees of freedom for the hypothesis. The second ratio is the sum
of squares for the hypothesis divided by the sum of squares for error. That is,© Gregory Carey, 1998 MANOVA: I - 8
F =
MSh
MSe
=
SS h
df
h
SSe
df
e
=
df
e
df
h
SSh
SSe
.
The role of mathematical laziness comes about by developing a new statistic--
called the A statistic here--that simplifies matters by removing the step of calculating the
mean squares. The A statistic is simply the F statistic multiplied by the degrees of
freedom for the hypothesis and divided by the degrees of freedom for error, or
A =
df
h
df
e
F =
df
h
df
e
df
e
df
h
æ
è
ç
ö
ø
÷
SSh
SSe
=
SSh
SSe
.
The chief advantage of using the A statistic is that one never has to go through the
trouble of calculating mean squares. Hence, given the overriding principle of mathematical
indolence, that, of course, must be adhered to at all costs in statistics, it is preferable to
develop ANOVA tables using the A statistic instead of the F statistic and to develop
tables for the critical values of the A statistic to replace tables of critical values for the F
statistic. Accordingly, we can refer to traditional ANOVA tables as “stupid,” and to
ANOVA tables using the F statistic as “enlightened.”
The following tables demonstrate the difference between a traditional, “stupid”
ANOVA and a modern, “enlightened” ANOVA.
Stupid ANOVA Table
Source
Sums of
Squares
Degrees of
Freedom
Mean
Squares F p
Hypothesis SSh dfh MSh
Error SSe df
e MSe
Total SSt df
t
Critical Values of F (a = .05) (Stupid)
df
e
(numerator)
dfh 1 2 3 4 5
1 161.45 199.50 215.71 224.58 230.16
2 18.51 19.00 19.16 19.25 19.30
3 10.13 9.55 9.28 9.12 9.01© Gregory Carey, 1998 MANOVA: I - 9
Enlightened ANOVA Table
Source
Sums of
Squares
Degrees of
Freedom A p
Hypothesis SSh dfh
Error SSe df
e
Total SSt df
t
Critical Values of A (a = .05)(Enlightened)
df
e
(numerator)
dfh 1 2 3 4 5
1 161.45 399.00 647.13 898.32 1150.80
2 9.26 19.00 28.74 38.50 48.25
3 3.38 6.37 9.28 12.16 15.02
2.3. Vectors and Matrices
The final step in understanding MANOVA is to express the information in terms
of vectors and matrices. Assume that instead of a single dependent variance in the
oneway ANOVA, there are three dependent variables. Under the null hypothesis, it is
assumed that scores on the three variables for each of the four groups are sampled from a
trivariate normal distribution mean vector
m =
m1
m2
m3
æ
è
ö
ø
and variance-covariance matrix
S =
s1
2
r12s1s2 r13s1s3
r12s1s2 s2
2
r23s2s3
r13s1s3 r23s2s3 s3
2
æ
è
ç
ç
ö
ø
.
(Recall that the quantity
r12s1s2
equals the covariance between variables 1 and 2.)
Under the null hypothesis, the scores for all those in group 1 will be sampled from
this distribution, as will the scores for all those individuals in groups 2, 3, and 4. The
observed and expected values for the four groups are given in Table 2.© Gregory Carey, 1998 MANOVA: I - 10
Table 2. Observed and expected statistics for the mean vectors and the variancecovariance matrices of four groups in a oneway MANOVA under the null hypothesis.
Group
1 2 3 4
Sample Size: n n n n
Mean Vector: Observed x 1 x 2 x 3 x 4
Expected m m m m
Covariance Matrix:
Observed S1 S2 S3 S4
Expected S S S S
Note the resemblance between Tables 1 and 2. The only difference is that Table 2
is written in matrix notation. Indeed, if we consider the elements in Table 1 as (1 by 1)
vectors or (1 by 1) matrices and then rewrite Table 1, we would get Table 2!
Once again, how can we obtain different estimates of S? Again, concentrate on
the rows marked variances in Table 2. The easiest way to estimate S is to add up the
covariance matrices for the four groups and divide by 4--or, in other words, take the
average observed covariance matrix:
S
ˆ
w =
S1 + S2 + S3 + S4
4
. (2.6)
Note how (2.6) is identical to (2.1) except that (2.6) is expressed in matrix notation.
What about the means? Under the null hypothesis, the means will be sampled
from a trivariate normal distribution with mean vector m and covariance matrix S/n.
Consequently, to obtain an estimate of S based on the mean vectors for the four groups,
we proceed with the same logic as that in the oneway ANOVAiven in section 2.1, but
now apply it to the vectors of means. That is, treat the means as if they were raw scores
and calculate the covariance matrix for the three “variables;” then multiply this result by
n. Let
X
ij
denote the mean for the ith group on the jth variable. The data would look like
this:
X
11 X
12 X
13
X
21 X
22 X
23
X
31 X
32 X
33
X
41 X
42 X
43© Gregory Carey, 1998 MANOVA: I - 11
This is identical to a data matrix with four observations and three variables. In this case,
the observations are the four groups and the “scores” or numbers are the means. Let
SSCPX
denote the sums of squares and cross products matrix used to calculate the
covariance matrix for these three variables. Then the estimate of S based on the means of
the four groups will be
ˆ
S
b = n
SSCPX
4 -1
. (2.7)
Note how (2.7) is identical in form to (2.2) except that (2.7) is expressed in matrix
notation.
We now have two different estimates of S. The first,
ˆ
S
w
, is derived from the
average covariance matrix within the groups, and the second,
ˆ
S
b
, is derived from the
covariance matrix for the group mean vectors. Because both of them measure the same
quantity, we expect
E(
ˆ
S
b
ˆ
S
w
-1
) = I. (2.8)
or an identity matrix. Note that (2.8) is identical to (2.3) except that (2.8) is written in
matrix notation. If the matrices in (2.8) were simply (1 by 1) matrices then their
expectation would be a (1 by 1) identity matrix or simply 1.0. Just what we got in (2.3)!
Now examine the alternative hypothesis. Under the alternative hypothesis it is
assumed that each group is sampled from a multivariate normal distribution that has the
same covariance matrix, say S. Consequently, the average of the covariance matrices for
the four groups in equation (2.6) remains an estimate of S. However, under the alternate
hypothesis, the mean vector for at least one group is sampled from a multivariate normal
with a different mean vector than the other groups. Consequently, the covariance matrix
for the observed mean vectors will reflect the covariance due to true mean differences, say
Sm
, plus the covariance matrix for sampling error, S/n. Thus, multiplying the covariance
matrix of the means by n now gives
nSm + S . (2.9)
Once again, (2.9) is the same as (2.4) except for its expression in matrix notation. What is
the expectation of the estimate from the observed means postmultiplied by the estimate
from the observed covariance matrices under the alternative hypothesis? Substituting the
expression in (2.9) for
ˆ
S
b
in Equation (2.8) gives
E(
ˆ
S
b
ˆ
S
w
-1
) = (nSm + S)S
-1
= nSmS + SS
-1
= nSmS + I . (2.10)© Gregory Carey, 1998 MANOVA: I - 12
Although it is a bit difficult to see how, (2.10) is the same as (2.5) except that (2.10) is
expressed in matrix notation. Note that (2.10) is the sum of an identity matrix and
another term. The identity matrix will have 1's on the diagonal, so the diagonals of the
result will always be "1 + something." That "something" will always be a positive number
[for technical reasons in matrix algebra]. Consequently, the diagonals in (2.10) will
always be greater than 1.0. Again, if we consider the matrices in (2.10) as (1 by 1)
matrices, we can verify that the expectation will always be greater than 1.0, just as we
found in (2.5).
This exercise demonstrates how MANOVA is a natural extension of ANOVA.
The only remaining point is to add the laziness of mathematical statisticians. In
ANOVA, we saw how this laziness saved a compuational step by avoiding calculations
of the means squares (which is simply another term for the variance) and expressing all
the information in terms of the sums of squares. The same applies to MANOVA, instead
of calculating the mean squares and mean products matrix (which is simply another term
for a covariance matrix), MANOVA avoids this step by expressing all the information in
terms of the sums of squares and cross products matrices.
The A statistic developed in section 2.2 was simply the ratio of the sum of
squares for an hypothesis and the sum of squares for error. Let H denote the hypothesis
sums of squares and cross products matrix, and let E denote the error sums of squares and
cross products matrix. The multivariate equivalent of the A statistic is the matrix A which
is
A = HE
-1
(2.11)
Verify how Equation 2.11 is the matrix analogue of the A statistic given in section 2.2.
Notice how mean squares (or, in other terms, covariance matrices) disappear from
MANOVA just as they did for ANOVA. All hypothesis tests may be performed on
matrix A. Parenthetically, note that because both H and E are symmetric, HE
-1
= E
-1
H.
This is one special case where the order of matrix multiplication does not matter.
3.0. Hypothesis Testing in MANOVA
All current MANOVA tests are made on A = E
-1
H. That's the good news. The
bad news is that there are four different multivariate tests that are made on E
-1
H. Each of
the four test statistics has its own associated F ratio. In some cases the four tests give an
exact F ratio for testing the null hypothesis and in other cases the F ratio is approximated.
The reason for four different statistics and for approximations is that the mathematics of
MANOVA get so complicated in some cases that no one has ever been able to solve
them. (Technically, the math folks can't figure out the sampling distribution of the F
statistic in some multivariate cases.)
To understand MANOVA, it is not necessary to understand the derivation of the
statistics. Here, all that is mentioned is their names and some properties. In terms of
notation, assume that there are q dependent variables in the MANOVA, and let li
denote
the ith eigenvalue of matrix A which, of course, equals HE
-1
.© Gregory Carey, 1998 MANOVA: I - 13
The first statistic is Pillai's trace. Some statisticians consider it to be the most
powerful and most robust of the four statistics. The formula is
Pillai's trace = trace[H(H + E)
-1
] =
li
1+ l i
i=1
q
å . (3.1)
The second test statistic is Hotelling-Lawley's trace.
Hotelling-Lawley's trace = trace(A ) = trace(HE
-1
) = li
i=1
q
å . (3.2)
The third is Wilk's lambda (L). (Here, the upper case, Greek L is used for Wilk’s lambda
to avoid confusion with the lower case, Greek l often used to denote an eigenvalue.
However, many texts use the lower case lambda as the notation for Wilk’s lambda.)
Wilk’s L was the first MANOVA test statistic developed and is very important for
several multivariate procedures in addition to MANOVA.
Wilk's lambda = L =
| E |
| H + E |
=
1
1+ li
i=1
q
Õ . (3.3)
The quantity (1 - L) is often interpreted as the proportion of variance in the dependent
variables explained by the model effect. However, this quantity is not unbiased and can
be quite misleading in small samples.
The fourth and last statistic is Roy's largest root. This gives an upper bound for
the F statistic.
Roy's largest root = max(li
). (3.4)
or the maximum eigenvalue of A = HE
-1
. (Recall that a "root" is another name for an
eigenvalue.) Hence, this statistic could also be called Roy's largest eigenvalue. (In case you
know where Roy's smallest root is, please let him know.)
Note how all the formula in equations (3.1) through (3.4) are based on the
eigenvalues of A = HE
-1
. This is the major reason why statstical programs such as SAS
print out the eigenvalues and eigenvectors of A = HE
-1
.
Once the statistics in (3.1) through (3.4) are obtained, they are translated into F
statistics in order to test the null hypothesis. The reason for this translation is identical
to the reason for converting Hotelling's T
2
--the easy availability of published tables of the
F distribution. The important issue to recognize is that in some cases, the F statistic is
exact and in other cases it is approximate. Good statistical packages will inform you
whether the F is exact or approximate.
In some cases, the four will generate identical F statistics and identical
probabilities. In other's they will differ. When they differ, Pillai's trace is often used
because it is the most powerful and robust. Because Roy's largest root is an upper bound© Gregory Carey, 1998 MANOVA: I - 14
on F, it will give a lower bound estimate of the probability of F. Thus, Roy's largest root
is generally disregarded when it is significant but the others are not significant.
Multivariate Analysis of Variance
(MANOVA)
Aaron French, Marcelo Macedo, John Poulsen, Tyler Waterson and Angela Yu
Keywords: MANCOVA, special cases, assumptions, further reading, computations
Introduction
Multivariate analysis of variance (MANOVA) is simply an ANOVA with several
dependent variables. That is to say, ANOVA tests for the difference in means
between two or more groups, while MANOVA tests for the difference in two or more
vectors of means.
For example, we may conduct a study where we try two different textbooks, and we
are interested in the students' improvements in math and physics. In that case,
improvements in math and physics are the two dependent variables, and our
hypothesis is that both together are affected by the difference in textbooks. A
multivariate analysis of variance (MANOVA) could be used to test this hypothesis.
Instead of a univariate F value, we would obtain a multivariate F value (Wilks' λ)
based on a comparison of the error variance/covariance matrix and the effect
variance/covariance matrix. Although we only mention Wilks' λ here, there are other
statistics that may be used, including Hotelling's trace and Pillai's criterion. The
"covariance" here is included because the two measures are probably correlated and
we must take this correlation into account when performing the significance test.
Testing the multiple dependent variables is accomplished by creating new dependent
variables that maximize group differences. These artificial dependent variables are
linear combinations of the measured dependent variables.
Research Questions
The main objective in using MANOVA is to determine if the response variables
(student improvement in the example mentioned above), are altered by the
observer’s manipulation of the independent variables. Therefore, there are several
types of research questions that may be answered by using MANOVA:
1) What are the main effects of the independent variables?
2) What are the interactions among the independent variables?
3) What is the importance of the dependent variables?4) What is the strength of association between dependent variables?
5) What are the effects of covariates? How may they be utilized?
Results
If the overall multivariate test is significant, we conclude that the respective effect
(e.g., textbook) is significant. However, our next question would of course be whether
only math skills improved, only physics skills improved, or both. In fact, after
obtaining a significant multivariate test for a particular main effect or interaction,
customarily one would examine the univariate F tests for each variable to interpret
the respective effect. In other words, one would identify the specific dependent
variables that contributed to the significant overall effect.
MANOVA is useful in experimental situations where at least some of the independent
variables are manipulated. It has several advantages over ANOVA. First, by
measuring several dependent variables in a single experiment, there is a better
chance of discovering which factor is truly important. Second, it can protect against
Type I errors that might occur if multiple ANOVA’s were conducted independently.
Additionally, it can reveal differences not discovered by ANOVA tests.
However, there are several cautions as well. It is a substantially more complicated
design than ANOVA, and therefore there can be some ambiguity about which
independent variable affects each dependent variable. Thus, the observer must
make many potentially subjective assumptions. Moreover, one degree of freedom is
lost for each dependent variable that is added. The gain of power obtained from
decreased SS error may be offset by the loss in these degrees of freedom. Finally,
the dependent variables should be largely uncorrelated. If the dependent variables
are highly correlated, there is little advantage in including more than one in the test
given the resultant loss in degrees of freedom. Under these circumstances, use of a
single ANOVA test would be preferable.
Assumptions
Normal Distribution: - The dependent variable should be normally distributed within
groups. Overall, the F test is robust to non-normality, if the non-normality is caused
by skewness rather than by outliers. Tests for outliers should be run before
performing a MANOVA, and outliers should be transformed or removed.
Linearity - MANOVA assumes that there are linear relationships among all pairs of
dependent variables, all pairs of covariates, and all dependent variable-covariate
pairs in each cell. Therefore, when the relationship deviates from linearity, the power
of the analysis will be compromised.
Homogeneity of Variances: - Homogeneity of variances assumes that the dependent
variables exhibit equal levels of variance across the range of predictor variables. Remember that the error variance is computed (SS error) by adding up the sums of
squares within each group. If the variances in the two groups are different from each
other, then adding the two together is not appropriate, and will not yield an estimate
of the common within-group variance. Homoscedasticity can be examined
graphically or by means of a number of statistical tests.
Homogeneity of Variances and Covariances: - In multivariate designs, with multiple
dependent measures, the homogeneity of variances assumption described earlier
also applies. However, since there are multiple dependent variables, it is also
required that their intercorrelations (covariances) are homogeneous across the cells
of the design. There are various specific tests of this assumption.
Special Cases
Two special cases arise in MANOVA, the inclusion of within-subjects independent
variables and unequal sample sizes in cells.
Unequal sample sizes - As in ANOVA, when cells in a factorial MANOVA have
different sample sizes, the sum of squares for effect plus error does not equal the
total sum of squares. This causes tests of main effects and interactions to be
correlated. SPSS offers and adjustment for unequal sample sizes in MANOVA.
Within-subjects design - Problems arise if the researcher measures several different
dependent variables on different occasions. This situation can be viewed as a withinsubject independent variable with as many levels as occasions, or it can be viewed
as separate dependent variables for each occasion. Tabachnick and Fidell (1996)
provide examples and solutions for each situation. This situation often lends itself to
the use of profile analysis, which is explained below.
Additional Limitations
Outliers - Like ANOVA, MANOVA is extremely sensitive to outliers. Outliers may
produce either a Type I or Type II error and give no indication as to which type of
error is occurring in the analysis. There are several programs available to test for
univariate and multivariate outliers.
Multicollinearity and Singularity - When there is high correlation between dependent
variables, one dependent variable becomes a near-linear combination of the other
dependent variables. Under such circumstances, it would become statistically
redundant and suspect to include both combinations.
MANCOVA
MANCOVA is an extension of ANCOVA. It is simply a MANOVA where the artificial
DVs are initially adjusted for differences in one or more covariates. This can reduce
error "noise" when error associated with the covariate is removed. For Further Reading:
Cooley, W.W. and P. R. Lohnes. 1971. Multivariate Data Analysis. John
Wiley & Sons, Inc.
George H. Dunteman (1984). Introduction to multivariate analysis.
Thousand Oaks, CA: Sage Publications. Chapter 5 covers
classification procedures and discriminant analysis.
Morrison, D.F. 1967. Multivariate Statistical Methods. McGraw-Hill: New
York.
Overall, J.E. and C.J. Klett. 1972. Applied Multivariate Analysis.
McGraw-Hill: New York.
Tabachnick, B.G. and L.S. Fidell. 1996. Using Multivariate Statistics.
Harper Collins College Publishers: New York.
Webpages:
Site Link
Statsoft text
entry on
MANOVA
http://www.statsoft.com/textbook/stathome.html
EPA Statistical
Primer
http://www.epa.gov/bioindicators/primer/html/manova.html
Introduction to
MANOVA
http://ibgwww.colorado.edu/~carey/p7291dir/handouts/manova1.pdf
Practical guide
to MANOVA for
SAS
http://ibgwww.colorado.edu/~carey/p7291dir/handouts/manova2.pdf
Computations
First, the total sum-of-squares is partitioned into the sum-of-squares between groups
(SSbg) and the sum-of-squares within groups (SSwg):
SStot
= SSbg + SSwg
This can be expressed as: The SSbg is then partitioned into variance for each IV and the interactions between
them.
In a case where there are two IVs (IV1 and IV2), the equation looks like this:
Therefore, the complete equation becomes:
Because in MANOVA there are multiple DVs, a column matrix (vector) of values for
each DV is used. For two DVs (a and b) with n values, this can be represented:
Similarly, there are column matrices for IVs - one matrix for each level of every IV.
Each matrix of IVs for each level is composed of means for every DV. For "n" DVs
and "m" levels of each IV, this is written:
Additional matrices are calculated for cell means averaged over the individuals in
each group.
Finally, a single matrix of grand means is calculated with one value for each DV
averaged across all individuals in matrix.
Differences are found by subtracting one matrix from another to produce new
matrices. From these new matrices the error term is found by subtracting the GM
matrix from each of the DV individual scores:
Next, each column matrix is multiplied by each row matrix:
These matrices are summed over rows and groups, just as squared differences are
summed in ANOVA. The result is an S matrix (also known as: "sum-of-squares and
cross-products," "cross-products," or "sum-of-products" matrices.)
For a two IV, two DV example:
Stot
= SIV1 + SIV2 + Sinteraction + Swithin-group error
Determinants (variance) of the S matrices are found. Wilks’ λ is the test statistic
preferred for MANOVA, and is found through a ratio of the determinants:
An estimate of F can be calculated through the following equations:
Where,
Finally, we need to measure the strength of the association. Since Wilks’ λ is equal to
the variance not accounted for by the combined DVs, then (1 – λ) is the variance that
is accounted for by the best linear combination of DVs.
However, because this is summed across all DVs, it can be greater than one and
therefore less useful than:
Other statistics can be calculated in addition to Wilks’ λ. The following is a short list of
some of the popularly reported test statistics for MANOVA:
• Wilks’ λ = pooled ratio of error variances to effect variance plus error variance
• This is the most commonly reported test statistic, but not always the
best choice.
• Gives an exact F-statistic
• Hotelling’s trace = pooled ratio of effect variance to error variance
∑
=
=
s
i
T i
1
λ
• Pillai-Bartlett criterion = pooled effect variances
• Often considered most robust and powerful test statistic.
• Gives most conservative F-statistic.
∑
=
+
=
S
i i
i
V
1
1 λ
λ
• Roy’s Largest Root = largest eigenvalue
o Gives an upper-bound of the F-statistic.
o Disregard if none of the other test statistics are significant.
MANOVA works well in situations where there are moderate correlations between
DVs. For very high or very low correlation in DVs, it is not suitable: if DVs are too
correlated, there is not enough variance left over after the first DV is fit, and if DVs
are uncorrelated, the multivariate test will lack power anyway, so why sacrifice
degrees of freedom?
Multivariate analysis of variance
(MANOVA)
Aaron French and John Poulsen
Keywords: MANCOVA, special cases, assumptions, further reading,
computations
Introduction
Multivariate analysis of variance (MANOVA) is simply an ANOVA with several
dependent variables. For example, we may conduct a study where we try two
different textbooks, and we are interested in the students' improvements in math
and physics. In that case, improvements in math and physics are the two
dependent variables, and our hypothesis is that both together are affected by the
difference in textbooks. A multivariate analysis of variance (MANOVA) could be
used to test this hypothesis. Instead of a univariate F value, we would obtain a
multivariate F value (Wilks' lambda) based on a comparison of the error
variance/covariance matrix and the effect variance/covariance matrix. Although
we only mention Wilks' lambda here, there are other statistics that may be used,
including Hotelling's trace and Pillai's criterion. The "covariance" here is included
because the two measures are probably correlated and we must take this
correlation into account when performing the significance test.
Testing the multiple dependent variables is accomplished by creating new
dependent variables that maximize group differences. These artificial dependent
variables are linear combinations of the measured dependent variables.
Results
If the overall multivariate test is significant, we conclude that the respective effect
(e.g., textbook) is significant. However, our next question would of course be
whether only math skills improved, only physics skills improved, or both. In fact,
after obtaining a significant multivariate test for a particular main effect or
interaction, customarily one would examine the univariate F tests for each
variable to interpret the respective effect. In other words, one would identify the
specific dependent variables that contributed to the significant overall effect.
MANOVA is useful in experimental situations where at least some of the
independent variables are manipulated. It has several advantages over
ANOVA. First, by measuring several dependent variables in a single experiment,
there is a better chance of discovering which factor is truly important. Second, it can protect against Type I errors that might occur if multiple ANOVA’s were
conducted independently. Additionally, it can reveal differences not discovered
by ANOVA tests.
However, there are several cautions as well. It is a substantially more
complicated design than ANOVA, and therefore there can be some ambiguity as
to which independent variable affects each dependent variable. Moreover, one
degree of freedom is lost for each dependent variable that is added. The gain of
power obtained from decreased SS error may be offset by the loss in these
degrees of freedom. Finally, the dependent variables should be largely
uncorrelated. If the dependent variables are highly correlated, there is little
advantage in including more than one in the test given the resultant loss in
degrees of freedom.
Assumptions
Normal Distribution:
The dependent variable should be normally distributed within groups. Overall,
the F test is robust to non-normality if it is caused by skewness rather than
outliers. Tests for outliers should be run before performing a MANOVA, and
outliers should be transformed or removed.
Homogeneity of Variances:
Homogeneity of variances assumes that the dependent variables exhibit equal
levels of variance across the range of predictor variables. Remember that the
error variance is computed (SS error) by adding up the sums of squares within
each group. If the variances in the two groups are different from each other, then
adding the two together is not appropriate, and will not yield an estimate of the
common within-group variance. Homoscedasticity can be examined graphically
or by means of a number of statistical tests.
Homogeneity of Variances and Covariances:
In multivariate designs, with multiple dependent measures, the homogeneity of
variances assumption described earlier also applies. However, since there are
multiple dependent variables, it is also required that their intercorrelations
(covariances) are homogeneous across the cells of the design. There are various
specific tests of this assumption.
Special Cases
Two special cases arise in MANOVA, the inclusion of within-subjects
independent variables and unequal sample sizes in cells.
Unequal sample sizes
As in ANOVA, when cells in a factorial MANOVA have different sample sizes, the
sum of squares for effect plus error does not equal the total sum of squares.This causes tests of main effects and interactions to be correlated. SPSS offers
and adjustment for unequal sample sizes in MANOVA.
Within-subjects design
Problems arise if the researcher measures several different dependent variables
on different occasions. This situation can be viewed as a within-subject
independent variable with as many levels as occasions. Or, it can be viewed as
a separate dependent variables for each occasion. Tabachnick and Fidell (1996)
provide examples and solutions for each situation.
MANCOVA
MANCOVA is an extension of ANCOVA. It is simply a MANOVA where the
artificial DVs are initially adjusted for differences in one or more covariates. This
can reduce error "noise" when error associated with the covariate is removed.
For Further Reading:
Cooley, W.W. and P. R. Lohnes. 1971. Multivariate Data Analysis.
John Wiley & Sons, Inc.
George H. Dunteman (1984). Introduction to multivariate analysis.
Thousand Oaks, CA: Sage Publications. Chapter 5 covers
classification procedures and discriminant analysis.
Morrison, D.F. 1967. Multivariate Statistical Methods. McGraw-Hill:
New York.
Overall, J.E. and C.J. Klett. 1972. Applied Multivariate Analysis.
McGraw-Hill: New York.
Tabachnick, B.G. and L.S. Fidell. 1996. Using Multivariate
Statistics. Harper Collins College Publishers: New York.
Webpages:
www.statsoft.com/textbook/stathome.html
Computations
First, the total sum-of-squares is partitioned into the sum-of-squares between
groups (SSbg) and the sum-of-squares within groups (SSwg):
SStot = SSbg + SSwg
This can be expressed as:The SSbg is then partitioned into variance for each IV and the interactions
between them.
In a case where there are two IVs (IV1 and IV2), the equation looks like this:
Therefore, the complete equation becomes:
Because in MANOVA there are multiple DVs, a column matrix (vector) of values
for each DV is used. For two DVs (a and b) with n values, this can be
represented:
Similarly, there are column matrices for IVs - one matrix for each level of every
IV. Each matrix of IVs for each level is composed of means for every DV. For "n"
DVs and "m" levels of each IV, this is written:Additional matrices are calculated for cell means averaged over the individuals in
each group.
Finally, a single matrix of grand means is calculated with one value for each DV
averaged across all individuals in matrix.
Differences are found by subtracting one matrix from another to produce new
matrices. From these new matrices the error term is found by subtracting the GM
matrix from each of the DV individual scores:
Next, each column matrix is multiplied by each row matrix:
These matrices are summed over rows and groups, just as squared differences
are summed in ANOVA. The result is an S matrix (also known as: "sum-ofsquares and cross-products," "cross-products," or "sum-of-products" matrices.)
For a two IV, two DV example:
Stot
= SIV1 + SIV2 + Sinteraction + Swithin-group errorDeterminants (variance) of the S matrices are found. Wilks’ Lambda is the test
statistic preferred for MANOVA, and is found through a ratio of the
determininants:
An estimate of F can be calculated through the following equations:
Where,
Finally, we need to measure the strength of the association. Since Wilks’ Lambda
is equal to the variance not accounted for by the combined DVs, then (1 –
Lambda) is the variance that is accounted for by the best linear combination of
DVs.
However, because this is summed across all DVs, it can be greater than one and
therefore less useful than:Other statistics can be calculated in addition to Wilks’ Lambda. The following is a
short list of some of the popularly reported test statistics for MANOVA:
· Wilks’ Lambda = pooled ratio of error variances to effect variance plus
error variance
· Hotelling’s trace = pooled ratio of effect variance to error variance
· Pillai’s criterion = pooled effect variances
Use Wilks’ Lambda because it is the most commonly available and reported,
however Pillai’s criterion is more robust and therefore more appropriate when
there are small or unequal sample sizes.
MANOVA works well in situations where there are moderate correlations
between DVs. For very high or very low correlation in DVs, it is not suitable: if
DVs are too correlated, there isn’t enough variance left over after the first DV is
fit, and if DVs are uncorrelated, the multivariate test will lack power anyway, so
why sacrifice degrees of freedom?
Multivariate Analysis of Variance (Manova)description | simple example | MAIA example | how it works | caveats
Description: Manova creates a linear combination of the dependent variables (DV's) and then tests
for differences in the new variable using methods similar to Anova. The independent variable (IV) used
to group the cases is categorical. Manova tests whether the categorical variable explains a significant
amount of variability in the new dependent variable.
{2 or more DV's} = f (1 or more categorical IV's).
Simple example: Suppose you want to test whether stream size and shape differ across ecoregions.
The independent categorical variable would be ecoregion and the set of dependent variables might
include {width, depth, flow, and gradient}. The dependent variables are correlated which is
appropriate for Manova. Manova constructs new variables from {width, depth, flow, and gradient} and
tests whether the new composite variables differs across ecoregions.
MAIA example: For the Maryland fish IBI, Roth et al. (1999) first grouped stream sites using cluster
analysis of fish species data. The cluster analysis computed the distance between each set of species
for every pair of sites and yielded a dendogram that grouped sites by cluster. Cluster analysis does not
have statistical testing associated with it, but the authors were interested in determining which
clusters were significantly different. To test for significance, they used a Manova model.
For the Manova model, the relative abundances of the different fish species were the dependent
variables; and they used cluster assignment as the independent variable. They tested each branching
point successively down the cluster tree for statistical significance.
Figure
Figure: Schematic representation of the Manova analysis for the Maryland fish index. (The complete cluster tree was too
complicated to illustrate here.) In the sketch above, the first branch of the cluster analysis was significant, A and B included
significantly different fish assemblages. At the next level, AA and AB were not significantly different but BA and BB were.
Successive branches are not pictured, but the same algorithm was applied down each branch. Thus, this diagram illustrates
three significant clusters, sites grouped as A, BA and BB.
How the method works: A new variable is created that combines all the dependent variables on the
left hand side of the equation such that the differences between group means are maximized. (The f-
statistic from Anova is maximized, that is, the ratio of explained variance to error variance). The
simplest significance test treats the first, new variable just like a single dependent variable in Anova,
and uses the tests as in Anova. Additional, multivariate tests can also be computed that involve
multiple new variables derived from the initial set of dependent variables.
Assumptions/limitations: Dependent variables can be correlated or independent of each other. Like
Anova, Manova isn't too bothered by slight departures from normality, but extreme outliers can be
more of a problem.
Manova can require rather large sample sizes for complicated models because the number of cases in
each category must be larger than the number of dependent variables. Manova also prefers that the
groups have a similar number of cases in each group. In addition, Manova expects that the variance of
dependent variables and the correlation between them are similar within groups.
2 - Manova 4.3.05 25
Multivariate Analysis of Variance
What Multivariate Analysis of Variance is
The general purpose of multivariate analysis of variance (MANOVA) is to determine
whether multiple levels of independent variables on their own or in combination with one
another have an effect on the dependent variables. MANOVA requires that the dependent
variables meet parametric requirements.
When do you need MANOVA?
MANOVA is used under the same circumstances as ANOVA but when there are multiple
dependent variables as well as independent variables within the model which the
researcher wishes to test. MANOVA is also considered a valid alternative to the repeated
measures ANOVA when sphericity is violated.
What kinds of data are necessary?
The dependent variables in MANOVA need to conform to the parametric assumptions.
Generally, it is better not to place highly correlated dependent variables in the same
model for two main reasons. First, it does not make scientific sense to place into a model
two or three dependent variables which the researcher knows measure the same aspect of 2 - Manova 4.3.05 26
outcome. (However, this is point will be influenced by the hypothesis which the
researcher is testing. For example, subscales from the same questionnaire may all be
included in a MANOVA to overcome problems associated with multiple testing.
Subscales from most questionnaires are related but may represent different aspects of the
dependent variable.) The second reason for trying to avoid including highly correlated
dependent variables is that the correlation between them can reduce the power of the
tests. If MANOVA is being used to reduce multiple testing, this loss in power needs to be
considered as a trade-off for the reduction in the chance of a Type I error occurring.
Homogeneity of variance from ANOVA and t tests becomes homogeneity of variance
covariance in MANOVA models. The amount of variance within each group needs to be
comparable so that it can be assumed that the groups have been drawn from a similar
population. Furthermore it is assumed that these results can be pooled to produce an error
value which is representative of all the groups in the analysis. If there is a large difference
in the amount of error within each group the estimated error measure for the model will
be misleading.
How much data?
There needs to be more participants than dependent variables. If there were only one
participant in any one of the combination of conditions, it would be impossible to
determine the amount of variance within that combination (since only one data point
would be available). Furthermore, the statistical power of any test is limited by a small
sample size. (A greater amount of variance will be attributed to error in smaller sample 2 - Manova 4.3.05 27
sizes, reducing the chances of a significant finding.) A value known as Box’s M, given by
most statistical programs, can be examined to determine whether the sample size is too
small. Box’s M determines whether the covariance in different groups is significantly
different and must not be significant if one wishes to demonstrate that the sample sizes in
each cell are adequate. An alternative is Levene’s test of homogeneity of variance which
tolerates violations of normality better than Box's M. However, rather than directly
testing the size of the sample it examines whether the amount of variance is equally
represented within the independent variable groups.
In complex MANOVA models the likelihood of achieving robust analysis is intrinsically
linked to the sample size. There are restrictions associated with the generalizability of the
results when the sample size is small and therefore researchers should be encouraged to
obtain as large a sample as possible.
Example of MANOVA
Considering an example may help to illustrate the difference between ANOVAs and
MANOVAs. Kaufman and McLean (1998) used a questionnaire to investigate the
relationship between interests and intelligence. They used the Kaufman Adolescent and
Adult Intelligence Test (KAIT) and the Strong Interest Inventory (SII) which contained
six subscales on occupational themes (GOT) and 23 Basic Interest Scales (BISs).
Kaufman et al. used a MANOVA model which had four independent variables: age,
gender, KAIT IQ and Fluid-Crystal intelligence (F-C). The dependent variables were the
six occupational theme subscales (GOT) and the twenty-three Basic Interest Scales (BIS). 2 - Manova 4.3.05 28
In Table 2.1 the dependent variables are listed in columns 3 and 4. The independent
variables are listed in column 2, with the increasingly complex interactions being shown
below the main variables.
If an ANOVA had been used to examine these data, each of the GOT and BIS subscales
would have been placed in a separate ANOVA. However, since the GOT and BIS scales
are related, the results from separate ANOVAs would not be independent. Using multiple
ANOVAs would increase the risk of a Type I error (a significant finding which occurs by
chance due to repeating the same test a number of times).
Kaufman and McLean used the Wilks’ lambda multivariate statistic (similar to the F
values in univariate analysis) to consider the significance of their results and reported
only the interactions which were significant. These are shown as Sig in Table 2.1. The
values which proved to be significant are the majority of the main effects and one of the
2-way interactions. Note that although KAIT IQ had a significant main effect none of the
interactions which included this variable were significant. On the other hand, age and
gender show a significant interaction in the effect which they have on the dependent
variables.
What a multivariate analysis of variance does
Like an ANOVA, MANOVA examines the degree of variance within the independent
variables and determines whether it is smaller than the degree of variance between the
independent variables. If the within subjects variance is smaller than the between subjects
variance it means the independent variable has had a significant effect on the dependent 2 - Manova 4.3.05 29
Table 2.1. The different aspects of the data considered by the MANOVA model used by
Kaufman and McLean (1998).
Level Independent variables 6 GOT
subscales
23 BIS
Age Sig
Gender Sig Sig
KAIT IQ Sig Sig
Main Effects
F-C
Age x Gender Sig Sig
Age x KAIT IQ
Age x F-C
Gender x KAIT IQ
Gender x F-C
2-way Interactions
KAIT IQ x F-C
Age x Gender x KAIT IQ
Age x Gender x F-C
Age x KAIT IQ x F-C
Gender x KAIT IQ x F-C
3-way Interactions
Age KAIT IQ x F-C
4-way Interactions Age x Gender x KAIT IQ x F-C
2 - Manova 4.3.05 30
variables. There are two main differences between MANOVAs and ANOVAs. The first is
that MANOVAs are able to take into account multiple independent and multiple
dependent variables within the same model, permitting greater complexity. Secondly,
rather than using the F value as the indicator of significance a number of multivariate
measures (Wilks’ lambda, Pillai’s trace, Hotelling trace and Roy’s largest root) are used.
(An explanation of these multivariate statistics is given below).
MANOVA deals with the multiple dependent variables by combining them in a linear
manner to produce a combination which best separates the independent variable groups.
An ANOVA is then performed on the newly developed dependent variable. In
MANOVAs the independent variables relevant to each main effect are weighted to give
them priority in the calculations performed. In interactions the independent variables are
equally weighted to determine whether or not they have an additive effect in terms of the
combined variance they account for in the dependent variable/s.
The main effects of the independent variables and of the interactions are examined with
“all else held constant”. The effect of each of the independent variables is tested
separately. Any multiple interactions are tested separately from one another and from any
significant main effects. Assuming there are equal sample sizes both in the main effects
and the interactions, each test performed will be independent of the next or previous
calculation (except for the error term which is calculated across the independent
variables).
There are two aspects of MANOVAs which are left to researchers: first, they decide
which variables are placed in the MANOVA. Variables are included in order to address a 2 - Manova 4.3.05 31
particular research question or hypothesis, and the best combination of dependent
variables is one in which they are not correlated with one another, as explained above.
Second, the researcher has to interpret a significant result. A statistical main effect of an
independent variable implies that the independent variable groups are significantly
different in terms of their scores on the dependent variable. (But this does not establish
that the independent variable has caused the changes in the dependent variable. In a study
which was poorly designed, differences in dependent variable scores may be the result of
extraneous, uncontrolled or confounding variables.)
To tease out higher level interactions in MANOVA, smaller ANOVA models which
include only the independent variables which were significant can be used in separate
analyses and followed by post hoc tests. Post hoc and preplanned comparisons compare
all the possible paired combinations of the independent variable groups e.g. for three
ethnic groups of white, African and Asian the comparisons would be: white v African,
white v Asian, African v Asian. The most frequently used preplanned and post hoc tests
are Least Squares Difference (LSD), Scheffe, Bonferroni, and Tukey. The tests will give
the mean difference between each group and a p value to indicate whether the two groups
differ significantly.
The post hoc and preplanned tests differ from one another in how they calculate the p
value for the mean difference between groups. Some are more conservative than others.
LSD perform a series of t tests only after the null hypothesis (that there is no overall
difference between the three groups) has been rejected. It is the most liberal of the post
hoc tests and has a high Type I error rate. The Scheffe test uses the F distribution rather
than the t distribution of the LSD tests and is considered more conservative. It has a high 2 - Manova 4.3.05 32
Type II error rate but is considered appropriate when there are a large number of groups
to be compared. The Bonferroni approach uses a series of t tests but corrects the
significance level for multiple testing by dividing the significance levels by the number of
tests being performed (for the example given above this would be 0.05/3). Since this test
corrects for the number of comparisons being performed, it is generally used when the
number of groups to be compared is small. Tukey's Honesty Significance Difference test
also corrects for multiple comparisons, but it considers the power of the study to detect
differences between groups rather than just the number of tests being carried out i.e. it
takes into account sample size as well as the number of tests being performed. This makes
it preferable when there are a large number of groups being compared, since it reduces the
chances of a Type I error occurring.
The statistical packages which perform MANOVAs produce many figures in their output,
only some of which are of interest to the researcher.
Sum of Squares: The sum of squares measure found in a MANOVA, like that reported in
the ANOVA, is the measure of the squared deviations from the mean both within and
between the independent variable. In MANOVA, the sums of squares are controlled for
covariance between the independent variables.
There are six different methods of calculating the sum of squares. Type I, hierarchical or
sequential sums of squares, is appropriate when the groups in the MANOVA are of equal
sizes. Type I sum of squares provides a breakdown of the sums of squares for the whole
model used in the MANOVA but it is particularly sensitive to the order in which the
independent variables are placed in the model. If a variable is entered first, it is not 2 - Manova 4.3.05 33
adjusted for any of the other variables; if it is entered second, it is adjusted for one other
variable (the first one entered); if it is placed third, it will be adjusted for the two other
variables already entered.
Type II, the partially sequential sum of squares, has the advantage over Type I in that it is
not affected by the order in which the variables are entered. It displays the sum of squares
after controlling for the effect of other main effects and interactions but is only robust
where there are even numbers of participants in each group.
Type III sum of squares can be used in models where there are uneven group sizes,
although there needs to be at least one participant in each cell. It calculates the sum of
squares after the independent variables have all been adjusted for the inclusion of all other
independent variables in the model.
Type IV sum of squares can be used when there are empty cells in the model but it is
generally thought more suitable to use Type III sum of squares under these conditions
since Type IV is not thought to be good at testing lower order effects.
Type V has been developed for use where there are cells with missing data. It has been
designed to examine the effects according to the degrees of freedom which are available
and if the degrees of freedom fall below a given level these effects are not taken into
account. The cells which remain in the model have at least the degrees of freedom the full
model would have without any cells being excluded. For those cells which remain in the
model the Type III sum of squares are calculated. However, the Type V sum of squares 2 - Manova 4.3.05 34
are sensitive to the order in which the independent variables are placed in the model and
the order in which they are entered will determine which cells are excluded.
Type VI sum of squares is used for testing hypotheses where the independent variables
are coded using negative and positive signs e.g. +1 = male, -1 = female.
Type III sum of squares is the most frequently used as it has the advantages of Types IV,
V and VI without the corresponding restrictions.
Mean Squares: The mean square is the sum of squares divided by the appropriate degrees
of freedom.
Multivariate Measures: In most of the statistical programs used to calculate MANOVAs
there are four multivariate measures: Wilks’ lambda, Pillai's trace, Hotelling-Lawley trace
and Roy’s largest root. The difference between the four measures is the way in which
they combine the dependent variables in order examine the amount of variance in the
data. Wilks’ lambda demonstrates the amount of variance accounted for in the dependent
variable by the independent variable; the smaller the value, the larger the difference
between the groups being analyzed. 1 minus Wilks’ lambda indicates the amount of
variance in the dependent variables accounted for by the independent variables. Pillai's
trace is considered the most reliable of the multivariate measures and offers the greatest
protection against Type I errors with small sample sizes. Pillai's trace is the sum of the
variance which can be explained by the calculation of discriminant variables. It calculates
the amount of variance in the dependent variable which is accounted for by the greatest
separation of the independent variables. The Hotelling-Lawley trace is generally 2 - Manova 4.3.05 35
converted to the Hotelling’s T-square. Hotelling’s T is used when the independent
variable forms two groups and represents the most significant linear combination of the
dependent variables. Roy’s largest root, also known as Roy’s largest eigenvalue, is
calculated in a similar fashion to Pillai's trace except it only considers the largest
eigenvalue (i.e. the largest loading onto a vector). As the sample sizes increase the values
produced by Pillai’s trace, Hotelling-Lawley trace and Roy’s largest root become similar.
As you may be able to tell from these very broad explanations, the Wilks’ lambda is the
easiest to understand and therefore the most frequently used measure.
Multivariate F value: This is similar to the univariate F value in that it is representative of
the degree of difference in the dependent variable created by the independent variable.
However, as well as being based on the sum of squares (as in ANOVA) the calculation
for F used in MANOVAs also takes into account the covariance of the variables.
Example of Output of SPSS
The data analysed in this example came from a large-scale study with three dependent
variables which were thought to measure distinct aspects of behaviour. The first was total
score on the Dissociative Experiences Scale, the second was reaction time when
completing a signal detection task and the third was a measure of people’s hallucinatory
experiences (the Launay Slade Hallucinations Scale, LSHS). The independent variable
was the degree to which participants were considered prone to psychotic experiences
(labeled PRONE in the output shown in Fig 2.1), divided into High, Mean and Low. 2 - Manova 4.3.05 36
The first part of the output is shown in Fig 2.1. The Between-Subjects Factors table
displays the independent variable levels. Here there is only one independent variable with
three levels. The number of participants in each of the independent variable groups are
displayed in the column on the far right.
The Multivariate Tests table displays the multivariate values: Pillai’s Trace, Wilks’
Lambda, Hotelling’s Trace and Roy’s Largest Root. These are the multivariate values for
the model as a whole. The F values for the Intercept, shown in the first part of the table
are all the same. Those for the independent variable labeled PRONE are all different, but
they are all significant above the 1% level (shown by Sig being .000), indicating that on
the dependent variables there is a significant difference between the three proneness
groups.
The Tests of Between-Subjects Effects table gives the sum of squares, degrees of
freedom, Mean Square value, the F values and the significance levels for each dependent
variable. (The Corrected Model is the variance in the dependent variables which the
independent variables accounts for without the intercept being taken into consideration.)
The section of the table which is of interest is where the source under consideration is the
independent variable, the row for PRONE. In this row it can be seen that two (DES,
Launay Slade Hallucinations Scale) out of the three dependent variables included in the
model are significant (p<.05), meaning that three proneness groups differ significantly in
their scores on the DES and the LSHS.
******** Insert Fig 2.1*********** 2 - Manova 4.3.05 37
Figure 2.1. Example of main output from SPSS for MANOVA.
To determine which of the three independent variable groups differ from one another on
the dependent variables, Least Squares difference comparisons were performed by
selecting the relevant option in SPSS. The output is shown in Fig 2.2. The table is split
into broad rows, one for each dependent variable, and within each row the three groups of
high, mean or low Proneness are compared one against the other. The mean difference
between the two groups under consideration are given in one column and then separate
columns show the Standard Error, Significance and the lower and upper 95% Confidence
intervals. For the DES dependent variable, the High group is significantly different from
the Low group (p=0.004) but not from the Mean group (p=0.076). Similarly, for the Total
mean reaction time dependent variable, the High group differs significantly from the Low
group but not from the Mean group. For both these dependent variables, the Mean group
do not differ from the High group nor from the Low group. For the Launay scale, the
High group differs significantly from the Mean and Low groups, both of which also differ
from each other.
******** Insert Fig 2.2 ***********
Figure 2.2. The Least Squares Difference output from SPSS MANOVA analysis. 2 - Manova 4.3.05 38
The means and standard deviations on the DES and Launay scales for the three proneness
groups are displayed in Fig 2.3, which was produced by SPSS. Tables such as this assist
in interpreting the results.
The MANOVA above could also have included another independent variable, such as
gender. The interaction between the two independent variables on the dependent variables
would have been reported in the Multivariate Statistics and Tests of Between Subjects
Effects tables.
******* Insert Fig 2.3 *********
Figure 2.3. Means for the three psychosis proneness groups on the Dissociative
Experiences Scale and the Launay Slade Hallucinations Scale.
Example of the use of MANOVA
From health
Snow and Bruce (2003) explored the factors involved in Australian teenage girls
smoking, collecting data from 241 participants aged between 13 and 16 years of age.
Respondents completed a number of questionnaires including an Adolescent Coping
Scale. They were also asked to indicate how frequently they smoked cigarettes and the
responses were used to divide the respondents into three smoking groups (current
smokers, experimental smokers, never smokers). In their analysis, Snow and Bruce used 2 - Manova 4.3.05 39
MANOVA with smoking group as the independent variable. In one of the MANOVAs,
the dependent variables were the measures on three different coping strategies. Snow and
Bruce were only interested in the main effects of smoking group on the dependent
variables so they converted the Wilk’s lambda to F values and significance levels. They
used Scheffe post hoc analysis to determine which of the three smoking groups differed
significantly on the dependent variables, and found that on the 'productive' coping
strategy there was a significant difference between the current and experimental smokers,
on the 'non-productive' coping strategy there was a difference between the current
smokers and those who had never smoked, and on the 'rely on others' coping strategy
there was a difference between current and experimental smokers.
FAQs
How do I maximise the power in a MANOVA?
Power refers to the sensitivity of your study design to detect true significant findings
when using statistical analysis. Power is determined by the significance level chosen, the
effect size and the sample size. (There are many free programmes available on the
internet which can be used to calculate effect sizes from previous studies by placing
means and sample sizes into the equations.) A small effect size will need a large sample
size for significant differences to be detected, while a large effect size will need a
relatively small sample to be detected.
In general, the power of your analysis will increase the larger the effect size and sample
size. Taking a practical approach, obtaining as large a sample as possible will maximise
the power of your study. 2 - Manova 4.3.05 40
What do I do if I have groups with uneven sample sizes?
Having unequal sample sizes can affect the integrity of the analysis. One possibility is to
recruit more participants into the groups which are under represented in the data set,
another is to randomly delete cases from the more numerous groups until they are equal
to the least numerous one.
What do I do if I think there may be a covariate with the dependent variables included in
a MANOVA model?
When a covariate is incorporated into a MANOVA it is usually referred to as a
MANCOVA model. The ‘best’ covariate for inclusion in a model should be highly
correlated with the dependent variables but not related to the independent variables. The
dependent variables included in a MANCOVA are adjusted for their association with the
covariate. Some experimenters include baseline data in as a covariate to control for any
individual differences in scores since even randomisation to different experimental
conditions does not completely control for individual differences.
Summary
MANOVA is used when there are multiple dependent variables as well as independent
variables in the study. MANOVA combines the multiple dependent variables in a linear
manner to produce a combination which best separates the independent variable groups.
An ANOVA is then performed on the newly developed dependent variable. 2 - Manova 4.3.05 41
Glossary
Additive: the effect of the independent variables on one another when placed in multi- or
univariate analysis of variance.
Interaction: the combined effect of two or more independent variables on the dependent
variables.
Main Effect: the effect of one independent variable on the dependent variables, examined
in isolation from all other independent variables.
Mean Squares: sum of squares expressed as a ratio of the degrees of freedom either
within or between the different groups.
Sum of Squares: the squared deviations from the mean within or between the groups. In
MANOVA they are controlled for covariance.
Wilks' lambda: a statistic which can vary between 0 and 1 and which indicates whether
the means of groups differ. A value of 1 indicates the groups have the same mean.
References
Kaufman, A.S. and McLean, J.E. (1998). An investigation into the relationship between
interests and intelligence. Journal of Clinical Psychology, 54, 279-295. 2 - Manova 4.3.05 42
Snow, P.C. and Bruce, D.D. (2003). Cigarette smoking in teenage girls: exploring the role
of peer reputations, self-concept and coping. Health Education Research Theory and
Practice, 18, 439-452.
Further Reading
Anderson, T.W. (2003). An Introduction to Multivariate Statistical Analysis. New York:
Wiley.
Rees, D.G. (2000). Essential Statistics. (4th ed.) London: Chapman and Hall/CRC.
Internet Sources
www.statsoftinc.com/textbook/
www2.chass.ncsu.edu/garson/pa765/manova.htm
Multivariate Analysis and MANOVA
The MANOVA (multivariate analysis of variance) is a type of multivariate analysis used to analyze data that involves more than one dependent variable at a time. MANOVA allows us to test hypotheses regarding the effect of one or more independent variables on two or more dependent variables.
A MANOVA analysis generates a p-value that is used to determine whether or not the null hypothesis can be rejected. See Statistical Data Analysis for more information.
MANOVA Example
Suppose we have a hypothesis that a new teaching style is better than the standard method for teaching math. We may want to look at the effect of teaching style (independent variable) on the average values of
several dependent variables such as student satisfaction, number of student absences and math scores. A MANOVA procedure allows us to test our hypothesis for all three dependent variables at once.
More About MANOVA
Like the example above, a MANOVA is often used to detect differences in the average values of the dependent variables between the different levels of the independent variable. Interestingly, in addition to detecting differences in the average values, a MANOVA test can also detect differences in correlations among the dependent variables between the different levels of the independent variable.
MANOVA is simply one of many multivariate analyses that can be performed using SPSS. The SPSS MANOVA procedure is a standard, well accepted means of performing this analysis.
Multiple Linear Regression is another type of multivariate analysis, which is described in its own tutorial topic.
Get the Statistics Help you need
When you hire me to do the data analysis for your dissertation Results Chapter, I will determine whichstatistical methods are appropriate for your hypotheses. I also provide ongoing statistical help as needed to ensure that you fully understand all of the statistics that I used for your dissertation.
Simply contact me by phone or email to get started.
Steve Creech
1-800-357-0321 or 1-630-705-9482 | [email protected]
Statistical Consultant Overview