manova multivariate analysis of variance. one way analysis of variance (anova) comparing k...

90
MANOVA Multivariate Analysis of Variance

Upload: doreen-conley

Post on 02-Jan-2016

285 views

Category:

Documents


11 download

TRANSCRIPT

Page 1: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

MANOVA

Multivariate Analysis of Variance

Page 2: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

One way Analysis of Variance (ANOVA)

Comparing k Populations

Page 3: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The F test – for comparing k means

Situation

• We have k normal populations

• Let i and denote the mean and standard deviation of population i.

• i = 1, 2, 3, … k.

• Note: we assume that the standard deviation for each population is the same.

1 = 2 = … = k =

Page 4: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

We want to test

kH 3210 :

against

jiH jiA ,pair oneleast at for :

Page 5: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The F statistic

k

i

n

jiijkN

k

iiik

j

xx

xxnF

1 1

21

1

2

11

where xij = the jth observation in the i th sample.

injki ,,2,1 and ,,2,1

kiin

x

x th

i

n

jij

i

i

,,2,1 sample for mean 1

size sample Total 1

k

iinN

mean Overall 1 1

N

x

x

k

i

n

jij

i

Page 6: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The ANOVA table

k

iiiB xxnSS

1

2

W

B

MS

MSF

k

iiikB xxnMS

1

2

11

k

i

n

jiijW

j

xxSS1 1

2

k

i

n

jiijkNW

j

xxMS1 1

21

1k

kN

Source S.S d.f, M.S. F

Between

Within

The ANOVA table is a tool for displaying the computations for the F test. It is very important when the Between Sample variability is due to two or more factors

Page 7: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Computing Formulae:

k

i

n

jij

i

x1 1

2

Compute

ixTin

jiji samplefor Total

1

Total Grand 1 11

k

i

n

jij

k

ii

i

xTG

size sample Total1

k

iinN

k

i i

i

n

T

1

2

1)

2)

3)

4)

5)

Page 8: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The data

• Assume we have collected data from each of k populations

• Let xi1, xi2 , xi3 , … denote the ni observations from population i.

• i = 1, 2, 3, … k.

Page 9: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Then

1)

2)

k

i i

ik

i

n

jijWithin n

TxSS

i

1

2

1 1

2

BetweenSS

k

i i

i

N

G

n

T

1

22

3)

kNSS

kSSF

Within

Between

1

Page 10: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Source d.f. Sum of Squares

Mean Square

F-ratio

Between k - 1 SSBetween MSBetween MSB /MSW

Within N - k SSWithin MSWithin

Total N - 1 SSTotal

Anova Table

SSMS

df

Page 11: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Example

In the following example we are comparing weight gains resulting from the following six diets

1. Diet 1 - High Protein , Beef

2. Diet 2 - High Protein , Cereal

3. Diet 3 - High Protein , Pork

4. Diet 4 - Low protein , Beef

5. Diet 5 - Low protein , Cereal

6. Diet 6 - Low protein , Pork

Page 12: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Gains in weight (grams) for rats under six diets differing in level of protein (High or Low) and source of protein (Beef, Cereal, or Pork)

Diet 1 2 3 4 5 6

73 98 94 90 107 49 102 74 79 76 95 82 118 56 96 90 97 73 104 111 98 64 80 86 81 95 102 86 98 81 107 88 102 51 74 97 100 82 108 72 74 106 87 77 91 90 67 70 117 86 120 95 89 61 111 92 105 78 58 82

Mean 100.0 85.9 99.5 79.2 83.9 78.7 Std. Dev. 15.14 15.02 10.92 13.89 15.71 16.55

x 1000 859 995 792 839 787 x2 102062 75819 100075 64462 72613 64401

Page 13: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Thus

115864678464794321

2

1 1

2

k

i i

ik

i

n

jijWithin n

TxSS

i

BetweenSS 933.461260

5272467846

2

1

22

k

i i

i

N

G

n

T

3.4

56.214

6.922

54/11586

5/933.46121

kNSS

kSSF

Within

Between

54 and 5 with 386.2 2105.0 F

Thus since F > 2.386 we reject H0

Page 14: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Source d.f. Sum of Squares

Mean Square

F-ratio

Between 5 4612.933 922.587 4.3**

(p = 0.0023)

Within 54 11586.000 214.556

Total 59 16198.933

Anova Table

* - Significant at 0.05 (not 0.01)

SSSSSS

** - Significant at 0.01

Page 15: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Equivalence of the F-test and the t-test when k = 2

mns

yxt

Pooled

11

2

11 22

mn

smsns yx

Pooled

the t-test

Page 16: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

the F-test

knsn

kxxn

s

sF

k

ii

k

iii

k

iii

Pooled

Between

11

2

1

2

2

2

1

1

211 21

211

211

212

211

nnsnsn

xxnxxn

212

211numerator xxnxxn

2r denominato pooleds

Page 17: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

2

21

221122

222

nn

xnxnxnxxn

2

21

221111

211

nn

xnxnxnxxn

2

21221

221 xxnn

nn

2

21221

221 xx

nn

nn

Page 18: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

2212

21

212

2212

222

11 xxnn

nnnnxxnxxn

221

21

21 xxnn

nn

221

21

11

1xx

nn

22

221

21

11

1t

s

xx

nn

FPooled

Hence

Page 19: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Factorial Experiments

Analysis of Variance

Page 20: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

• Dependent variable Y

• k Categorical independent variables A, B, C, … (the Factors)

• Let– a = the number of categories of A– b = the number of categories of B– c = the number of categories of C– etc.

Page 21: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The Completely Randomized Design

• We form the set of all treatment combinations – the set of all combinations of the k factors

• Total number of treatment combinations– t = abc….

• In the completely randomized design n experimental units (test animals , test plots, etc. are randomly assigned to each treatment combination.– Total number of experimental units N = nt=nabc..

Page 22: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The treatment combinations can thought to be arranged in a k-dimensional rectangular block

A

1

2

a

B1 2 b

Page 23: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

A

B

C

Page 24: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

• The Completely Randomized Design is called balanced

• If the number of observations per treatment combination is unequal the design is called unbalanced. (resulting mathematically more complex analysis and computations)

• If for some of the treatment combinations there are no observations the design is called incomplete. (In this case it may happen that some of the parameters - main effects and interactions - cannot be estimated.)

Page 25: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Example

In this example we are examining the effect of

We have n = 10 test animals randomly assigned to k = 6 diets

The level of protein A (High or Low) and the source of protein B (Beef, Cereal, or Pork) on weight gains (grams) in rats.

Page 26: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The k = 6 diets are the 6 = 3×2 Level-Source combinations

1. High - Beef

2. High - Cereal

3. High - Pork

4. Low - Beef

5. Low - Cereal

6. Low - Pork

Page 27: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

TableGains in weight (grams) for rats under six diets differing in level of protein (High or Low) and s

ource of protein (Beef, Cereal, or Pork)

Levelof Protein High Protein Low protein

Sourceof Protein Beef Cereal Pork Beef Cereal Pork

Diet 1 2 3 4 5 6

73 98 94 90 107 49102 74 79 76 95 82118 56 96 90 97 73104 111 98 64 80 86

81 95 102 86 98 81107 88 102 51 74 97100 82 108 72 74 106

87 77 91 90 67 70117 86 120 95 89 61111 92 105 78 58 82

Mean 100.0 85.9 99.5 79.2 83.9 78.7Std. Dev. 15.14 15.02 10.92 13.89 15.71 16.55

Page 28: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Source of Protein

Level of Protein

Beef Cereal Pork

High

Low

Treatment combinations

Diet 1 Diet 2 Diet 3

Diet 4 Diet 5 Diet 6

Page 29: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Level of Protein Beef Cereal Pork Overall

Low 79.20 83.90 78.70 80.60

Source of Protein

High 100.00 85.90 99.50 95.13

Overall 89.60 84.90 89.10 87.87

Summary Table of Means

Page 30: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Profiles of the response relative to a factor

A graphical representation of the effect of a factor on a reponse variable (dependent variable)

Page 31: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Profile Y for AY

Levels of A

a1 2 3 …

This could be for an individual case or averaged over a group of cases

This could be for specific level of another factor or averaged levels of another factor

Page 32: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

70

80

90

100

110

Beef Cereal Pork

Wei

ght

Gai

n

High Protein

Low Protein

Overall

Profiles of Weight Gain for Source and Level of Protein

Page 33: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

70

80

90

100

110

High Protein Low Protein

Wei

ght

Gai

nBeef

Cereal

Pork

Overall

Profiles of Weight Gain for Source and Level of Protein

Page 34: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Example – Four factor experiment

Four factors are studied for their effect on Y (luster of paint film). The four factors are:

Two observations of film luster (Y) are taken for each treatment combination

1) Film Thickness - (1 or 2 mils)

2) Drying conditions (Regular or Special) 3) Length of wash (10,30,40 or 60 Minutes), and

4) Temperature of wash (92 ˚C or 100 ˚C)

Page 35: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The data is tabulated below:Regular Dry Special Dry

Minutes 92 C 100 C 92C 100 C1-mil Thickness

20 3.4 3.4 19.6 14.5 2.1 3.8 17.2 13.430 4.1 4.1 17.5 17.0 4.0 4.6 13.5 14.340 4.9 4.2 17.6 15.2 5.1 3.3 16.0 17.860 5.0 4.9 20.9 17.1 8.3 4.3 17.5 13.9

2-mil Thickness20 5.5 3.7 26.6 29.5 4.5 4.5 25.6 22.530 5.7 6.1 31.6 30.2 5.9 5.9 29.2 29.840 5.5 5.6 30.5 30.2 5.5 5.8 32.6 27.460 7.2 6.0 31.4 29.6 8.0 9.9 33.5 29.5

Page 36: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Definition:

A factor is said to not affect the response if the profile of the factor is horizontal for all combinations of levels of the other factors:

No change in the response when you change the levels of the factor (true for all combinations of levels of the other factors)

Otherwise the factor is said to affect the response:

Page 37: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Profile Y for A – A affects the response

Y

Levels of A

a1 2 3 …

Levels of B

Page 38: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Profile Y for A – no affect on the response

Y

Levels of A

a1 2 3 …

Levels of B

Page 39: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Definition:• Two (or more) factors are said to interact if

changes in the response when you change the level of one factor depend on the level(s) of the other factor(s).

• Profiles of the factor for different levels of the other factor(s) are not parallel

• Otherwise the factors are said to be additive .

• Profiles of the factor for different levels of the other factor(s) are parallel.

Page 40: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Interacting factors A and BY

Levels of A

a1 2 3 …

Levels of B

Page 41: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Additive factors A and BY

Levels of A

a1 2 3 …

Levels of B

Page 42: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

• If two (or more) factors interact each factor effects the response.

• If two (or more) factors are additive it still remains to be determined if the factors affect the response

• In factorial experiments we are interested in determining

– which factors effect the response and

– which groups of factors interact .

Page 43: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The testing in factorial experiments 1. Test first the higher order interactions.2. If an interaction is present there is no need

to test lower order interactions or main effects involving those factors. All factors in the interaction affect the response and they interact

3. The testing continues with for lower order interactions and main effects for factors which have not yet been determined to affect the response.

Page 44: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Models for factorial Experiments

Page 45: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The Single Factor Experiment

Situation

• We have t = a treatment combinations

• Let i and denote the mean and standard deviation of observations from treatment i.

• i = 1, 2, 3, … a.

• Note: we assume that the standard deviation for each population is the same.

1 = 2 = … = a =

Page 46: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The data

• Assume we have collected data for each of the a treatments

• Let yi1, yi2 , yi3 , … , yin denote the n

observations for treatment i.

• i = 1, 2, 3, … a.

Page 47: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The model

Note:

ij i ij i i ijy y

i ij i ij

where ij ij iy

1

1 k

iik

i i

has N(0,2) distribution

(overall mean effect)

(Effect of Factor A)

Note:1

0a

ii

by their definition.

Page 48: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Model 1:

ij i ijy

yij (i = 1, … , a; j = 1, …, n) are independent Normal with mean i and variance 2.

Model 2:

where ij (i = 1, … , a; j = 1, …, n) are independent Normal with mean 0 and variance 2.

ij i ijy Model 3:

where ij (i = 1, … , a; j = 1, …, n) are independent Normal with mean 0 and variance 2 and

1

0a

ii

Page 49: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The Two Factor Experiment

Situation

• We have t = ab treatment combinations

• Let ij and denote the mean and standard deviation of observations from the treatment combination when A = i and B = j.

• i = 1, 2, 3, … a, j = 1, 2, 3, … b.

Page 50: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The data

• Assume we have collected data (n observations) for each of the t = ab treatment combinations.

• Let yij1, yij2 , yij3 , … , yijn denote the n observations for treatment combination - A = i, B = j.

• i = 1, 2, 3, … a, j = 1, 2, 3, … b.

Page 51: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The modelNote:

ijk ij ijk ij ij ijky y

i j ij i j ij

where ijk ijk ijy

1 1 1 1

1 1 1, and

a b b a

ij i ij j iji j j iab b a

, ,i i j j

has N(0,2) distribution

and

i j ijkij

ij i jij

Page 52: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The modelNote:

ijk ij ijk ij ij ijky y

i j ij i j ij

where ijk ijk ijy

1 1 1 1

1 1 1, and

a b b a

ij i ij j iji j j iab b a

, ,i i j j

has N(0,2) distribution

Note:1

0a

ii

by their definition.

i j ijkij

Page 53: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

ijk i j ijkijy

Model :

where ijk (i = 1, … , a; j = 1, …, b ; k = 1, …, n) are independent Normal with mean 0 and variance 2 and

1

0a

ii

1

0b

jj

1 1

and 0a b

ij iji j

Main effectsInteraction Effect

Mean Error

Page 54: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

ijk i j ijkijy

Maximum Likelihood Estimates

where ijk (i = 1, … , a; j = 1, …, b ; k = 1, …, n) are independent Normal with mean 0 and variance 2 and

1 1 1

ˆa b n

ijki j k

y y abn

1 1

ˆb n

i i ijkj k

y y y bn y

1 1

ˆa n

j j ijki k

y y y an y

Page 55: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

^

ij i jijy y y y

1

n

ijk i jk

y n y y y

22

1 1 1

a b n

ijk iji j k

y ynab

2

1 1 1

^1 ˆˆˆa b n

ijk i j iji j k

ynab

This is not an unbiased estimator of 2 (usually the case when estimating variance.)

The unbiased estimator results when we divide by ab(n -1) instead of abn

Page 56: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

22

1 1 1

1

1

a b n

ijk iji j k

s y yab n

2

1 1 1

^1 ˆˆˆ1

a b n

ijk i j iji j k

yab n

The unbiased estimator of 2 is

1

1 Error ErrorSS MSab n

2

1 1 1

a b n

Error ijk iji j k

SS y y

where

Page 57: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

22

1 1 1 1

^a b a b

AB ij i jiji j i j

SS y y y y

Testing for Interaction:

1

1 1 AB

AB

Error Error

SSa bMS

FMS MS

where

We want to test:

H0: ()ij = 0 for all i and j, against

HA: ()ij ≠ 0 for at least one i and j.

The test statistic

Page 58: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

( 1)( 1), ( 1)AB

Error

MSF F a b ab n

MS

We reject

H0: ()ij = 0 for all i and j,

If

Page 59: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

22

1 1

ˆa a

A i ii i

SS y y

Testing for the Main Effect of A:

1

1 A

A

Error Error

SSaMS

FMS MS

where

We want to test:

H0: i = 0 for all i, against

HA: i ≠ 0 for at least one i.

The test statistic

Page 60: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

( 1), ( 1)A

Error

MSF F a ab n

MS

We reject

H0: i = 0 for all i,

If

Page 61: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

22

1 1

ˆb b

B j jj j

SS y y

Testing for the Main Effect of B:

1

1 B

B

Error Error

SSbMS

FMS MS

where

We want to test:

H0: j = 0 for all j, against

HA: j ≠ 0 for at least one j.

The test statistic

Page 62: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

( 1), ( 1)B

Error

MSF F b ab n

MS

We reject

H0: j = 0 for all j,

If

Page 63: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The ANOVA Table

Source S.S. d.f. MS =SS/df F

A SSA a - 1 MSA MSA / MSError

B SSB b - 1 MSB MSB / MSError

AB SSAB (a - 1)(b - 1) MSAB MSAB/ MSError

Error SSError ab(n - 1) MSError

Total SSTotal abn - 1

Page 64: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Computing Formulae

1 1 1

Let a b n

ijki j k

T y

1 1 1 1 1

, , b n a n n

i ijk j ijk ij ijkj k i k k

T y T y T y

2

2 •••

1 1 1

Then a b n

Total ijki j k

TSS y

nab

22 2 2

• ••• ••• •••

1 1

, a a

jiA B

i i

TT T TSS SS

nb nab na nab

2 22 2• • ••• •••

1 1 1

,a a a

ij jiAB

i i i

T TT TSS

n nb na nab

Page 65: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

and Error Total A B ABSS SS SS SS SS

Page 66: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

MANOVA

Multivariate Analysis of Variance

Page 67: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

One way Multivariate Analysis of Variance (MANOVA)

Comparing k p-variate Normal Populations

Page 68: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The F test – for comparing k means

Situation

• We have k normal populations

• Let denote the mean vector and covariance matrix of population i.

• i = 1, 2, 3, … k.

• Note: we assume that the covariance matrix for each population is the same.

and i

1 2 k

Page 69: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

We want to test

0 1 2 3: kH

against

: for at least one pair ,A i jH i j

Page 70: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

The data

• Assume we have collected data from each of k populations

• Let denote the n observations from population i.

• i = 1, 2, 3, … k.

1 2, , ,i i inx x x

Page 71: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Computing Formulae:

Compute

1

Total vector for sample n

i ijj

T x i

1

1 1 1

Grand Total vector ink k

i iji i j

p

G

G T x

G

1)

2)

11 1

1

n

ijj i

npi

pijj

xT

Tx

Total sample size N kn 3)

Page 72: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

21 1

1 1 1 1

1 12

11 1 1 1

k n k n

ij ij piji j i j

k n

ij iji j k n k n

ij pij piji j i j

x x x

x x

x x x

21 1

1 1

12

11 1

1 1

1

1 1

k k

i i pii ik

i ii k k

i pi pii i

T T Tn n

TTn

T T Tn n

4)

5)

Page 73: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Let

1

1 1k

i ii

H TT GGn N

212 1

1 11 1

21 2 1

1 11 1

1 1

1 1

k kp

i i pii i

k kp

i pi ii i

G GGT T T

n N n N

G G GT T T

n N n N

2

1 1 1 11 1

2

1 11 1

k k

i i pi pi i

k k

i pi p pi pi i

n x x n x x x x

n x x x x n x x

= the Between SS and SP matrix

Page 74: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Let1 1 1

1k n k

ij ij i ii j i

E x x TTn

2 21 1 1 1

1 1 1 1 1 1

2 21 1

1 1 1 1 1 1

1 1

1 1

k n k k n k

ij i ij pij i pii j i i j i

k n k k n k

ij pij i pi pij pii j i i j i

x T x x T Tn n

x x T T x Tn n

2

1 1 1 11 1 1 1

2

1 11 1 1 1

k n k n

ij i ij i pij pii j i j

k n k n

ij i pij pi pij pii j i j

x x x x x x

x x x x x x

= the Within SS and SP matrix

Page 75: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Source SS and SP matrix

Between

Within

The Manova Table

11 1

1

p

p pp

h h

H

h h

11 1

1

p

p pp

e e

E

e e

Page 76: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

There are several test statistics for testing

0 1 2 3: kH

against

: for at least one pair ,A i jH i j

Page 77: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

1. Roy’s largest root1

1 largest eigenvalue of HE

This test statistic is derived using Roy’s union intersection principle

2. Wilk’s lambda ()

1

1

E

H E HE I

This test statistic is derived using the generalized Likelihood ratio principle

Page 78: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

3. Lawley-Hotelling trace statistic

2 1 10 sum of the eigenvalues of T tr HE HE

4. Pillai trace statistic (V)

1V tr

H H E

Page 79: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Example

In the following study, n = 15 first year university students from three different School regions (A, B and C) who were each taking the following four courses (Math, biology, English and Sociology) were observed: The marks on these courses is tabulated on the following slide:

Page 80: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Student Math Biology English Sociology Student Math Biology English Sociology Student Math Biology English Sociology1 62 65 67 76 1 65 55 35 43 1 47 47 98 782 54 61 75 70 2 87 81 59 64 2 57 69 68 453 53 53 53 59 3 75 67 56 68 3 65 71 77 624 48 56 73 81 4 74 70 55 66 4 41 64 68 585 60 55 49 60 5 83 71 40 52 5 56 54 86 646 55 52 34 41 6 59 48 48 57 6 63 73 88 767 76 71 35 40 7 61 47 46 54 7 43 62 84 788 58 52 58 46 8 81 77 51 45 8 28 47 65 589 75 71 60 59 9 77 68 42 49 9 47 54 90 78

10 55 51 69 75 10 82 84 63 70 10 42 44 79 7311 72 74 64 59 11 68 64 35 44 11 50 53 89 8912 72 75 51 47 12 60 53 60 65 12 46 61 91 8213 76 69 69 57 13 94 88 51 63 13 74 78 99 8614 44 48 65 65 14 96 88 67 81 14 63 66 94 8615 89 71 59 67 15 84 75 46 67 15 69 82 78 73

Educational RegionA B C

The data

Page 81: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Summary Statistics

63.267 61.600 58.733 60.133

160.638 104.829 -32.638 -47.110104.829 92.543 -4.900 -22.229-32.638 -4.900 155.638 128.967-47.110 -22.229 128.967 159.552

Ax

A S

Bx

B S

Cx

C S

76.400 69.067 50.267 59.200

141.257 155.829 45.100 60.914155.829 185.924 61.767 71.05745.100 61.767 96.495 93.37160.914 71.057 93.371 123.600

52.733 61.667 83.600 72.400

156.067 116.976 53.814 35.257116.976 136.381 3.143 -0.42953.814 3.143 116.543 114.88635.257 -0.429 114.886 156.400

15 15 15

45 45 45A B Cx x x x

14 14 14

42 42 42Pooled A B C S S S S

64.133 64.111 64.200 63.911

152.654 125.878 22.092 16.354125.878 138.283 20.003 16.133

22.092 20.003 122.892 112.40816.354 16.133 112.408 146.517

Page 82: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Computations :

1

Total vector for sample n

i ijj

T x i

1

1 1 1

Grand Total vector ink k

i iji i j

p

G

G T x

G

1)

2)

Total sample size = 45N kn 3)

Math Biology English SociologyA 949 924 881 902B 1146 1036 754 888C 791 925 1254 1086

Grand Totals G 2886 2885 2889 2876

Totals

Page 83: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

21 1

1 1 1 1

1 12

11 1 1 1

k n k n

ij ij piji j i j

k n

ij iji j k n k n

ij pij piji j i j

x x x

x x

x x x

4)

195718 191674 180399 182865191674 191321 184516 184542180399 184516 199641 193125182865 184542 193125 191590

=

Page 84: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

21 1

1 1

12

11 1

1 1

1

1 1

k k

i i pii ik

i ii k k

i pi pii i

T T Tn n

TTn

T T Tn n

=

5)

189306.53 186387.13 179471.13 182178.13186387.13 185513.13 183675.87 183864.40179471.13 183675.87 194479.53 188403.87182178.13 183864.40 188403.87 185436.27

Page 85: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Now

1

1 1k

i ii

H TT GGn N

= the Between SS and SP matrix

4217.733333 1362.466667 -5810.066667 -2269.3333331362.466667 552.5777778 -1541.133333 -519.1555556

-5810.066667 -1541.133333 9005.733333 3764.666667-2269.333333 -519.1555556 3764.666667 1627.911111

=

Page 86: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Let1 1 1

1k n k

ij ij i ii j i

E x x TTn

2 21 1 1 1

1 1 1 1 1 1

2 21 1

1 1 1 1 1 1

1 1

1 1

k n k k n k

ij i ij pij i pii j i i j i

k n k k n k

ij pij i pi pij pii j i i j i

x T x x T Tn n

x x T T x Tn n

= the Within SS and SP matrix

6411.467 5286.867 927.867 686.8675286.867 5807.867 840.133 677.600927.867 840.133 5161.467 4721.133686.867 677.600 4721.133 6153.733

=

Page 87: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Using SPSS to perform MANOVA

Page 88: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Selecting the variables and the Factors

Page 89: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Multivariate Testsc

.984 586.890a 4.000 39.000 .000

.016 586.890a 4.000 39.000 .000

60.194 586.890a 4.000 39.000 .000

60.194 586.890a 4.000 39.000 .000

.883 7.913 8.000 80.000 .000

.161 14.571a 8.000 78.000 .000

4.947 23.501 8.000 76.000 .000

4.891 48.913b 4.000 40.000 .000

Pillai's Trace

Wilks' Lambda

Hotelling's Trace

Roy's Largest Root

Pillai's Trace

Wilks' Lambda

Hotelling's Trace

Roy's Largest Root

EffectIntercept

High_School

Value F Hypothesis df Error df Sig.

Exact statistica.

The statistic is an upper bound on F that yields a lower bound on the significance level.b.

Design: Intercept+High_Schoolc.

The output

Page 90: MANOVA Multivariate Analysis of Variance. One way Analysis of Variance (ANOVA) Comparing k Populations

Univariate TestsTests of Between-Subjects Effects

4217.733a 2 2108.867 13.815 .000

552.578b 2 276.289 1.998 .148

9005.733c 2 4502.867 36.641 .000

1627.911d 2 813.956 5.555 .007

185088.800 1 185088.800 1212.473 .000

184960.556 1 184960.556 1337.555 .000

185473.800 1 185473.800 1509.241 .000

183808.356 1 183808.356 1254.515 .000

4217.733 2 2108.867 13.815 .000

552.578 2 276.289 1.998 .148

9005.733 2 4502.867 36.641 .000

1627.911 2 813.956 5.555 .007

6411.467 42 152.654

5807.867 42 138.283

5161.467 42 122.892

6153.733 42 146.517

195718.000 45

191321.000 45

199641.000 45

191590.000 45

10629.200 44

6360.444 44

14167.200 44

7781.644 44

Dependent VariableMath

Biology

English

Sociology

Math

Biology

English

Sociology

Math

Biology

English

Sociology

Math

Biology

English

Sociology

Math

Biology

English

Sociology

Math

Biology

English

Sociology

SourceCorrected Model

Intercept

High_School

Error

Total

Corrected Total

Type III Sumof Squares df Mean Square F Sig.

R Squared = .397 (Adjusted R Squared = .368)a.

R Squared = .087 (Adjusted R Squared = .043)b.

R Squared = .636 (Adjusted R Squared = .618)c.

R Squared = .209 (Adjusted R Squared = .172)d.