statistical analysis

Statistical Analysis

Professor Lynne Stokes

Department of Statistical Science

Lecture 18

Random Effects

Fixed vs. Random Factors

Fixed FactorsLevels are preselected, inferences limited

to these specific levels

Factors ShaftSleeveLubricant

ManufacturerSpeed

Levels Steel, AluminumPorous, NonporousLub 1, Lub 2, Lub 3, Lub 4

A, BHigh, Low

Fixed Factors (Effects)

Fixed FactorsLevels are preselected, inferences limited

to these specific levels

Fixed Levels Changes in the meani

Changes in the meani

One-Factor Modelyij = + i + eij

Main Effectsi - i Parameters

Random Factors (Effects)

Random Factor Levels are a random sample from a large population of possible levels.

Inferences are desired on the population of levels.

Factors Lawnmower

Levels 1, 2, 3, 4, 5, 6

One-Factor Modelyij = + ai + eij

Random Levels

Random Factors (Effects)

One-Factor Modelyij = + ai + eij

Main Effects

a2

Random ai

Variability =Estimate VarianceComponents

a2 , 2

Skin Swelling Measurements

Factors Laboratory animals (Random) Location of the measurement: Back, Ear (Fixed) Repeat measurements (2 / location)

Source df SS MS F p-ValueAnimals 5 0.4436 0.0887 11.85 0.003Locations 1 1.0626 1.0626 141.88 0.001Error 17 0.1273 0.0075Total 23 1.6355

Automatic Cutoff Times

Factors Manufacturers: A, B (Fixed) Lawnmowers: 3 for each manufacturer (Random) Speeds: High, Low (Fixed)

Source df SS MS F p-ValueManufacturers 1 2,521 2,521 3.54 0.133Lawnmowers 4 2,853 713 5.39 0.006Speed 1 20,886 20,886 157.87 0.000M x S 1 11 11 0.08 0.780Error 16 2,117 132Total 23 23,388

MGH Table 13.6

}

Random Factor Effects

AssumptionFactor levels are a random sample from a large

population of possible levels

AssumptionFactor levels are a random sample from a large

population of possible levels

Subjects (people) in a medical study Laboratory animals Batches of raw materials Fields or farms in an agricultural study Blocks in a block design

Inferences are desired on the population of levels, NOT just on the levels

included in the design

Inferences are desired on the population of levels, NOT just on the levels

included in the design

Random Effects Model Assumptions(All Factors Random)

Levels of each factor are a random sample of all possible levels of the factor

Random factor effects and model error terms are distributed as mutually independent zero-mean normal variates; e.g., ei~NID(0,e

2) , ai~NID(0,a2), mutually

independent

Analysis of variance model contains randomvariables for each random factor and interaction

Analysis of variance model contains randomvariables for each random factor and interaction

Interactions of random factorsare assumed random

Interactions of random factorsare assumed random

Skin Color Measurements

Factors Participants -- representative of those from one ethnic group, in a well-defined geographic region of the U.S. Weeks -- No skin treatment, studying week-to-week variation (No Repeats -- must be able to assume no interaction)

Source df SS MS F p-ValueSubjects 6 1904.58 317.43 186.17 0.000Weeks 2 2.75 1.37 0.81 0.468Error 12 20.46 1.71Total 20 1927.78

MGH Table 10.3

Two-Factor Random Effects Model: Main Effects Only

Two-Factor Main Effects Modelyijk = + ai + bj + eijk i = 1, ..., a

j = 1, ..., b

a i ~ ( , )NID a0 2

bi ~ ( , )NID b0 2

e ijk ~ ( , )NID e0 2Mutually Independent

0

0 No Effect

Two-Factor Random Effects Model

Two-Factor Modelyijk = + ai + bj + (ab)ij + eijk i = 1, ..., a

j = 1, ..., bk = 1, ..., r

a i ~ ( , )NID a0 2

bi ~ ( , )NID b0 2

(ab) ij ~ ( , )NID ab0 2

e ijk ~ ( , )NID e0 2Mutually Independent

0

0 No Effect

Two-Factor Model Differences

ij = + i + j + ()ij

y e2 2

Mean

Variance

Fixed Effects

ij =

y a b ab e2 2 2 2 2

Random Effects

Variance ComponentsVariance Components

Change the MeanChange the Mean

Expected Mean Squares

Functions of model parameters Identify testable hypotheses

Components set to zero under H0

Identify appropriate F statistic ratios Under H0, two E(MS) are identical

Properties of Quadratic Forms in Normally Distributed Random Variables

A)A(tr)Ayy(E

),(N~y

2)(),(~ 2 AAyyEAyy


One Factor, Fixed Effects

yij = + i + eiji = 1, ... , a ; j = 1, ... , r

eij ~ NID(0,e2)

y NID ri i e~ ( , ) 1 2



Sum of Squares

1a

)JaI(1a

1Q , rQ}MS{E

)JaI(r

)1a(}/SS{E

y)JaI(yrSS

)y, ... ,y(=y

a

1i

2i

12eA

12e

2eA

1A

a1

y NID ri i e~ ( , ) 1 2



Sum of Squares

E{MSA)=e2 = = ... = a

y = (y ... , y )

SS

, Q

1 a

A

,

( )

{ / } ( ) ( )

{ } ( )

ry I a J y

E SS ar

I a J

E MS rQa

I a J

A ee

A e

1

22

1

2 1

1

1

1

y NID ri i e~ ( , ) 1 2


Three-Factor Fixed Effects Model

Source Mean Square Expected Mean Square

A MSA e2 + bcr Q

AB MSAB e2 + cr Q

ABC MSABC e + r Q

Error MSE e2

Q , etc.

1

12

1a ii

a

( )•All effects tested against error•All effects tested against error

Typical Main Typical Main Effects and Effects and InteractionsInteractions


One Factor, Random Effects

yij = + ai + eiji = 1, ... , a ; j = 1, ... , r

ai ~ NID(0,a2) , eij ~ NID(0,e

2)Independent

y NID ri a e~ ( , ) 2 1 2



Sum of Squares

y NID ri a e~ ( , ) 2 1 2

2a

2eA

2e

12aA

1A

a1

r}MS{E

r)1a()}r/(SS{E

y)JaI(yrSS

)y, ... ,y(=y



Sum of Squares

E{MSA)=e2

a2 = 0

y NID ri a e~ ( , ) 2 1 2

2a

2eA

2e

12aA

1A

a1

r}MS{E

r)1a()}r/(SS{E

y)JaI(yrSS

)y, ... ,y(=y

Skin Color Measurements

Factors Participants -- representative of those from one ethnic group, in a well-defined geographic region of the U.S. Weeks -- No skin treatment, studying week-to-week variation (No Repeats -- Must be Able to Assume No Interaction)

Source df SS MS E(MS)Subjects 6 1904.58 317.43Weeks 2 2.75 1.37Error 12 20.46 1.71Total 20 1927.78

2a

2 32w

2 72


Three-Factor Random Effects Model

Source Mean Square Expected Mean Square

A MSA e2 + rabc

+ crab

+ brac + bcra

AB MSAB e2 + rabc

+ crab

ABC MSABC e + rabc

Error MSE e2

•Effects not necessarily tested against error•Test main effects even if interactions are significant•May not be an exact test (three or more factors, random or mixed effects models; e.g. main effect for A)

•Effects not necessarily tested against error•Test main effects even if interactions are significant•May not be an exact test (three or more factors, random or mixed effects models; e.g. main effect for A)

Expected Mean SquaresBalanced Random Effects Models

Each E(MS) includes the error variance component Each E(MS) includes the variance component for the

corresponding main effect or interaction Each E(MS) includes all higher-order interaction

variance components that include the effect The multipliers on the variance components equal the

number of data values in factor-level combination defined by the subscript(s) of the effect

e.g., E(MSAB) = e2 + rabc

+crab

Expected Mean SquaresBalanced Experimental Designs

1. Specify the ANOVA Model

yijk = + i + j + ()ij + eijk

Two Factors, Fixed EffectsTwo Factors, Fixed Effects

MGH Appendix to Chapter 10MGH Appendix to Chapter 10


2. Label a Two-Way Table

a. One column for each model subscriptb. Row for each effect in the model

-- Ignore the constant term-- Express the error term as a nested effect

Two Factors, Fixed Effects


i j k

i

j

(ij

ek(ij)


3. Column Subscript Corresponds to a Fixed Effect.

a. If the column subscript appears in the row effect & no other subscripts in the row effect are nested withinthe column subscript

-- Enter 0 if the column effect is in a fixed row effect

b. If the column subscript appears in the row effect & one or more other subscripts in the row effect are nestedwithin the column subscript

-- Enter 1

c. If the column subscript does not appear in the row effect

-- Enter the number of levels of the factor


Step 3a

i j k

i 0

j 0

(ij 0 0

ek(ij)



Step 3b

i j k

i 0

j 0

(ij 0 0

ek(ij) 1 1



Step 3c

i j k

i 0 b

j a 0

(ij 0 0

ek(ij) 1 1



4. Column Subscript Corresponds to a Random Effect

a. If the column subscript appears in the row effect

-- Enter 1

b. If the column subscript does not appear in the row effect

-- Enter the number of levels of the factor


Step 4a

i j k

i 0 b

j a 0

(ij 0 0

ek(ij) 1 1 1



Step 4b

i j k

i 0 b r

j a 0 r

(ij 0 0 r

ek(ij) 1 1 1



5. Notation

a. = Qfactor(s) for fixed main effects and interactions

b. = factor(s)2 for random main effects and interactions

List each parameter in a column on the same line as its corresponding model term.


i j k

i 0 b r Q

j a 0 r Q

(ij 0 b r Q

ek(ij) 1 1 1 e

Step 5



6. MS = Mean Square, C = Set of All Subscripts for the Corresponding Model Term

a. Identify the parameters whose model terms contain all the subscripts in C (Note: can have more than those in C)

b. Multipliers for each :

-- Eliminate all columns having the subscripts in C

-- Eliminate all rows not in 6a.

-- Multiply remaining constants across rows for each c. E(MS) is the linear combination of the coefficients from

6b and the corresponding parameters; E(MSE) = e2.


E(MS)

e Q, Q Q

B eQ, Q Q

AB e Q Q

Error e

e

Step 6a



Step 6b: MSAB

i j k

i 0 b r Q

j a 0 r Q

(ij 0 b r Q

ek(ij) 1 1 1 e



Step 6c


E(MS)

eQQ

B eQQ

AB e rQ

Error e


Step 6b: MSB

i j k

i 0 b r Q

j a 0 r Q

(ij 0 b r Q

ek(ij) 1 1 1 e



Step 6c


E(MS)

eQQ

B earQ

AB e rQ

Error e


Source df SS E(MS)

A a-1 SSA

ebrQA

B b-1 SSB

e arQA

AB (a-1)(b-1) SSAB

erQAB

Error ab(r-1) SSE

e

Total abr-1 TSS

Under appropriate null hypotheses,Under appropriate null hypotheses,E(MS) for A, B, and AB same as E(MSE(MS) for A, B, and AB same as E(MSEE))

F = MS / MSF = MS / MSEE

Under appropriate null hypotheses,Under appropriate null hypotheses,E(MS) for A, B, and AB same as E(MSE(MS) for A, B, and AB same as E(MSEE))

F = MS / MSF = MS / MSEE

statistical analysis

Documents

squaresone factor

factorrandom factor

random sample

random effectsfixed

meanmionefactor modelyij

manufacturer random

ai eiji

ai bj eijk