To go further: intra- versus interindividual variability

2

Up to now: irrespective of models• Multilevel data

– Definition– Diagram

• Simple effects and contrasts• Correction for the number of tests• RM Anova versus mixed linear models

– Advantages– Disadvantages– Conclusion: when to use one or the other

• Interpretation of parameters, regression equation, and drawing an expected mean graph

• Conditions of applications• Mediation, confounding and moderation

3

Up to now: specific for models

• Specific to RM ANOVA– Equality of covariance and sphericity (and

corrections when sphericity not met)• Specific to multilevel models– Fixed versus random effects– Selection of random effects in mixed linear models

4

Definition: multilevel data

• Data can be non-independent for various reasons:– Clustered in groups (children in classes)– Several measures from same individual – Conditions while taking the measures (e.g., half of

them taken in the morning, half in the afternoon)

5

Graphically: 2 levels

6

When is an effect not estimable

• When there is only one observation for a combination of cells…

• Why?• Because the effect is indistinguishable from

residual/individual variability

7

Contrasts

• Allow to test specific patterns in group effects• But…• They can be significant when the data do not

fit your hypothesized pattern– Some data get ignored

• They encourage researchers to “fish” (and you will finally get something)– Be careful what you wish for because you may get it

8

Advantages of RM ANOVA

• Global test over all levels of categorical variables

• Unbiased (especially important for small sample sizes)

• Well-accepted in psychology

9

Disadvantages of RM ANOVA

• Number of observations by cell have to be equal (even one missing data means all observations for this subject are useless)

• Number of cells have to be equal (for subjects)• Compound symmetry or sphericity

assumption (is it substantially credible?)

10

When to use multilevel models

• When you have missing data• When by design you expect a different

number of observations per level 2 unit (e.g., follow-up of individuals)

• When you want to include a continuous level-1 variable

• When random intercept or random slope is of substantive interest

Conditions of applications

• RM ANOVA and multilevel modeling have 2 conditions of application in common:– Normality of the DV by cell of the IV• Few outliers

– Homoscedasticity (equality of variance)

– (Linearity: trivial in ANOVA since we only estimate mean differences)

12

Confounding

X

Y

X

Y

3 levels of Z

X and Y are associated within each level of Z

3 levels of Z

X and Y are not associated within each level of Z

13

Mediation: a special case of confounding

IV C DV Mediation

IV DV Confounding

C

IV A C Confounding 2

DV

14

Mediation: Baron and Kenny 1986

1. There must be a significant relationship between the independent variable and the dependent variable,

2. There must be a significant relationship between the independent variable and the mediating variable, and

3. The mediator must be a significant predictor of the outcome variable in an equation including both the mediator and the independent variable.

4. 4. The mediator must be caused by the IV and causing the DV

MacKinnon, D. P., Krull, J. L., Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science. 1(4). 173-181.

15

Moderation and interaction

• A variable moderates another when the association between the IV and the DV is different for one level of the moderator variable than for another level.

• Example: the effect of antibiotics on response time is very small for people not drinking alcohol but is large for people drinking alcohol (alcohol is the moderator variable)

• “The difference in response time due to antibiotics differs across alcohol level”– 2 different in the sentence

16

How to proceed

• Look at interaction plots• Estimate the interaction effect• If significant:– Estimate simple effects (relationship between IV

and DV for a specific level of a third (moderator) variable)

17

RM ANOVA: condition of application

• If compound symmetry is met, OK• If not, then test sphericity (e is an estimate of

sphericity, range 0-1 with 1 = sphericity met)• If sphericity not OK, – you could do a MANOVA (i.e., treat each condition as a

DV and look at all the DVs together)– You could correct for sphericity (this in fact corrects the

df). Best practice:• e>.75, use Huynh-Feldt correction• e<.75, use Greenhouse-Geisser correction

18

How to determine whether an IV is fixed or random

• Are the level of the IV chosen in a planned manner?

• Or were they randomly selected among other possible instances?

19

Comparing model

• Likelihood ratio test, if estimation is done with maximum likelihood (ML)…

• But ML produces biased estimates.– 2(Log(L1)-log(L0)) follows a Chi-square(p1-p2)

that is only asymptotically true, p value is overestimated• AIC/BIC and other information criteria– No absolute meaning– Can only be used to compare models: relative fit: The

lower the value, the better the model

20

Interindividual variability

• Autists who have more stereotypy have less quality of life

• In other words, the variability in stereotypy between 2 different individuals (interindividual variability) has an impact on quality of life

22

Intraindividual variability

• An autist who increases in stereotypy will have a lower quality of life

• In other words, the variability in stereotypy within an individual (intraindividual variability) has an impact on quality of life