Analyzing Longitudinal Rating Data: A three-level HLM

Shenyang Guo, Ph.D.Shenyang Guo, Ph.D.University of TennesseeUniversity of Tennessee

sguo@utk.edusguo@utk.eduUNC at Chapel Hill

(Starting from Aug. 2002)

For details of this presentation, see

Guo, S., & Hussey, D. (1999). “Analyzing Longitudinal Rating Data: A three-level hierarchical linear model”. Social Work Research 23(4): 258-269.

You could not step You could not step twice into the twice into the same river, for same river, for other waters are other waters are ever flowing onto ever flowing onto you.you. -- -- Heraclitus Heraclitus (B.C.530-(B.C.530-470)470)

010203040506070

0 2 4 6 8 10Time

Y

010203040506070

0 2 4 6 8 10Time

Y

Rater ARater B

010203040506070

0 2 4 6 8 10Time

Y

010203040506070

0 2 4 6 8 10Time

Y

Hypothetical Data:Hypothetical Data:Two raters’ ratings on a single subject Two raters’ ratings on a single subject

1a 1b 1c 1d

Subject Rater Time Total Program ______________________ _______________________ (Days) DSMD

ID Gender ID Relation Scoreto Subject

_________ _________ _________ __________ _________ _________ _________1 Boy 1 Teacher 0 57.7 Other

1 Teacher 99 52.5 Other1 Teacher 219 50.6 Other2 Teacher 309 56.4 Other3 Caretaker 100 54.4 Other

2 Boy 1 Caretaker 0 56.1 Residential2 Teacher 93 60.8 Residential2 Teacher 218 43.5 Residential3 Teacher 334 63.8 Residential3 Teacher 383 74.8 Other3 Teacher 415 73.0 Other3 Teacher 512 60.7 Other3 Teacher 625 75.1 Other4 Caretaker 93 62.9 Residential4 Caretaker 218 59.7 Residential4 Caretaker 334 62.8 Residential4 Caretaker 390 61.9 Other

. . . . . .143 Girl 1 Teacher 0 55.7 Other

2 Teacher 262 53.2 Other

144 Boy 1 Teacher 0 40.0 Other1 Teacher 111 40.0 Other2 Caretaker 111 62.7 Other

A segment of data

The Generalizability Theory-- Cronbach et al. (1972) The theory effectively demonstrates

that measurement error is multifaceted.

Using the G theory, we may conceptualize the longitudinal rating data collected by multiple raters as a two-facet design (that is, rater and occasion) with study subjects as the object of measurement.

Sources of variability:A two-facet cross design

1. Subjects 2s

2. Raters 2r

3. Occasions 2o

4. Interaction of subject and rater 2sr

5. Interaction of subject and occasion 2so

6. Interaction of rater and occasion 2ro

7. Residual 2sro,e

Sources of non-random variation associated with raters

Some raters may understand the rating rules differently from others -- increased 2

r

A rater may be particularly lenient in rating a particular child -- increased 2

sr

A rater may give a rating more stringent one day than on other days -- increased 2

ro

A General Measurement Model Developed by Diggle, Liang, & Zeger (1995, pp.79-81)

The error term of the linear model

Y= X + can be expressed as

ij = d'ij Ui + Wi(t ij) + Z ij

where dij are r-element vectors of explanatory variables attached to individual measurements, and the Ui , the {Wi(t ij)} and the Z ij correspond to random effects, serial correlation and measurement error, respectively.

Estimated variance components:A two-facet nested model

Variance of children 2s : 70%

Variance associated with raters 2r,sr: 24%

(Non-negligible)

Variance of residual 2o,so,or,sor,e: 6%

A two-level specification (Fixed-rater model):

Level 1: Ytj = 0j + 1j (Time) tj + 2j

(Residential) tj + etj

Level 2: 0j = 00 + 01 (Teacher)j + 02 (Boy)j + u0j

1j = 10 + 11j (Teacher)j

2j = 20

Adverse consequences(when the 2-level model is used):

Biased estimation of coefficients, larger residuals, and poorer fit;

Misleading significance tests; Misleading decomposition of

variability into various sources;

A three-level specification (Random-rater model):

Level 1: Ytij = 0ij + 1ij (Time) tij + 2ij (Residential) tij +

etij

Level 2: 0ij = 00j + 01j (Teacher)ij + r0ij

1ij = 10j + 11j (Teacher)ij 2ij = 20j

Level 3: 00j = 000 + 001 (Boy)j + u00j 01j = 010

10j = 100

11j = 110

20j = 200

Fixed & Random Effects Three-level Two-level and Model Evaluation ________________________________________

B SE B SEFixed Effect

Intercept 69.670 ** 1.717 69.527 ** 1.699 Time -0.007 ** 0.003 -0.005 * 0.003

Program Residential (Other is the reference) 1.597 0.991 1.616 1.038 Rater Teacher (Primary caretaker is the reference) -10.625 ** 0.951 -9.996 ** 0.827 Gender Boy (Girl is the reference) -5.925 ** 1.625 -5.880 ** 1.595

Interaction: Time by Rater Teacher 0.008 * 0.003 0.002 0.003

Evaluation statistics: Goodness-of-fit: AIC -3481.89 -3542.32 Residual: Theil's U 0.042 0.065 Deviation of predicted mean from observed mean: At Day 0 (n=181) -0.1236 -0.3754 At Day 99 (n=19) -0.8216 -1.8368 At Day 262 (n=32) 0.6753 1.4147

** p < .01, * p < .05

HLM comparison

Sample mean trajectoriesSample mean trajectories

Substantive results

The children changed their behavior over time at a rate of decreasing the DSMD total score by .007 per day (p<.01), or 5.114 in a two-year period.

Initially, the average DSMD total for girls placed in a non-residential program and rated by caretakers is 69.67 (p<.01).

Other things being equal, boys were judged to be less behaviorally disturbed than girls at any point in time by 5.925 (p<.01)

Substantive results continuedSubstantive results continued

Children placed in the residential program evidenced greater behavioral disturbance (1.597 higher) than others at any point in time. Teachers rated the children more positively (10.625 lower) than caretakers at any point in time (p<.01). This also indicates that these children presented more disturbed behavior at home or in the residential milieu than in the school setting.Children rated by teachers actually increased DSMD total score at a rate of .001 per day (-.007+.008=.001, p<.05), or no change over the two year span (i.e. increased .7305 in two years).

Emerging Applicability-- Situations in which the 3-level model may be useful

Multi-wave panel data collected by multiple raters

Multi-wave panel data collected via two methods: face-to-face and telephone interviews (e.g., AHEAD survey data)

Tau-equivalent measures: graphic presentation versus a verbal questionnaire

Recent Progresses in HLM (1)-- HLM and Structural Equation Modeling (SEM)

HLM and SEM share common assumptions and may yield same results.

Similarities between HLM and Latent Growth Curve Modeling (Willett & Sayer, 1994).

Intercept Slope

Yt=0Yt=1 Yt=2 Yt=3 Yt=4

e0e1 e2 e3 e4

di ds

W

1 1 1 1 1

1 1 11 1

01 2 3 4

1 1

Latent GrowthCurve Model

A Latent Growth CurveA Latent Growth CurveModel and its Equivalent HLMModel and its Equivalent HLM-- Hox (2000), in Little et al. (edited) pp.27-29-- Hox (2000), in Little et al. (edited) pp.27-29

HLM

Ytj = 0j + 1j (Time) tj + etj

0j = 00 + 01 (W)j + u0j

1j = 10 + 11j (W)j + u1j

Recent Progresses in HLM (2)-- Modeling covariance structure within subjects

HLM is a special case of Mixed model. In general, a linear mixed-effects model is any model that satisfies the following specifications (Laird & Ware, 1982):

Yi = Xi + Zibi + i

bi ~ N (0,D),

i ~ N (0,i),

b1 …, bN, 1…, N independent,

SAS Proc MIXED allows specification of the covariance structure within subjects, that is, the covariance structure of i. The choices are compound symmetric, autoregressive order one, and more (Littell et al, 1996, pp.87-134). This is an idea borrowed from econometric time-series models.

Modeling multivariate change: whether two change trajectories (outcome measures) correlate over time? (MacCallum & Kim, 2000, in Little et al. (edited) pp.51-68). Examples:

Whether benefits clients gained from an intervention over time negatively correlate with the intervention’s side effects? Whether clients’ change in physical health correlates with their change in mental health? Whether a program’s designed change in outcome (e.g., abstinence from alcohol or substance abuse) correlates with clients’ level of depression?

Recent Progresses in HLM (3)-- Analyzing more than one dependent variable in the HLM framework

Important References (1) Laird & Ware (1982). Random-effects models

for longitudinal data. Biometrics, 38, 964-974.

Bryk & Raudenbush (1992). Hierarchical linear models: Applications and data analysis methods. Sage Publications.

Diggle, Liang, & Zeger (1995). Analysis of longitudinal data. Oxford: Clarendon Press.

Littell, Milliken, Stroup, Wolfinger (1996). SAS system for mixed models. Cary: SAS Institute, Inc.

Important References (2) Verbeke & Molenberghs (2000). Linear mixed

models for longitudinal data. Springer. Little, Schnabel, & Baumert (2000). Modeling

longitudinal and multilevel data. Lawrence Erlbaum Associates, Publishers.

Sampson, Raudenbush, & Earls (1997). Neighborhoods and violent crime: A multilevel study of collective efficacy. Science 277, 918-924.