modeling item response profiles using factor models, latent class models, and latent variable...
TRANSCRIPT
Modeling item response profiles using factor models,
latent class models, and latent variable hybrids
Dena Pastor
James Madison University
Purposes of the Presentation
• To present the model-implied item response profiles (IRPs) that correspond to latent variable models used with dichotomous item response data
• To provide an example of how these models can be used in practice
Item Response Profiles (IRPs)
1 2 3 40
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Item Number
Pro
port
ion
Res
pond
ing
Cor
rect
ly
Pattern Differences
IRPs for classes of examinees with different patterns
Elevation Differences
IRPs for classes of examinees with the same pattern, but differences in elevation
Latent Variable Model
PARALLELNON-PARALLEL
C
Latent Class Model
C is a latent categorical
variable with as many levels as #
of classes
C is a nominal latent variable
C is a ordinal latent variable
Exploratory Process
• In latent class modeling a variety of models are fit to the data with differing numbers of classes– 1-class model, 2-class model, 3-class
model, etc.
• Use fit indices and a priori expectations to determine the number of classes to retain
• Can allow latent categorical variable to be nominal and examine resulting profiles; can also constrain latent categorical variable to be ordinal
Alternative Model for Parallel Profiles
Do we have 3 classes, with no variability within class?
OR
Do we have 1 profile with systematic variability within class?
F
Factor ModelF is a latent continuous
variable
Different Models for Different IRPs
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
1 profile…
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
…+ within profile variability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
2 parallel profiles…
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
…+ within profile variability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
2 non-parallel profiles…
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
…+ within profile variability
LCM: 1 class
Factor Model
LCM: 2 classes (C is
ordinal)
Semi-parametric
Factor Model
LCM: 2 classes(C is nominal)
Factor Mixture Model
Latent Variable Hybrids
Deci
sion
sM
od
els
IRP
s
1
Number of profiles?(number of
classes)
no
Latent class model(LCM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
yes
Factor model(FM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
Systematic variability
within profiles?
1+
Nature of profile
differences?
Parallel
LCM with
parallel
profiles
Semi-parametric factor model
(SPFM)
Systematic variability
within profiles?
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
no
yes
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
Non-parallel
Factor mixtur
e model(FMM
)
LCM with non-
parallel profiles
Systematic variability
within profiles?
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
no
yes
Sem
i-p
ara
metr
ic f
act
or
mod
el
(SP
FM
)
F C
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
2 classes
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
FC
1 class: Factor Model!
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
2 classes, w/in class factor variance = 0
2 classes
Fact
or
mix
ture
mod
el
(FM
M)
F C
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
1 class: Factor Model!
F C
CLatent class model(LCM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 40
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
2 classes, w/in class factor variance = 0
1
( 1| ) ( 1| )K
i k ik
P u P u c
Marginal probability of getting an item correct is sum across classes of probability of getting item correct conditional on class membership
Conditional probability differs across models
exp( )
1 (exp( ))ki ki k
ki ki k
F
F
~ (0, )k kF N F C
Factor mixtur
e model(FMM
) exp( )
1 (exp( ))i i k
i i k
F
F
~ ( , )k k kF N F CSemi-
parametric factor model
(SPFM)
Cexp( )
1 (exp( ))ki
ki
Latent class model(LCM)
C
Latent class model(LCM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
C
C
IRPPath diagram
Latent Variable Distribution
C is ordinal
C is nominal
Semi-Parametric Factor Model
(SPFM)
IRP Path diagramLatent Variable
Distribution
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
F C
Measurement Invariance
Same measurement model parameters (thresholds, loadings) for each class
Quantitative differences between
classes
Factor Mixture Model(FMM)
IRP Path diagramLatent Variable
Distribution
F C
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
Measurement Non-Invariance
Different measurement model parameters (thresholds, loadings) for each class
Qualitative differences between classes
Example
• 9 dichotomously scored items measuring 3 aspects of psychosocial research:
1. Confidentiality
2. Generalizability
3. Informed Consent
• Sample 2,259 incoming freshmen tested in low-stakes conditions prior to start of classes
Exploratory Model Selection
• Exploratory model selection approach to answer the question, “What type and number of latent variables are most salient for our data?”
• Reasons to believe that IRPs would differ in pattern and/or elevation because students differ in:• Completion of psychosocial coursework
• Effort they put forth on test
Model Fit Indices
Model LL # paras BIC SSA-BIC LMRFM 1f -12628 18 25395 25338 NA
2f -12576 26 25352 25270 NA 3f -12547 33 25348 25243 NA
LCM 1c -12938 9 25946 25918 NA 2c -12649 19 25445 25384 0.00 3c -12591 29 25406 25314 0.07 4c -12538 39 25377 25253 0.01 5c -12520 49 25418 25262 0.14
SPFM 1f2c -12618 21 25399 25332 0.01
FMM 1f2c -12539 35 25348 25237 0.00
IRPs of 4 Class LCM
0.18 0.36 0.25 0.20
generalizability
2-class FMM
0.44
0.56
ˆ 0.27
ˆ 3.96X
Y
26. Which ethical practice is not considered by Marty?a) She failed to obtain
informed consent from her participants
b) She failed to randomly select participants
c) …d) …
Factor Variability Within Each Class
Visually Conveying Loading Information
Item Content Item Number Class X Class YConfidentiality 17 0.26 0.36
34 0.28 0.7439 0.16 0.81
Generalizability 25 0.46 -0.0327 0.49 -0.0736 0.46 0.27
Informed Consent 23 0.51 0.3026 0.88 0.3532 0.44 0.31
Standardized Loadings
ˆ 0.27
ˆ 3.96X
Y
X Y
Validity Evidence for 2-class FMM Solution
• Students with higher SAT-V scores, who reported put forth more effort on the test, and who have completed psychosocial coursework more likely to be in Class X
• Positive relationship between SAT-V, coursework completion and factor scores in that class (negative relationship with effort)
• Negative relationship between number of missing responses and factor scores in Class Y
X Y
Correspondence Between Models
A & B from
LCM, X from FMM
C & D from
LCM, Y from FMM
X & Y from FMMwith
intervals
Parting Thoughts…
• These models are like potato chips…– It was so much easier to settle on a brand of
chip when I had a limited number of brands to choose from
– But I also like having more brands because it increases my chances of finding the brand that is right for me
– With all these brands, it is possible that some are selling essentially the same chip….but which ones?
– When two brands are essentially the same chip, what criteria do I use to choose between the two brands?
Pastor, D. A., & Gagné, P. (2013). Mean and covariance structure mixture models. In G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd Ed.). Greenwich, CT: Information Age.
Pastor, D. A., Lau, A. R., & Setzer, J. C. (2007, August). Modeling item response profiles using factor models, latent class models, and latent variable hybrids. Poster presented at the annual meeting of the American Psychological Association, San Francisco.