psycho metrics 08

8/3/2019 Psycho Metrics 08

1/118

Delphine Courvoisier & Olivier Renaud Psychometric models summer 07 1

Questionnaires and measures validation:Psychometric models: Latent State-Trait

models and Multitrait-Multimethod models

Delphine Courvoisier, Olivier Renaud

Section of Psychology

University of Geneva


2/118

General Information

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Optional course (3 ECTS) with practicals included.

Teachers:

Delphine Courvoisier ([email protected])

Olivier Renaud ([email protected])

Time: tuesday 16h15 18h

Place: UniMail M2160 / M5183

Website: https://dokeos.unige.ch/home/courses/751514

Work load: 3 ECTS = 6 work hours per week:

course: 2 hours

practicals with Mplus: 2 hours

home reading (6 articles): 2 hours


3/118

Materials for course and practicals

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Dokeos. Every document used in this course and practicals can befound on the centralised website called Dokeos. You should down-load and print these document because they will not be distributed

during the course. If you do not have a computer, the faculty hasseveral computer rooms with internet connexion.The documents on Dokeos are very summarized and thereforeinsufficient to understand the course and practicals.

Materials should be available at the latest the week-end before thecourse.


4/118

Exam

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


The exam will be in july (or september). Grades range from 0 to 6(4=sufficient). They will be based on:

10%: Summary of your analysis of a data set: prepared beforethe written exam.

90%: Written exam (2 hours) with questions on:

Theory Interpretation of the output of your analysis

For the extraordinary exam session (january-february), the evalua-

tion will be an oral exam.


5/118

Bibliography: Validity

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Borsboom, D., Mellenbergh, G. J., van Heerden, J. (2004).The concept of validity. Psychological Methods, 4, 10611071.

Schmitt, M. (2006). Conceptual, theoretical, and historicalfoundations of multimethod assessment. In M. Eid, E. Diener(Eds.) Handbook of multimethod measurement in psychology.

Washington, DC: American Psychological Association (pp. 925).


6/118

Bibliography: Structural Equation Models

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Bacher, F. (1987). Les modeles structuraux en psychologie.Presentation dun modele Lisrel. Premiere partie. Le TravailHumain, 50, 347370.

Bacher, F. (1987). Les modeles structuraux en psychologie.Presentation dun modele Lisrel. Deuxieme partie. Le TravailHumain, 51, 273288.

Hoyle, R. H. (1995). The structural equation modeling ap-proach: Basic concepts and fundamental issues. In R. H. Hoyle(Ed.), Structural equation modeling (pp. 115). ThousandsOaks, CA: Sage Publications.

Schermelleh-Engel, K., Moosbrugger, H., Muller, H. (2003).Evaluating the fit of structural equation models: tests of sig-nificance and descriptive goodness-of-fit measures. Methods

of Psychological Research Online, 8(2), 23-74.


7/118

Bibliography: Latent State Trait theory and models

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Steyer, R., Ferring, D., Schmitt, M. J. (1992). States andtraits in psychological assessment. European Journal of Psy-chological Assessment, 8(2), 79-98.

Steyer, R. ; Schmitt, M. & Eid, M. (1999). Latent state-traittheory and research in personality and individual differences.European Journal of Personality, 13, 389-408.

Courvoisier, D. S. (2006). Unfolding the constituents of psy-chological scores: Development and application of mixture and

multitrait-multimethod LST models. Unpublished doctoraldissertation, University of Geneva, Switzerland. (pp. 1423).


8/118

Bibliography: Multitrait Multimethod theory and models

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Eid, M. ; Lischetzke, T. ; Nubeck, F. & Trierweiler, L. (2003).Separating trait effects from trait-specific method effects inmultitrait-multimethod analysis: A multiple indicator CTC(M-

1) model. Psychological Methods, 8, 38-60. Eid, M. (2000). A multitrait-multimethod model with minimal

assumptions. Psychometrika, 65, 241-261.

Eid. M, Lischetzke, T., Nussbeck, F. W. (2006). Structuralequation models for multitrait-multimethod data. In M. Eid,E. Diener (Eds.) Handbook of multimethod measurement inpsychology. Washington, DC: American Psychological Asso-

ciation (pp. 283299). Courvoisier, D. S. (2006). Unfolding the constituents of psy-

chological scores: Development and application of mixture and

multitrait-multimethod LST models. Unpublished doctoral

dissertation, University of Geneva, Switzerland. (pp. 6269).


9/118

Bibliography: Monte Carlo studies

Introduction

Information

Materials

Exam

Bibliography

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Muthen, L. K., Muthen, B. O. (2002). How to use a montecarlo study to decide on sample size and determine power.Structural Equation Modeling, 9, 599620.

Courvoisier, D. S., Eid, M. & Nussbeck, F. W. (2007). Mix-ture Distribution Latent State-Trait Analysis: Basic Ideas andApplications. Psychological Methods, 12(1), 80104.


10/118

Validity: definition

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

ReliabilitiesVisually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A test is valid if it measures what it purports to measure(Kelley, 1927, p. 14)

Do the empirical relations between test scores match theoret-ical relations in a nomological network (Cronbach & Meehl,1955)

Are interpretations and actions based on test scores justified

not only in the light of scientific evidence but with respect tosocial and ethical consequences of test use? (Messick, 1989)


11/118

Validity: definition

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A test is valid for measuring an attribute if and only if a) theattribute exists and b) variations in the attribute causally pro-duce variations in the outcomes of the measurement procedure

(Borsboom, Mellenbergh & van Heerden, 2004).

Correlations cannot provide more than circumstantial evidence forvalidity. Problem of validity cannot be solved by psychometric tech-

niques or models alone. (Borsboom, et al., 2004)

Schmitt, M. (2006)


12/118

Validities

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Internal validity: Certainty with which a researcher can saythat observed changes function of the conditions of treatmentof the experience were really caused by the independent vari-

able (Myers & Hansen, 2003, p. 101). External validity: Quality of an experiment results that can

be generalized or applied to other subjects and other situationsthat were not directly tested. (Myers & Hansen, 2003, p.

101).

Convergent validity: Measures of the same construct mea-sured by different methods should be similar.

Discriminant validity: Measures of several constructs shoulddiffer.

Incremental validity: Measures of a new construct shouldincrease the explanatory power of other related variables.


13/118

Validities (2)

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Construct validity: How much are the measures used repre-sentative of the concepts and domains of our hypothesis? Isthe relation observed between the independent and the depen-

dent variables also correct at the construct level?Risks:

Bad operationalization

Not all modalities of the IV were measured

Only one method was used to measure the construct

Interaction between treatments (all things being equal)

Face validity: Does the measure seem valid?

and (too?) many more . . .

Borsboom, D. (2004)


14/118

Reliability: definition

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Constant: Measure stay constant every time it is assessed andon every object assessed.

A reliable car starts every time the keys are turned.

Coherent: Several measures are coherent.

Researchers measuring the same subject with different instru-ments should obtain the same result.

When using a reliable operational definition to measure a specific

characteristic in similar groups, it should yield the same result.

WARNING: some characteristics change across time!


15/118

Reliabilities

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Interjudge. Used to assess the degree of similarity betweenseveral judges measuring the same phenomenon

Test-retest. Used to assess the stability of a measure acrosstime (inappropriate for changing phenomenon)

Parallel forms. Used to assess the coherence of measuresbuild in the same way with questions coming from the same

set of questions.

Internal coherence. Used to assess the coherence of itemsof the same measure.

. . .


16/118

Graphical representation of reliability and validity

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example



17/118

Validation of questionnaire: Why?

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Measuring imprecisely

Measuring precisely several underlying constructs

To assess validities by way of reliabilities

But reliability is still not equal to validity.Correlations cannot provide more than circumstantial evidence forvalidity. Problem of validity cannot be solved by psychometric tech-

niques or models alone. (Borsboom, et al., 2004)+ Estimate the different sources of influence on the measure

trait

occasion of measurement

experimenter

method

error of measurement

. . .


18/118

Validation of questionnaire: How?

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Correlations:

Interjudge: between judges

Test-retest: between the two occasions of measurement Parallel forms: between the two forms

Internal coherence: mean of correlations between eachitem and the sum of the other items OR correlation be-tween two randomly created subtests (ex. odd vs. evenitems)

Cronbachs

Structural Equations Models use the correlation (or covari-ance) matrix of all items


19/118

Psychometrics: definition

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Definition: Science studying measurement techniques (in psy-chology but also in other sciences) as well as the validationtechniques of these measures.

Initially developped for measuring intellectual performances(mental ages, intellectual quotient, development quotient foryoung children, . . . ) and personality (affectivity, emotions,

. . . ) . An indicator that does not have good psychometric properties

limits the interpretations of the results of an empirical research.

The signal may be lost in the noise. Researchers must estimate the psychometric properties of their

measures.


20/118

Classical test theory

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


We suppose that for each measure of an object Y, measurementerror E stop us from obtaining the true score T of this object.Moreover, this error can have a random component er as well as asystematic component es:

Y = T + E

E = es + er

The main goal of psychometrics is to avoid that the measurementerror E be:

unknown

large

systematic (non-random)

related to the constructs studied


21/118

Graphical representation of measurement error

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Random error (left) decrease precision of the results, and thus con-fidence in the reesults. However, results are not biased.


22/118

Graphical representation of measurement error

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Random error (left) decrease precision of the results, and thus con-fidence in the reesults. However, results are not biased.

On the contrary, systematic error (right) introduce bias in the re-sults, thereby decreasing validity.


23/118

Influences on measures

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.


Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A psychological score Y may vary as a function of:


24/118


25/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A psychological score Yjk may vary as a function of:

Stable construct j

Method of measurement k


26/118

I fl


27/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A psychological score Yjklc may vary as a function of:

Stable construct j


Momentary situations l

Structural differences c

I fl


28/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A psychological score Yjklc+ may vary as a function of:

Stable construct j


Momentary situations l


Measurement error

I fl


29/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example



LST

Stable construct j X


Momentary situations l X


Measurement error X

I fl


30/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example



LST MTMM

Stable construct j X X

Method of measurement k X



Measurement error X X



31/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example



LST MTMM Mixture




Structural differences c X




32/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

Measures

Influences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example



LST MTMM Mixture






To assess variation, one must measure a construct several timeswhile varying the source(s) of influence of interest.



33/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

MeasuresInfluences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A psychological score Yjklc + may vary as a function of:

mixture LST

Stable construct j X




Measurement error X



34/118


Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

MeasuresInfluences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


A psychological score Yjklc + may vary as a function of:

mixture LST MTMM-LST



Momentary situations l X X




35/118

Sources of influence


36/118

Sources of influence

Introduction

Psychometrics

Validity: Def.

Validities

Reliability: Def.

Reliabilities

Visually

Validation: Why

Validation: How

Psychometrics:Def

Classical test

Error

MeasuresInfluences

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Traits (ex. depression, well-being)

Response modes (ex. motor, verbal, physiological)

Traits and response modes may overlap (ex. cognitive, conativeand emotional response modes to depression could be threesubtraits of depression)

Dimensions of measurement (ex. frequency, number of errors)

Settings of measurement (ex. classroom, home, lab) Sources of measurement (ex. self-rating, friend rating)

Instruments of measurement (ex. implicit or explicit measures)

Methods of measurement (ex. interview, questionnaire, be-havioral observation)

Occasions of measurement

. . .

Confirmatory factor analysis (CFA) and structural equa-i d l (SEM)


37/118

tion models (SEM)

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


CFA often follows an exploratory factor analysis (EFA). In a CFA,one specifies:

number of factors

non zero loadings

correlations between factors

SEM are a very large class of data analysis. For example, CFA isa specific type of SEM in which existing factors can only correlatewith other factors.

Bacher, F. (1987) OR Hoyle, R.H. (1995)

SEM: definition


38/118

S

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


SEM are also called:

Simultaneous equation modeling

Covariance structure analysis

Causal modeling Causal analysis

LiSRel or EQS models

Path analysis

. . .

Definitions :

SEM are a formal description of the relations between specific

observed and latent variables

Hypothetical pattern of relations between variables

Confirmatory techniques used to validate or falsify hypothesesabout a specific analysis model.


39/118

Null hypothesis in SEM


40/118

yp

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


H0 : = p

So, researchers hope NOT to reject the null hypothesis. (p )!!!

violation of assumptions: observed variables are supposedto be multivariate normal and sample size is supposed to besufficiently large.

model complexity: 2 decreases when p increases (not par-

simonious) dependence on sample size: 2 increases with sample size.

E(2) = df

bad fit discrepancemodel data discrepancemodel reality

good fit coherence

model data

model reality????

Goodness-of-fit criteria


41/118

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


2df

= 2: good fit, 2df

= 3: acceptable fit

RMSEA: root mean square error of approximation

< .05 good fit, .05 .08 acceptable fit

SRMR: standardized root mean square residual

< .05 good fit, < .10 acceptable fit

Descriptive measures based on model comparisons:

NFI, NNFI (TLI), CFI, GFI, AGFI> .97 good fit, > .95 acceptable fit

Descriptive measures based on parsimony:

PGFI, PNFI (> .97 good fit, > .95 acceptable fit) Badness of fit:

AIC, CAIC, ECVI (the best model has the smallest index)

Schermelleh-Engel, K. (2004)


42/118

Steps to test SEM


43/118

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


1. choose variables of interest

2. explore characteristics and relations of chosen variables

3. propose a structure (specific and parsimonious)

4. draw the diagram of this structure

5. propose alternate models

6. draw the diagrams of the alternate models

7. test the prefered model as well as the alternate models

8. check the results

9. compare the goodness-of-fit criteria

Each SEM has equivalent models, i.e. models that have the samep but that differ on the theoretical level. One must take this into

Submodels of complex SEM


44/118

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


CFA are SEM in which factors can only correlate.Some SEM propose more complex relations between factors. For

example, regression between factors.

In complex SEM, one can distinguish two submodels:1. Measurement model: to define latent variables

Similar to factor analysis

2. Structural model: to define relations between latent variables

Similar to either correlation or regression

Best practice: test all measurement models. THEN test the struc-tural models.

Factor identification in SEM (scaling)


45/118

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Factors, being latent variables, are not scaled. It is therefore neces-sary to establish artificially a scale.

Two possibilities to identifiy the factors :

1. fix their variance (for example to 1).2. fix a loading coefficient to 1.

To identify the model, two conditions are necessary (but not suf-

ficient):1. each factor is identified

2. the degrees of freedom are not negative

Moreover, these two conditions must be satisfied locally, that is foreach measurement model and for the structural model, and globally.

Informations often missing in articles using SEM


46/118

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


1. Theoretical justification of proposed model

2. Definition or explanation of causality

3. Justification of the estimation method and proof of the validityof the underlying assumptions

4. Several goodness-of-fit criteria

5. Theoretical and statistical justification of the modification ofthe model

6. Enough information to allow replication of the analyses


47/118

Hershberger, 2003 (suite)


48/118

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Diagram elements


49/118

Introduction

Psychometrics

SEM: Theory

CFA-SEM

Definition

PrinciplesH0

Goodness-of-fit

Steps

Submodels

Scaling

Missing info

Popularity

Diagram

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


To understand and present models analyzing the data structure, itis necessary to use a precise graphical representation, a diagram.

Observed variable

Latent variable

Asymetrical effect (regression weight)

Symetrical effect (variance and covariance)

Simple regression


50/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression

Multiple regressionCFA-2 simple

AFC-2 complex

Saturated

Independence

Equivalent

M+ input: syntaxData

M+ output

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


x Y =1

Vx 2

Y = x +

where Y and x are standardized (m = 0, s = 1) and has a mean

of zero and a variance of

2

.

In this model, the value of Y for an individual i is Yi = xi + i

And the variance of Y is: V(Y) = 2V(x) + 2

dl =2(2 + 1)

2 3 = 0


51/118


52/118

2 factors confirmatory factor analysis, complex structure


53/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent


M+ output

LST: Theory

LST: Example

MTMM: TheoryMTMM: Examples

MC Study: Theory

MC Study: Example


1

f1

=1

f1

f1

=1

x1

12

u1

u1

x2

22

u2

u2

x3

32

u3

u3

x4

42

u4

u4

x5

52

u5

u5

x6

62

u6

u6

f2

=1

f2

f2

=1

2 3 4 6 7

Cf1

f2

=1 =1 =1 =1 =1 =1

5

dl = (6(6 + 1)/2) 14 = 7

V(x4) = 24 +

25 + 24C(f1; f2)5 +

24

C(x4; x5) = 4C(f1; f2)6 + 56

Saturated model


54/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent


M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


Saturated model

A B C D E F

All variables correlate and have a variance.

dl =6(6 + 1)

2 21 = 0

Independence model


55/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent


M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


Independence model

A B C D E F

Variables are uncorrelated and all variables have a variance.

dl = 6(6 + 1)2 6 = 15


56/118

Questions: Psychometrics and SEM


57/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent


M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


Define reliability

Define validation

Define validity

How can we test validity?

What is the difference between using SEM to estimate a mul-

tiple regression and estimating a multiple regression with or-dinary least square?

What are the df of a model with 5 observed variables and one

common factor. Draw the diagram and indicate all parameters.

What is a saturated and an independent model?

Mplus input: syntax


58/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent


M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


By convention: LATENT VARIABLES IN CAPITALS (LV)

observed variables in small letters (obs)

In Mplus:

Each line must end by ; and maximum length of 80 characters.

TITLE: title text will be written in the first lines of the output

DATA: FILE IS specifies where is the file FORMAT: 33F8.2 means 33 variables taking 8 charac-

ters per variables (5 characters for integer, one dot and 2

decimals) TYPE IS individual for raw data VARIABLE gives list of variables names USEVARIABLES indicates which variables will be used in

the analysis

Mplus input: syntax(2)


59/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent


M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


ANALYSIS:

TYPE IS general meanstructure (estimate the covariancesand the mean structure)

ESTIMATOR: for example, maximum likelihood (ML),maximum likelihood robust (MLR)

MODEL: begins description of the model.

First loading is fixed to 1 by Mplus even if written LV by obs1

OUTPUT: indicates which outputs should be printed.

sampstat: gives sample statistics modindices(5.0): gives modification indices above 5

residual: gives residual matrix (distances between ex-pected and observed variance-covariance matrix) standardized: gives standardized parameters Tech 1: gives parameters specification and starting val-

ues (useful only for debugging)

Mplus input: syntax(3)


60/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent


M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


parameter syntax example

LV variance LV;LV mean [] [LV];

obs measurement error (variance) obs;obs intercept [] [obs];

loadings by LV by obs1 obs2;correlations with LV1 with LV2;

obs1 with obs2;regressions on LV1 on LV2 (Y on X);fixed parameters @ LV by obs1@1 obs2@1;parameters fixed equal (x) LV1(1); LV2(1);

From Eid, 1994


61/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent

M+ input: syntax

Data

M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


Well-being: likert scale (1 to 5)

happy unhappy

good not good well unwell

content not content

N = 501

Mean age = 31.2 years (min. 17, max. 78)

292 females, 211 males

Mplus output


62/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

Simple regression


AFC-2 complex

Saturated

Independence

Equivalent

M+ input: syntax

Data

M+ output

LST: Theory

LST: Example


MC Study: Theory

MC Study: Example


SAMPLE STATISTICS: means of observed variables and ob-served variance-covariance and correlation matrices

TESTS OF MODEL FIT: 2, CFI/TLI, loglikelihood, informa-tion criteria, RMSEA, SRMR

MODEL RESULTS: parameters:

estimates (unstandardized)

standard error (S.E)

ratio of estimates on S.E

estimates (standardized by the parameter)

estimates (fully standardized)

RESIDUAL OUTPUT: (observed estimated) MODIFICATION INDICES:

for each additional parameter, decrease in 2

TECH OUTPUT

Diagram


63/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

Diagram

Observed variables

True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Measures

Y11

Y21

Y12

Y22

Y13

Y23

Y14

Y24

Yi

Diagram


64/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

Diagram

Observed variables

True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Measures

Y11

Y21

Y12

Y22

Y13

Y23

Y14

Y24

Yil

Diagram


65/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

Diagram

Observed variables

True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Measures are influenced by stable traits

Y11

Y21

Y12

Y22

Y13

Y23

Y14

Y24

1

1

1

1 21

T1

22

23

24

Yil = il + ilT


66/118

Diagram


67/118

IntroductionPsychometrics

SEM: Theory

SEM: Examples

LST: Theory

Diagram

Observed variables

True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Measures are influenced by stable traits and momentary occasions

Y11

Y21

Y12

Y22

Y13

Y23

Y14

Y24

O

1

1

1

1

1

1

1

1

1

1 21

1

1

1

1

1

1

1

O2

O3

O4

IST2

T1

22

23

24

Yil = il + ilT + iilISTi + ilOk

Diagram


68/118


SEM: Theory

SEM: Examples

LST: Theory

Diagram

Observed variables

True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Measures are influenced by stable traits and momentary occasionsbut there is still an error of measurement

Y11

Y21

Y12

Y22

Y13

Y23

Y14

Y24

O1

1

1

1

1

1

1

1

1

1 21

E11

1

1

1

1

1

1

1

O2

O3

O4

IST2

T1

22

23

24

Yil = il + ilT + iilISTi + ilOk + Eil

Observed variables


69/118


SEM: Theory

SEM: Examples

LST: Theory

DiagramObserved variables

True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


The values of one random variable Yil are the scores on an item orscale i on an occasion l.The values of the mapping pU: U are the observational unitsu U (e.g., individuals).

pU() = u, for each .

The values of the mappings pSitl : Sitl, l = {1, . . . , p} are thesituations sit

l Sit

l:

pSitl() = sitl, for each .

Steyer, R. (1992) AND one of the other

Observed variables (2)


70/118


SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Both the persons and the situations are supposed to be pickedat random during the experiment.

pU() and pSitl() are assumed to be independent.

Assumption: Occasions cause random fluctuations around thetrait.

This implies: The mean of the occasion-specific variables is zero.

The correlations between trait and occasion-specific variables

are zero.

True score variable


71/118


SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


The latent true-scores Sil are defined as the expectation of theobserved variables given the individuals and the situations:

Sil := E(Yil|pU, pSit).

Sil characterizes a person-in-a-situation on an indicator i on anoccasion l.

The residual variables can be defined as the observed variables minusthe conditional expectation presented above:

Eil := Yil E(Yil|pU, pSit).

The proof that residuals are necessarily uncorrelated with the inde-pendent variable of the regression is given in Steyer, 1988.

Latent variables


72/118


SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


The observed variables Yil can each be decomposed into:

Yil = Sil + Eil

= E(Yil|pU) + [E(Yil|pU, pSit) E(Yil|pU)] + Eil

= Til + Oil + Eil,

where Til := E(Yil|pU) and Oil := [E(Yil|pU, pSit) E(Yil|pU)].

Til characterizes the person itself across situations.

The residual Oil represents effects of the situations and/orperson-situation-interactions.

Visually

E11


73/118


SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Y11

Y21

Y12

Y22

Y13

Y23

Y14

Y24

E11

T1

T1

IST

IST

IST

IST

T1

T1

T1

T1

T1

T1

O

O

O

O

O

O

O

O

Representation


74/118


SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Assumption of a common linear factor for each type of latent vari-able (T, IST and O):

Til = il + TilTi,E(Til|T1l) = ISTilISTi,

Oil = OilOl.

The complete linear regression determining the observed variablesYil is then:

Yil = il + TilTi + ISTilISTi + OilOl + Eil

where ISTi is the indicator-specific deviation from the common traitT1.

ISTi = 0 when i = 1.

Decomposition of variance


75/118


SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


V ar(Yil) = 2Til

V ar(Ti)+2ISTil

V ar(ISTi)+2Oil

V ar(Ol)+V ar(Eil)

The decomposition contains no covariance because, for each ob-

served variable, the latent variables are independent.

Definition of several variance component coefficients. The coeffi-cients are computed for each observed variables.

One coefficient concerns measurement error:

Unrel(Yil) =V ar(Eil)

V ar(Yil).

Conversely, the reliability is:

rel(Yil) = 1 Unrel(Yil).

Decomposition of variance: stable dispositions

2 2


76/118


SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Con(Yil) = 2TilV ar(Ti) +

2ISTiV ar(ISTi)

V ar(Yil),

ComCon(Yil) =2TilV ar(Ti)

V ar(Yil),

MetSpe(Yil) =2ISTiV ar(ISTi)

V ar(Yil).

The consistency coefficient Con(Yil) represents the proportionof stable disposition in the psychological score Yil.

The common consistency coefficient ComCon(Yil) representsthe proportion of stable disposition due to the common trait

in the psychological score Yil. The method specificity coefficient MetSpe(Yil) represents the

proportion of stable disposition due to the indicator-specifictrait in the psychological score Yil.

Decomposition of variance: momentary influences


77/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


OSpe(Yil) =2OilV ar(Ol)

V ar(Yil).

The occasion specificity coefficient OSpe(Yil) represents theproportion of situational influences in the psychological scoreYil.

Uniqueness

I t d ti


78/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


If the latent variables of the model define an LST model, any lineartransformation of these latent variables will still yield an LST model.

Ti = Ti + Ti Ti,

O

l = Ol Ol,

il = il Ti TilTi

,

Til = Til/Ti ,

Oil = Oil/Oil ,

where Ti , Ti , Oil , R and Ti , Oil > 0

For example: the observed variables measure skin temperature. Thelatent trait variable can be scaled on Celsius or Fahrenheit degreesand still be an LST model.

Meaningfulness

Introduction


79/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Ti(1) Ti(2)Ti(3) Ti(4)

= T

i (1) T

i (2)Ti (3) T

i (4),

for [Ti(3) Ti(4)] and [T

i (3) T

i (4)] = 0.The ratio of the difference of the value of two individuals u

1and

u2 on the trait variable Ti on the difference of the value of twoother individuals u3 and u4 on this trait variable is equal to thesame ratio for the same individuals on a transformed trait variable.

Ol(1)

Ol(2)=

Ol(1)

Ol(2),

for Ol(2) and O

l(2) = 0.

The occasion-specific deviation value Ol(1) of a person u1 is n-times (larger or smaller than) the value Ol(2) of a person u2.

Meaningfulness(2)

Introduction


80/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


2TilV ar(Ti) = 2Til

V ar(Ti ),

Cor(Ti, Ti) = Cor(T

i , T

i),

2

OilV ar(Ol) =

2

OilV ar(O

l),

Although variances of latent variables by themselves are notmeaningful, variances of latent variables multiplied by their

squared loadings are.

These products are used for consistency and specificity co-efficients.

The correlations between latent trait variables are also mean-ingful and can therefore be interpreted.

Testability

Introduction


81/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Cov(Eil, Ti) = 0,

Cov(Eil, Ol) = 0,

Cov(Ti, Ol) = 0,Cov(Ol, Ol) = 0,

Cov(E(il), E(il)) = 0.

Testability (2)

Introduction


82/118

Introduction

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


From the description of the null covariances, it follows that thecovariance structure of the model is:

= TT

T +OO

O +

T is the vector of loadings of the trait variables on the ob-served variables,

O is the vector of loadings of the occasion-specific variableson the observed variables,

T is the matrix of variances and covariances between traitvariables

O is the matrix of variances and covariances between occa-sion variables

is the diagonal matrix of the measurement error variables

Identifiability

Introduction


83/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Procedure:

1. Write all the equations of the expected variances and covari-ances.

2. Solve the system of

v(v+1)

2 equations with p unknown param-eters.

NO PROGRAM CAN SOLVE THESE

SYSTEMS OF NON-LINEAR EQUATIONS.

However, the identification conditions of the LST models have al-ready been studied:

at least three indicators of the same construct measured at three different occasions

minimum of nine observed variables.

Questions: LST models

Introduction


84/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory


True score variable

Latent variables

Visually

Representation

VarianceUniqueness

Meaningfulness

Testability

Identifiability

LST: Example

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


What assumptions are necessary for the LST model to be de-fined?

For each assumptions, how can one test if they are fulfilled?

What does the true score represent? And the error?

Without the assumption of common linear factors, how manylatent variables are there for an LST model with 3 indicatorsand 3 occasions of measurement and ISTs?

What is the advantage of the additive decomposition of vari-ance?

Describe a model that does not have an additive decomposi-tion of variance.

Describe the variance component coefficients. Write all variance-covariance equations of an LST model with

2 indicators and 2 occasions of measurement and IST.


85/118

Analysis procedure (2)

Introduction


86/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

Analysis

Model comparison

M+ input: LST

Results: sample

Results: gof

Results: varianceResults: param.

Exercises

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


End with most restricted model

most restricted model with one common trait. Estimate:

all loadings fixed to one

all intercepts fixed to zero

variance of the trait variable

variances of the occasion-specific variables fixed equal

mean of the commmon trait variable error variables fixed equal


87/118

Mplus input: LST model

1 F d l f i


88/118


1 Freeest model pu free.inp2 Fix the loading equal to one ex-

cept for the loading of the com-mon trait variables on the second

indicator observed variables

pu fixload.inp

3 Fix also the intercepts to zero ex-cept for the intercepts of the sec-ond indicator observed variables

pu fixload fixint.inp

4 Model with occasion-specific vari-ables variances fixed equal

pu fixload fixint =varO.inp

5 Model with all measurement er-rors fixed equal

pu fixload fixint =varO =El.inp

6 Model with the first measurementerrors fixed equal and all the othermeasurement errors fixed equal

pu fixload fixint =varO =E1 =El.inp

Mplus input: LST model (2)

Introduction

M d l 2 i i i f h i


89/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

Analysis

Model comparison

M+ input: LST

Results: sample

Results: gof


Exercises

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Model 2 is measurement invariant for the common trait

the common trait has the same meaning across time

Model 3 is measurement invariant for the common trait and

the common indicator-specific trait

Model 4 is measurement invariant for the trait and occasion-specific variables

Model 5 and 6 are perfectly measurement invariant


90/118

Results: goodness-of-fit statistics

Model ID 2 df p CF I RMSEA 2diff df p CAIC BIC aBIC

M1: one trait 65 9 16 00 0 98 08 18590 3 18708 4 18619 5


91/118


M1: one trait 65.9 16 .00 0.98 .08 18590.3 18708.4 18619.5M2: M1 plus indicator-specific trait

10.3 12 .59 1.00 .00 51.5 4 .00 18701.1 18669.1 18567.5

M3: M2 plus measurement

invariance with respect tothe loadings, and occasion-specific variances

34.7 30 .25 1.00 .02 24.5 18 .14 18598.3 18584.3 18539.9

M4: M3 plus measurementinvariance with respect to

the mean structure

46.7 36 .11 1.00 .02 12.4 6 .05 18567.5 18559.5 18534.1

M5: M4 plus autoregressivestructure

42.8 32 .10 1.00 .03 4.1 4 .40 18591.3 18579.3 18541.3

Note: CAIC = Consistent Akaike Information Criterion, BIC = Bayesian Information

Criterion; aBIC = sample-size adjusted Bayesian Information Criterion; The 2

diff- test refersto the 2-difference test comparing a model with a model in the preceding line. The appropriateapproach for the MLR estimation method has been used. Please note that model M1 is nestedwithin Model M2, and that model M4 is nested in Model M5. In the two other cases a model isnested in a model that is presented in the preceding line.


92/118

Results: parameters

Introduction

P h i

MeansMeans Trait

Variances

Trait

Loadings

Error

Variances

Occasion-

specificVariances


93/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

Analysis

Model comparison

M+ input: LST

Results: sample

Results: gof


Exercises

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Y11

0.64 (0.13)3.55 (0.44)15.15 (0.11)

0 0.56 (0.10)

Y21

Y12

Y22

Y13

Y23

Y14

Y24

7.76 (0.42)

7.76 (0.42)

7.76 (0.42)

7.76 (0.42)

0

0

0

0

1

1

1

1

1

1

1

1

11.02

(0.03)

1.02

1.02

1.02

2.00 (0.17)

2.00 (0.17)

1.23 (0.07)

1.23 (0.07)

1.23 (0.07)

1.23 (0.07)

1.23 (0.07)

1.23 (0.07)

1

1

1

1

1

1

1

Variances Loadings Variances specificVariances

Standard errors in parentheses, indicated only once for trait loadings. Interceptsof the model: 0.70(.48) for the i = 2 indicator variables.

Exercises

Introduction

P h t i Data from Cole (1995) on depression and anxiety in children


94/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

Analysis

Model comparison

M+ input: LST

Results: sample

Results: gof


Exercises

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Data from Cole (1995) on depression and anxiety in children.

Depression measured by CDI for children self-report (26items). The scale was split in two test-halves of 13 items.

Depression measured by TRID for teacher-report (26items). The scale was split in two test-halves of 13 items.

Anxiety measured by RCMAS for children self-report

Anxiety measured by TRIA for teacher-report

TRID and TRIA are teachers version of the childrens scales. n = 375

Subjects: children in grade 4 and then 5. Subjects weremeasured once a semester on 4 semesters.

variables are presented in this form: ds th1 1 where thefirst letter is for depression or anxiety, the second letter isfor self or teacher rating, the second part is for Test-Half1 or 2 and the third part is for wave (1, 2, 3 or 4).

Exercises

Introduction

Psychometrics Analyze with LST models the following data set


95/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

Analysis

Model comparison

M+ input: LST

Results: sample

Results: gof


Exercises

MTMM: Theory

MTMM: Examples

MC Study: Theory

MC Study: Example


Analyze with LST models the following data set(cole-4o-dast-miss.dat). You can choose either de-pression or anxiety as the trait (but not both). Please chooseself-report and not teacher report. A draft of the input can

be found on dokeos: adst empty miss.inp. Try to find the most parsimonious and yet precise and inter-

pretable model.

Note: The commands CLUSTER IS class; and TYPE IScomplex; are necessary because the data is multilevel. Thissubject will not be discussed in this course. Just let the com-mands and proceed with your LST models.

Question: Interpret thoroughly your findings

Question: Explain the path you took to restrict your modeland why you took this path.

MTMM models: generalities

Introduction

Psychometrics There exist more than 20 different MTMM models


96/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


There exist more than 20 different MTMM models.

Methods can be any kind of repeated measurement for the sameconstruct:

raters

occasions of measurement

in fact, any kind of sources of influence presented on page 22except trait.

MTMM models are useful to determine convergent and discriminant

validity.

Eid, Lischetzke and Nussbeck (2006) AND one other

Choosing an MTMM model

Introduction

Psychometrics Distinctions to choose the model most appropriate for a specificd t t


97/118

Psychometrics

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


Distinctions to choose the model most appropriate for a specificdata set:

Single indicator vs. multiple indicators MTMM models.

If possible, always choose multiple indicators models becausethey allow a separation of measurement error from method-specific variables.

Allows correlated methods or not.

MTMM correlation (Correlated Trait CT) model

Model deviations from the trait or model specific method fac-tors.


98/118

Correlated trait model

Introduction

Psychometrics allows:


99/118

y

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


a o s

estimation of the latent correlations between the trait-method units

the construction of a latent MTMM correlation matrix(error-free variant of the MTMM matrix)

does not allow:

separation of trait and method effects

estimation of general method effects

estimation of general trait effects

Correlated trait model: diagram

Introduction

Psychometrics Y111

E111

1

1


100/118

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


111

Y211

Y121

Y221

Y112

Y212

Y122

Y222

Y113

Y213

Y123

Y223

M.12M212

M.22M222

M213

M223

M222

M.13M213

M.23M223

11

11

1

1

1

1

M222

1

M211

M.21M222

M213

M223

M221

M212

M.22M222

11

11

1

1

1

1

M.11

Correlated Trait Correlated Uniqueness model

Introduction



101/118

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


estimation of common trait and discriminant validity

estimation of the generalizability of method effects within

traits (by correlations between method factors belongingto the same trait)

inclusion of covariates on the method level

does not allow:

estimation of a common method factor

estimation of the generalizability of method effects across

traits (correlations of method factors belonging to differ-ent traits not allowed)

quantification of trait, method and error influences

Correlated Trait Correlated Uniqueness model (2)

Introduction

Psychometrics can be extended to:


102/118

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


Correlated Trait Uncorrelated Method (CTUM): if themethod factors belonging to the same method are iden-

tical Correlated Trait Correlated Method (CTCM): if CTUM

and the method factors can correlate.

These models are very restrictive because they imply a perfectunidimensionality of the method influences belonging to thesame method.

Correlated Trait Correlated Uniqueness model: diagram

Introduction

Psychometrics Y111

E111

1

1


103/118

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


Y211

Y121

Y221

Y112

Y212

Y122

Y222

Y113

Y213

Y123

Y223

M.12

T11.

T21.

T12.

T22.

1

1

1

1

T112

M212

M.22M222

M213

M223

M222

M.13M213

M.23M223

T113

T122

T123

T222

T223

T212

T213

11

11

1

1

1

1

M211

M.21

M222M221

11

11

M.11

Correlated trait Correlated Method 1 model

Introduction



104/118

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


estimation of the generalizability of method effects withintraits (by correlations between method factors belongingto the same trait)

estimation of the generalizability of method effects acrosstraits (by correlations between method factors belongingto different traits)

quantification of trait, method and error influences

estimation of heteromethod coefficients of discriminantvalidity (by correlations between method factors of a traitand trait factor of another trait)

does not allow: estimation of method effect for the standardmethod

This model can be used only if one method can be chosen as areference.

Correlated trait Correlated Method 1 model: diagram

Introduction

Psychometrics Y111

E111


105/118

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM models

Choice of model

CT

CTCU

CTC(M1)

MTMM: Examples

MC Study: Theory

MC Study: Example


a

b

a

b

Y211

Y121

Y221

Y112

Y212

Y122

Y222

Y113

Y213

Y123

Y223

M.12

T11.

T21.

T12.

T22.

1

1

1

1

M.22

M.13

M.23

11

11

1

1

1

1

From Kocum, 2006

Introduction

Psychometrics Traits:


106/118

SEM: Theory

SEM: Examples

LST: Theory

LST: Example

MTMM: Theory

MTMM: Examples

Data

Analysis

ResultsExercises: Cole

MC Study: Theory

MC Study: Example


Work competence (5 items)

Activity/potency (10 items)

Democratic Approach (8 items)

measured for:

manager female manager

covariate: Neosexism (11 items)

data with all items: Kocum2006.dat


107/118

Analysis (2)

Introduction

Psychometrics CTC(M1) model

psycho metrics 08

Documents