How to Fool Yourself with SEM

James G. Anderson, Ph.D

Purdue University

Tripping at the Starting Line: Specification

1. Specify the model after the data are collected rather than before.

2. Omit causes that are correlated with other variables in a structural model.

3. Fail to have sufficient numbers of indicator latent variables.

4. Use psychometrically inadequate measures.

Tripping at the Starting Line: Specification (2)

5. Fail to give careful consideration to the question of directionality.

6. Specify feedback effects in structural models as a way to mask uncertainty about directionality.

7. Overfit the model (e.g., forget the goal of parsimony.

Tripping at the Starting Line: Specification (3)

8. Add disturbance or measurement error correlations without substantive reason.

9. Specify that indicators load on more than one factor without substantive reason.

Improper Care and Feeding: Data

10. Don’t check the accuracy of data input or coding.

11. Ignore whether the pattern of data loss is random or systematic.

12. Fail to examine distributional characteristics.

13. Don’t screen for outliers.

14. Assume that all relations are linear.

Checking Critical Judgment at the Door: Analysis and Re-specification

15. Re-specify a model based entirely upon statistical criteria.

16. Fail to check the accuracy of your programming.

17. Analyze a correlation matrix when it is inappropriate.

18. Analyze variables so highly correlated that the solution is unstable.

Checking Critical Judgment at the Door: Analysis and Re-specification (2)

19. Estimate a very complex model with a small sample.

20. Set scales for latent variables inappropriately.

21. Ignore the problem of starting values or provide grossly inaccurate ones.

22. When identification status is uncertain, fail to conduct tests of solution uniqueness.

Checking Critical Judgment at the Door: Analysis and Re-specification (3)

23.Fail to recognize empirical under-identification.

24. Fail to separately evaluate the measurement and structural portions of a hybrid model.

The Garden Path: Interpretation

25. Look only at indices of overall fit and ignore other types of fit information.

26. Interpret goodness-of-fit as meaning that the model is “proved”.

27. Interpret goodness-of-fit as meaning that the endogenous variables are strongly predicted.

28. Rely too much upon significance tests.

The Garden Path: Interpretation (2)29. Interpret the standardized solution in

inappropriate ways.

30. Fail to consider equivalent models.

31. Fail to consider alternative models.

32. Reify Factors

The Garden Path: Interpretation(3)

33. Believe that a strong analytical method like SEM can compensate for poor study design or poor ideas.

34. As the researcher, fail to report enough information so that your readers can reproduce your results.

35. Interpret estimates of large direct effects from a structural model as “proof” of causality.

Reference

R.B. Kline, Principles and Practice of Structural Equation Modeling, NY:

Guilford Press, 1998.