covariance structure modeling in windows: a …joophox.net/publist/bms95.pdf · model fit, amos...

71-

COVARIANCE STRUCTURE MODELING INWINDOWS: A MULTITRAIT-

MULTIMETHOD ANALYSIS USING AMOS,EQS, AND LISREL

by

Joop J. Hox(University of Amsterdam, the Netherlands.

IJsbaanpad 9. 1076 CV Amsterdam; email [email protected])

Résumé. Modélisation de la structure de la covariance sous Windows: Une analyse multitrats-multiméthodea utilisant Amos, Eqs, et Lisrel. Cet article étudie les trois grands logiciels d’analyse dela structure de la covariance sous Windows. Le point essentiel de la comparaison réside en une analysed’une matrice 5x5 multitraits-multiméthodes. On montre ainsi que la possibilité de différents fonctionsd’ajustement et de procédures bootstrap dans les programmes actuels d’analyse de la structure de lacovariances permettent une analyse fine de données à problèmes. Modélisation de la structure de lacovariance. Modélisation d’équations structurales. Modélisation de la causalité Bootstrap.Fonctions d’ajustement. Analyses multitraits-multiméthodes.

Abstract. This article examines the three major packages that allow covariance strucure analysis (CSA)under Windows. The main point of comparison is the analysis of a 5x5 multitrait-multimethod matrix.This shows that the availability of different fit functions and bootstrap procedures in modern CSAprograms allow a detalled analysis of problematic data. Covariance Structure Analysis, StructuralEquation Modeling. Causal Modeling. Bootstrap, Fit Functions. Multitrait-Multimethod.

INTRODUCTION

Covariance Structure Analysis (CSA) is a powerful analysistechnique that allows examination of structural models with bothmanifest and latent variables (Bollen, 1989). For a long time, evenmedium sized models required using a mainframe computer.Recently, personal computers have become powerful enough to runstate-of-the art CSA programs. Also, they have become fast enoughto allow the routine application of computationally intensivemethods such as bootstrapping. For many large computerapplications. Windows is becoming the preferred work environment,because it provides access to all of the computer’s memory insteadof the 640kb that DOS allows, lets several programs run

at UNIVERSITEIT UTRECHT on February 1, 2010 http://bms.sagepub.comDownloaded from

http://bms.sagepub.com

p. 72

simultaneously, and has a graphical interface. Several SEM

programs are now also available in a Windows version. In this articleI use the Windows implementations of the program packages AMOS(3.1), EQS (4.02), and LISREL (version 8.03). This review starts witha short description of each program, followed by a comparison of thefeatures they offer and of some problems encountered during testruns. Next, a data set from a multitrait-multimethod design is

analyzed, which is known to have some problematic features.Various techniques offered by the programs are applied to deal withthese problems. The last section discusses the application of CSAfrom a users point of view.

AMOS

AMOS (Analysis of MOment Structures) is distributed as a

package that includes standard and Dos-extended versions and aWindows version. All versions accept the same command language.The user must build a command file that specifies names for theobserved and latent variables (including error terms, which Amosmodels as unique factors with a path fixed at one), and theirrelations. Amos is equation oriented. Thus, a model with a latentfactor ’Alienation’ measured by the observed variable ’Anomia’ wouldbe specified by either the equation:

Anomia = Alienation + (1) Error

or the graphical equivalent:

Anomia <-- AlienationAnomia <-- (1) Error

Amos can handle multigroup models, equality constraints,and models with means and intercepts. It offers five estimationmethods: Maximum Likelihood (ML), Generalized Least Squares(GLS), Unweighted Least Squares (ULS), Scale-free Least Squares(SLS), and Asymptotically Distribution Free (ADF) estimation (for adiscussion of these methods see Browne, 1982). Complexconstraints and polychoric correlations are not supported. To assessmodel fit, Amos computes an assortment of fit indices in addition tothe usual chi-square. A nice feature of Amos is that it allows thespecification of multiple models in a single analysis run. In thatcase, Amos will compute a chi-square test and several comparativefit indices for each pair of nested models. Another nice feature is thevery user-friendly approach to bootstrapping. Bootstrappedstandard errors and confidence intervals are available for allestimates, including bias-corrected intervals (Stine, 1989) and



p. 73

Bollen and Stine’s corrected bootstrap for goodness-of-fit measures(Bollen & Stine. 1992). All these are available by including theappropriate command in the command file; Amos will perform thespecified number of bootstraps and include the bootstrap standarderrors in the output file next to the asymptotic estimates.

Amos for Windows consists of two separate programs, Amosand AmosDraw. AmosDraw lets you specify your model by drawing apath diagram, employing the usual symbols: circles (and ellipses) forlatent factors, squares (and rectangles) for observed variables, andsingle and double headed arrows for relations. Equality restrictionsare specified by labeling parameters with identical labels. The

program comes with a variety of functions that Windows and Macusers have come to expect of a drawing program: tools to resize,move and copy different elements of the picture, to automaticallyalign selected elements, and various name- and title bars. The usercan personalize AmosDraw by putting the Windows Buttons thatcall up these tools in different boxes and hiding those tools that areseldom used. For example, I have settled for a layout with threeboxes: one for program directives such as opening/closing files, onefor Amos directives such as specifying groups or requesting ananalysis, and a big toolbox with the drawing tools I use most. The

drawing toolbox is within easy reach of the mouse pointer, the othertwo toolboxes are tucked away in different screen comers.

The path diagram can be printed or passed to another

program via the clipboard. Unfortunately, it cannot be saved as a

separate graphics file. The path diagram can also be used as amodel specification for analysis. This requires the user to write acommand file defining the datafile and special options such asmissing value estimation or bootstraps. There is no built-incommand generator; one has to use an editor (Amos/Draw providesone, but Windows’ Write or any other editor can also be used) andhave the manual ready to look up the details. After the analysis,AmosDraw displays the parameter estimates on the path diagram.Again, the drawing tools are available to change the appearance ofthe picture, for instance to italicize significant parameters. The printquality depends on the printer setup used in Windows, but even ona simple inkjet printer the quality is high.

The original Amos manual was organized as a collection oflecture notes, humorous, and very good at explaining difficultstatistical concepts in nontechnical language. However, this

organization makes it difficult to find procedures and commandswhen you are setting up a command file. The latest version of Amos(3.5) includes a rewritten manual with a separate tutorial andreference part.



p.74 -

EQS

EQS (EQuationS. pronounced ’Ex’) uses the Bentler-Weeksmodel representation (Bentler & Weeks, 1980). This representationdoes not distinguish between the structural and the measurementpart of the model. Eqs allows four kinds of variables: observedVariables (V-variables), latent Factors (F-variables). Error factorsthat generate the residuals in observed variables (E-variables), andDisturbances that generate the residuals in the latent factors (D-variables). Relationships between variables can be in the form ofregression coefficients or covariances. In this representation, errorand disturbance variables are simply independent latent variables.This makes it possible to specify nonstandard models, such asmodels with effects of error variables on various other variables.(Note that although the Amos manual does not state precisely whatmodel it implements, Amos apparently uses a similar

representation). Eqs uses an equation-oriented command language.Thus, a model with a latent factor Fl=’Alienation’ measured by theobserved variable V 1=’Anomia’ would be specified by the equation:

Labels can be used to name the V- and F-variables in themodel. Eqs can handle multigroup models, (in)equality and linearconstraints, and models with means and intercepts. Eqsincorporates seven estimation methods: Maximum Likelihood (ML),Generalized Least Squares (GLS). Unweighted Least Squares (ULS.which Eqs calls Least squares (LS)); Elliptical variants of all of theabove, and Asymptotically Distribution Free estimation (ADF, whichEqs calls Arbitrary distribution GLS (AGLS)). Eqs can handle ordinalvariables using polychoric correlations and methods described byPoon and Lee (1987) and Lee, Poon and Bentler (1992). The programcomputes various goodness-of-fit indices, including the Satorra-Bentler robust chi-square (Bentler & Wu, 1993) and robust standarderrors for all estimation methods except ADF. Eqs has built-infacilities for bootstrapping and simulation; the bootstrap parameterestimates are written to a file to be analyzed separately. Specialcommands to control the analysis or to change the default values oftechnical parameters.

Eqs/Windows consists of two main parts. The first part (theprogram DMAS) is the graphical user interface (GUI), which acts asa Windows-based preprocessor for Eqs. When an Eqs analysis isrequested, the Windows program passes the command file to

Eqs/Dos, and the results are passed back to Windows. The

graphical user interface is splendid. It includes a spreadsheet for at UNIVERSITEIT UTRECHT on February 1, 2010 http://bms.sagepub.comDownloaded from


p. 75

inputting data (which can also read ASCII, BMDP, Lotus 123 anddBase files), and has many data analysis options. The graphicalpower of Windows is used efficiently in a variety of visual displays(among which histograms with normal overlays, pie charts, boxplots, scatter plots, quantile plots, normal probability plots, and arotating 3D plot). All displays can be customized with labels and titlebars and printed. Selected cases (such as suspected outliers) can bemarked and automatically excluded from the analyses. There is achoice of several methods to deal with missing values. Again theWindows graphical environment is used to good purpose byproviding displays that show the pattern of missing values byvariables or subjects. The analysis options include descriptivestatistics, t-tests, crosstabulation, Anova, regression analysis, andexplorative factor analysis.

The Eqs model is specified with a command generator calledBuild_Eqs. Build_Eqs presents a series of dialog boxes, andtransfers the results to a file. Most dialog boxes contain simplequestion-answer sequences that establish specifications like thenumber of variables and factors: some (such as Create New

Equations) use a graphical display and mouse clicks to selectvariables and relations from a list. After the command file is

completed, it can either be run directly or examined in an editor.The latest version of Eqs (version 5, at the time of writing in beta-review) contains a program called the Diagrammer. which allowsusers to specify a CSA model by drawing its path diagram. This isthen used as input for the Eqs program. After computations by Eqs,the diagrammer includes the estimates, and the results can beprinted as a publication-quality print.

The standard graphical output from Eqs is somewhat

disappointing. Graphical output is produced by stipulating the

layout of the path diagram in the command file. The layoutcommands are non-obvious: careful examination of the appropriatesection in the Eqs manual is needed. The output is a PostScript fileproduced by Eqs/Dos. The only way to change the appearance of thepath diagram is to change the command file and resubmit it to Eqs.

The Eqs/Windows manual is very good. It consists of twobooks: one describing Eqs proper, and one describing the Windowsinterface and some new Eqs features introduced with Eqs/Windows.The Eqs manual contains a thorough introduction to the Bentler-Weeks model. The writing style is terse, packed with information,with many references to the specialized literature. The Eqs/Windowsmanual is written as a user’s guide, that guides the analyst throughall available procedures. There is also a lot of information availablein the Windows Help system.



p.76

LISREL

Lisrel (LInear Structural RELations) models consist of ameasurement model, which links the observed variables to thelatent factors, and a structural model, which defines the relationsbetween the latent factors. The original Lisrel model makes a sharpdistinction between exogenous and endogenous variables, but inLisrel 8 this distinction is diminished by introducing a new matrix,appropriately called theta delta-epsilon, to model correlated errorsbetween exogenous and endogenous variables. Lisrel 8 even has aprovision to model ’general covariance structures,’ which allowsestimation of models that cannot be specified as conventional Lisrelmodels.

Lisrel has a matrix-oriented command language. Relations arespecified by indicating that specific elements in specific parametermatrices must be estimated. The parameter matrices have Greeknames, for instance Gamma for the path coefficients from latentexogenous to latent endogenous factors. Lisrel 8 also contains a

simplified command language called Simplis, which is equation-oriented and for novices much easier to use. Thus, a model with alatent factor ’Alienation’ measured by the observed variable ’Anomia’would be specified by:

Anomia = Alienation

Simplis is very flexible. For example, to specify correlatederrors one could use either ’Let the errors of Anomia67 andAnomia7l correlate’ or ’Set the error covariance between Anomia67and Anomia7l free’. Simplis makes a number of assumptions aboutthe model which let the user specify standard models with aminimal number of commands. There are a few specialized Lisrelcommands, such as nonlinear constraints, that do not have a

Simplis counterpart. Also, I have found that Simplis is confused bycomplex models, such as multigroup problems with differentnumbers of variables in each group (the kind of setup used in

estimating models with systematically missing values and multilevelpath models). Simplis attempts to translate the Simplis model inLisrel terms, and fails. Fortunately, the failure is spectacular, andeven inexperienced users can see that there is something very wrongabout the model.

Lisrel contains seven estimation methods: InstrumentalVariables (IV), Two-Stage Least Squares (TSLS), MaximumLikelihood (ML), Unweighted Least Squares (ULS), Generalized LeastSquares (GLS). Asymptotically Distribution Free (ADF. which Lisrel



p. 77

calls Generally Weighted Least Squares (WLS)). and DiagonallyWeighted Least Squares (DWLS). Lisrel handles ordinal variables

using polychoric correlations and GLS estimation. For this, the

program comes with PRELIS, a program that is used for multivariatedata screening, missing value imputation, and calculation of

polychoric correlations with their asymptotic covariance matrix.

Lisrel computes a large number of goodness-of-fit indices.Bootstrapping is done by letting Prelis compute covariance matricesfor a specified number of bootstrap samples, followed by Lisrel

analyses of all covariance matrices in one run. Lisrel can write

parameter estimates and other information on a file to be analyzedseparately. Monte Carlo simulation works in a similar way. An

interesting new option in Prelis is the ability to estimate the

asymptotic covariance matrix by bootstrapping, which makes it

possible to use WLS and DWLS with rank correlations.

Lisrel for Windows feels a bit old-fashioned. The programstarts by asking the user to edit a (new or existing) command file.There is no special help available. Next, the program presents achoice between running Prelis or Lisrel. When that is done, two newWindows buttons have appeared on the screen: ’Show OLJTPLJT file’and ’Path Diagram’. In the path diagram, the mouse can be used toindicate arrows to be deleted and paths to be added. When a path isadded, Lisrel produces an approximate estimate of the parametervalue: proper estimates are obtained by clicking on ’Re-estimate.’There are no interactive options to change the layout of the pathdiagram, but Lisrel generally manages to produce an aestheticallypleasing diagram. It is possible to save the path diagram as agraphic (BMP) file. There is no Windows Help system. In short, theWindows implementation of Lisrel acts as a convenient shell to runPrelis and Lisrel, but the graphical interface is not much more

powerful than the HALO graphics drivers included in the Dosversions.

Lisrel 8 comes with a new manual that describes the new

Simplis command language. For the Lisrel language users mustobtain the older Lisrel 7 manual. There are short reference guidesfor Lisrel 8 and Prelis 2 that document the new features. The Prelis2 reference guide is usable, but personally I am glad that I still havethe original Prelis 1 manual, which gives more information andexamples. The Lisrel 8 version I used (8.02) contained a number ofannoying bugs: Scientific Software Inc. has promised to rectify thisby shipping a free upgrade to all registered Lisrel 8 users.



p. 78

COMPARISONS

Fit Indices

A problem with the chi-square test in covariance structureanalysis is that its power depends on the sample size. In largesamples models may be rejected because of minor misspecifications,and in small samples models with major misspecifications may beaccepted. Many indices have been proposed that evaluate the degreeof fit of a model. For a description and critical discussion of theseand other indices I refer to Bollen (1989) and Gerbing and Anderson(1992). All three programs used here offer a large number of fitindices. For a list of the indices offered by Amos 3.1, Eqs 4.02 andLisrel 8.03 see Hox (1995). Newer versions of these programs haveadded even more fit indices. One should realize that it is not clearwhich indices are superior. Some of these indices have beencriticized because their value depends on the sample size, or

because they appear to be sensitive only to large misspecifications.Gerbing and Anderson (1992) recommend using as a stand-aloneindex McDonalds NCI (available in Eqs as MFI), and as incrementalfit index Bollen’s delta 2 (available in Amos as delta-2, in Eqs andLisrel as IFI). Other indices recommended by Gerbing and Andersonare only found in one or two of the three programs used here.However, all programs produce sufficient information to allowresearchers to compute those indices that are not yet offered

automatically, such as the various parsimony-corrected fit indicesproposed by Mulaik et aL (1989).

Categorical Variables

’

Amos has no provision for categorical (ordinal) variables.Lisrel 8 uses Prelis 2 to handle categorical data. Prelis uses the

marginal univariate distribution of the observed categorical variablesto estimate thresholds for the underlying latent normal variable.These thresholds are then used to estimate polychoric correlationsand the associated asymptotic weight matrix (Jreskog & Srbom,1988). Prelis 1 (bundled with Lisrel 7) used to estimate the

asymptotic covariance matrix under unrealistic assumptions, a

situation corrected in Prelis 2. The polychoric correlation matrix andthe associated weight matrix are input into Lisrel for furtheranalysis by Weighted Least Squares.

Eqs estimates the thresholds and polychoric correlations

simultaneously for each pair of variables, and the corresponding at UNIVERSITEIT UTRECHT on February 1, 2010 http://bms.sagepub.comDownloaded from


p. 79

asymptotic covariance matrix is estimated from the informationmatrix. The correlation matrix is analyzed by GLS, modified to

guarantee that the entries on the diagonal of the estimatedcovariance matrix equal one (Bentler & Wu, 1993, p 165) . Hox (1995)presents a small simulation study which shows that in practice theLisrel and Eqs estimates are very close. Since the Eqs approach isvery time consuming, it is not practical with more than 10

categorical variables.

Bootstrapping

Bootstrapping (Efron, 1982) is a method for estimatingapproximate standard errors that do not rely on the distributionalassumptions required for the asymptotic standard errors offered bythe maximum likelihood and other estimation procedures.Bootstrapping uses repeated sampling with replacement from theactual sample obtained, and uses the variability of the parameterestimates of the bootstrap samples to estimate empirical standarderrors. Compared to asymptotic results, bootstrapping has twoadvantages: it automatically encompasses nonnormal distributions,and it can be used to generate standard errors for statistics forwhich no asymptotic standard error is available, such as the

multiple correlations and fit-indices in CSA. The disadvantage ofbootstrapping is that it requires a fairly large memory, and needslong computations.

Amos, Eqs and Lisrel all include facilities for bootstrapping.Amos offers the most complete facilities: bootstrapped standarderrors and confidence intervals are automatically displayed next tothe asymptotic results, and it includes bias-correction (Stine, 1989)and Bollen and Stine’s bootstrap for goodness-of-fit (Bollen & Stine,1992). Eqs and Lisrel both write the bootstrap estimates on a file,and the analyst must use a separate program to compute empiricalstandard errors and employ the necessary corrections.

ANALYSIS OF MULTITRAIT-MULTIMETHOD DATA

Campbell and Fiske (1959) proposed to judge construct

validity by examining both convergent and divergent validity. Thus.measures must correlate highly with other measures that relate tothe same theoretical construct, and negligibly with those that relateto a different construct. To determine convergent and discriminantvalidity, Campbell and Fiske proposed to combine measures ofdifferent traits and by different methods in one study, and to



p.80 -

examine the resulting correlations arranged in a multitraitmultimethod matrix (MTMM). The assessment of convergent anddiscriminant validity proceeds by comparing three blocks ofcorrelations: correlations between different methods measuring thesame trait (convergent validity), correlations between different traitsmeasured by the same method (divergent validity), and differenttraits measured by different methods. Later, confirmatory factor

analysis using CSA has become the preferred method to analyzemultitrait multimethod matrices (Widaman, 1985).

The data used here are from a study on well-being (Hox,1986). The MTMM contains five traits: global happiness, globalsatisfaction, and satisfaction about income, housing and health.There are also five methods: five question types denoted as

standard, social comparison, ladder, faces, and circles (cf. Andrews& Withey, 1976). Thus there are 5x5=25 variables in the MTMM. Thesample size is 501, after removal of missing values the effectivesample size is 473. An interesting feature of these MTMM data isthat they show positive manifold: all covariances in the covariancematrix are positive. Thus, one criteria for a plausible model is thatall estimates must be positive as well.

The standard MTMM factor model has a separate factor foreach trait and for each method. In our case there are five trait andfive method factors. The restrictions on the factor matrix are thateach variable has only two loadings: one on the corresponding traitfactor and one on the corresponding method factor. The covariancesbetween the trait factors and the method factors are zero, the othercovariances are free.

Earlier analyses on this data set using a mainframe version ofLisrel 3 (Hox, 1986) have shown that factor solutions for this MTMMconverged slowly, and that it is impossible to find a model that isnot rejected by the statistical test within the constraints imposed bythe MTMM model. An analysis using Maximum Likelihoodestimation with Amos. Eqs and Lisrel confirms this conclusion: allprograms produce the same results: the model is rejected with a chi-square of 450 (df--230, p=0.00).

One conclusion could be that the basic MTMM model is

indeed a bad model for these data: it could be that a multiplicativemodel is better. On the other hand, there are some reasons thatexplain why the basic MTMM model is rejected by the chi-squaretest. First of all, the sample size is large enough that minor

discrepancies between the model and the data will lead to statisticalrejection. A fit index other than the chi-square test might indicatethat the model actually describes the data rather well. Indeed, thereis some evidence for this argument: the goodness of fit index GFI is



p. 81

0.93, and the incremental fit index IFI which Gerbing and Andersonprefer is 0.98. Secondly, the statistical test assumes continuousdata and multivariate normality. In this data set, the questions hadseven-point answer scales, and most univariate distributions exhibitconsiderable skewness. The probable result of such violations is achi-square that is biased upwards, i.e. which too often leads to a

rejection of the model (cf. Boomsma, 1983).

Modem CSA software offers a number of options to addressthese problems. One option that seems attractive is to estimate

polychoric correlations for the categorical variables. Polychoriccorrelations estimate the correlations between latent variables thatare measured by ordinal categorical variables, such as the seven-point scales used in our items. For this approach, only Usrel isuseful: the Eqs approach is too time-consuming with 25 categoricalvariables. However, when the polychoric correlations are estimatedusing Prelis, the program tells us that the hypothesis of underlyingnormality is rejected for almost all variable pairs. Thus, the data donot conform to a central assumption behind polychoric correlations.Indeed, if we estimate the basic MTMM model on these polychoriccorrelations, we obtain a chi-square of 1075, which is a worse fitthan the ordinary estimates. Furthermore, we find one residualvariance which is negative, and a negative correlation between somefactors, which is unreasonable since the raw correlations betweenall variables are positive. Clearly, polychoric correlations are not anattractive solution in our case.

An second approach towards modeling this problematic dataset is to choose a different fit function, one that leans less heavily onthe assumption of multivariate normality. All three programs offerGeneralized Least Squares (GLS) and Asymptotically DistributionFree (ADF) estimation. GLS estimates do not assume multivariatenormality, instead, they assume that all observed variables comefrom the same population distribution. ADF makes no assumptionabout the population distribution of the variables, but requires largesample sizes. Eqs offers two other variants: elliptical distributionsand a robust estimate of the chi-square. Elliptical estimation

assumes that the variables have a symmetric distribution with

longer tails than the normal distribution. The robust chi-squareproduces a test-statistic that follows the theoretical chi-squaredistribution more closely than the usual estimate, especially whenthe distributional assumptions are not fully met. Since the mainproblem of the data is skewness, the elliptical estimators seem lessappropriate for these data. Although the skewness is not equal forall variables, GLS estimation appears attractive. With Eqs, GLSestimation can be combined with the robust chi-square statistic.ADF estimation also appears attractive. A disadvantage of ADF is



p. 82

that the sample size of 473 is rather small for ADF estimation, andthe robust chi-square is not available for ADF.

Table 1 below presents a number of fit indices with standardML, GLS, robust GLS and ADF estimation of the basic MTMMmodel.

Table 1. Fit indices for the MTMM model for differentestimation methods

Note: the basic MTMM model has 230 degrees of freedom

An inspection of the fit indices shows that a model with GLSestimation appears the most adequate for these data. All fit indicesare the highest for GLS estimation: the GFI and AGFI indicesintroduced in Lisrel 4 are above the minimum value of 0.90: and theNormed and Non Normed Fit Indices NFI and NNFI proposed byBentler and Bonett (1980) equal 1.00, indicating almost perfect fit.The Incremental Fit Index IFI, preferred by Gerbing and Anderson(1992), is also equal to one with GLS estimation. The statistical chi-square test still rejects the model for all estimation methods, but therobust chi-square suggest that this may at least partly be the resultof the violation of the assumptions behind the usual asymptotic chi-square test. ADF estimation does not work very well with this dataset. Not only are the fit indices low, there are also negativecorrelations between some factors, a conclusion not supported bythe covariances between the observed variables.

A third approach to testing the fit of the MTMM model is touse bootstrap methods. Bollen and Stine (1992) have proposed anadjusted bootstrap for goodness-of-fit measures. In this approach,the chi-square obtained in the data is compared, not with thetheoretical chi-square distribution, but with an empirical chi-squaredistribution obtained by drawing a large number of bootstrapsamples from the original sample and repeating the modelestimation on all these bootstrap samples. Since the bootstrapsamples use the actual sample as a representation of the



p. 83

population, the bootstrap method includes all distributionalpeculiarities in the estimation of both the actual chi-square and theempirical reference distribution. Since the robust chi-square,another method to deal with violation of distributional assumptions,makes a rather large difference (cf. Table 1), it makes sense also touse bootstrapped instead of asymptotic standard errors (Stine,1989). Since Amos has the simplest approach to bootstrapping andincludes all refinements suggested by Stine (1989) and by Bollenand Stine ( 1992), this program is used to produce all bootstrapresults. If Eqs or Lisrel were used, all refmements to produceoptimal estimates would have to be made by hand.

The bootstrapped chi-square distribution (GLS estimation.500 bootstrap samples) estimates the p-value at 0.02, which is closeto the p-value produced by the robust chi-square in Eqs. Table 2below presents the GLS estimates of the standardized factor

loadings, with both asymptotic and bootstrap (200 bootstrapsamples) standard errors.

Table 2. Factor loadings and (asymptotic/bootstrap)standard errors (GLS).



p. 84 -

(decimal point omitted)

A comparison of the asymptotic and bootstrap standard errorsin Table 2 shows that the asymptotic standard errors underestimatethe variability of the estimates. The estimated factor loadings andbootstrap standard errors show that factor I (general happiness) andfactor II (general satisfaction) are not measured too well. The specificwell-being factors III (income), IV (health) and V (housing) are

measured with high accuracy. The fourth measure (happy/unhappyfaces) has the least amount of specific measurement variance. Thecorrelation between the trait factors is generally low, with the

exception of general happiness and general satisfaction: thesecorrelate 0.83 (asymptotic s.e.: .03: bootstrap s.e.: .13).

CONCLUSION

The analyses of the multitrait multimethod data show that thealmost automatic choice for a ’standard’ method such as maximumlikelihood estimation and the associated asymptotic chi-square andstandard errors does not always lead to the ’best’ estimates. The PC-versions of current CSA software offer powerful alternatives. Givencheap computations, it is quite feasible to examine a number ofapproaches in detail, and choose estimation methods that producethe best fit and the most realistic estimates of the variability of theestimates over replications.

REFERENCES

Andrews, F.M. & Withey S.B. (1976). Social indicators of well-being. New York: Plenum.

Bentler, P.M. (1990). Comparative fit indices in structuralmodels. Psychologica 107, 238-246.



p. 85

Bentler, P.M. & Bonett, D.G. (1980). Significance tests andgoodness of fit in the analysis of covariance structures. PsychologicalBulletin, 107, 238-246.

Bentler, P.M & Weeks, D.G. (1980). Linear structural

equations with latent variables. Psychometrika, 45, 289-308.

Bentler, P.M. & Wu, E.J.C. (1993). Eqs/Windows user’s guide.Los Angeles: BMDP Statistical Software, Inc.

Bollen, K.A. (1989). Structural equations with latent variables.New York: Wiley

Bollen, K.A. & Stine, R.A. (1992). Bootstrapping goodness-of-fit measures in structural equation models. Sociological Methods &

Research, 21, 205-229.

Boomsma, A. (1983). On the robustness of LISREL (maximumlikelihood) against small sample size and non-normality. Groningen:Ph.D. Thesis.

Browne, M.W. (1982). Covariance structures. In D.M. Hawkins(Ed.), Topics in applied multivariate analysis (pp. 72-141).Cambridge, UK: Cambridge University Press.

Browne, M.W. & Cudeck, R. (1989). Single sample cross-

validation indices for covariance structures. Multivariate Behavior

Research, 24, 445-455.

Browne, M.W. & Cudeck, R. (1993). Alternative ways of

assessing model fit. In K.A. Bollen & J.S. Long (Eds.) Testingstructural equation models. Beverly Hills, CA: Sage.

Campbell, D.T. & Fiske, D.W. (1959). Convergent anddiscriminant validation by the multitrait-multimethod matrix.

Psychological Bulletin. 56, 81-105.

Efron, B. (1982). The jackknife, the bootstrap, and other

resampling plans. Philadelphia: SIAM.

Gerbing, D.W. & Anderson, J.C. (1992). Monte Carloevaluations of goodness of fit indices for structural equation models.Sociological Methods & Research, 21, 132-160.

Hoelter, J.W. (1983). The analysis of covariance structures:goodness-of-fit indices. Sociological Methods & Research. 11, 325-344.



p. 86 -

Hox, J.J. (1986). Het gebruik van hulptheorieen bÿoperationaliseren. [Using auxiliary theories for operationalization.](doctoral dissertation). Amsterdam: University of Amsterdam.

Hox, J.J. (1995). AMOS, EQS, and LISREL for Windows: Acomparative review. Structural Equation Modeling, 2, 79-91.

Jreskog, K.G. & Srbom, D. (1988). PRELIS. A program formultivariate data screening and data summarization. Mooresville, IN:Scientific Software Inc.

Jreskog, K.G. & Srbom, D. (1989). Lisrel 7 - A guide to theprogram and applications. Chicago: Scientific Software Inc.

Lee, S.-Y., Poon W.-Y. & Bentler, P.M. (1992). Structural

equation models with continuous and polytomous variables.

Psychometrika, 57, 89-362.

McDonald, R.P. (1989). An index of goodness-of-fit based onnoncentrality. Journal of Classification, 6, 97-103.

Mulaik, S., James, L., Van Alstine, J., Bennett, N., Lind, S., &Stilwell, C. (1989). An evaluation of goodness of fit indices forstructural equation models. Psychological Bulletin, 105, 430-445.

Poon, W.-Y. & Lee, S.-Y. (1987). Maximum likelihoodestimation of multivariate polyserial and polychoric correlationcoefficients. Psychometrika, 52, 409-430.

Stine, R. (1989). An introduction to bootstrap methods:

examples and ideas. Sociological Methods & Research, 18, 243-291.

Tucker, L.R. & Lewis, C. (1973). A reliability coefficient formaximum likelihood factor analysis. Psychometrika, 38, 1-10.

Widaman, K.F. (1985). Hierarchically tested covariance

structure models for multitrait-multimethod data. AppliedPsychological Measurement, 9, 1-26.

Software Information

AMOS 3.5. SmallWaters Corp. 1507 East 53rd Str., #452,Chicago, IL 60615, USA; tel 1 312 6678635: email

smallwaters®acm. org



p. 87

EQS 5.0. Multivariate Software, Inc. 4924 Balboa Blvd., #368,Encino, CA 91316, USA: tel 1 818 9060740; fax 1 818 9068206:email [email protected]

Lisrel 8.2. Scientific Software Int., Suite 906, 1525 East 53thStr., Chicago, IL 60615-4530, USA: tel 1 201 6664110: fax 1 2026662394: email [email protected]

In Europe, EQS and LISREL are available from ProGamma,P.O.B. 841, NL-9700 AV Groningen, the Netherlands: tel 31 50636900: fax 31 50 636687; email [email protected]



covariance structure modeling in windows: a …joophox.net/publist/bms95.pdf · model fit, amos...

Documents