This article was downloaded by: [Northeastern University]On: 21 November 2014, At: 11:00Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

Structural Equation Modeling:A Multidisciplinary JournalPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/hsem20

Nested structural equationmodels: Noncentrality andpower of restriction testTenko Raykov a & Spiridon Penev ba Department of Psychology , University ofMelbourne , Parkville, Victoria, 3052, Australia E-mail:b Department of Statistics , University of NewSouth Wales ,Published online: 03 Nov 2009.

To cite this article: Tenko Raykov & Spiridon Penev (1998) Nested structuralequation models: Noncentrality and power of restriction test, StructuralEquation Modeling: A Multidisciplinary Journal, 5:3, 229-246, DOI:10.1080/10705519809540103

To link to this article: http://dx.doi.org/10.1080/10705519809540103

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views ofthe authors, and are not the views of or endorsed by Taylor & Francis.The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor andFrancis shall not be liable for any losses, actions, claims, proceedings,

demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, inrelation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

STRUCTURAL EQUATION MODELING, 5(3), 229-246Copyright © 1998, Lawrence Erlbaum Associates, Inc.

Nested Structural Equation Models:Noncentrality and Power of

Restriction Test

Tenko RaykovDepartment of Psychology

University of Melbourne

Spiridon PenevDepartment of Statistics

University of New South Wales

This article is concerned with the difference in noncentrality parameters of nestedstructural equation models and their utility in evaluating statistical power associatedwith the pertinent restriction test. Based on the seminal work by Browne and Du Toit(1992), Steiger (1989, 1990), Steiger and Lind (1980), and Steiger, Shapiro, andBrowne (1985), asymptotic confidence intervals for that difference are discussed.The intervals represent a useful adjunct to widely employed goodness-of-fit indexesand test statistics when assessing plausibility of constraints in nested models. The ap-proach also permits estimating power of the test of validity of the nesting restrictions(cf. MacCallum, Browne, & Sugawara, 1996). It is illustrated on data from a 2-groupcognitive training study.

Applications of the popular structural equation modeling (SEM) in the social, behav-ioral, and educational sciences typically require assessment of model fit. Recentyears have witnessed progressing interest in goodness-of-fit (GOF) indexes thatevaluate fit either (a) in relation to another model, frequently called a baseline or nullmodel (but other models can also be used for such comparative purposes, e.g., Sobel& Bohrnstedt, 1985); or (b) in the absolute metric of real numbers (e.g., Bentler,

Requests for reprints should be sent to Tenko Raykov, Department of Psychology, University ofMelbourne, Parkville, Victoria 3052, Australia. E-mail: raykov@psych.unimelb.edu.au

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

2 3 0 RAYKOV AND PENEV

1990; McDonald, 1989). In a seminal article, Steiger and Lind (1980) introducedthe approach of confidence interval construction for model fit indexes (see alsoSteiger, 1989,1990). Such intervals can be obtained using, for example, the struc-tural modeling programs EzPATH (Steiger, 1989), LISREL (Jöreskog & Sörbom,1993), RAMONA (Browne & Mels, 1994), or the module FITMOD (Browne,1991). The underlying interval construction idea that is basic to this article has athreefold goal: (a) to provide a didactic exposition of a considerable part of recentresearch into model evaluation via noncentrality parameter assessment; (b) to com-bine and extend ideas from Browne and Du Toit (1992), MacCalhim, Browne, andSugawara (1996), and Steiger, Shapiro, and Browne (1985) in the context of a fre-quently pursued aim in empirical research, namely assessment of plausibility of im-posed constraints in nested structural equation models; and (c) to demonstrate theutility of the noncentrality parameter difference for evaluating power associatedwith the test of the constraint(s) introduced in the more restrictive model.

NOTATION AND PRELIMINARIES

This article follows notational conventions used in the SEM literature (e.g., Bollen,1989). Accordingly, let 0 denote the vector containing all s parameters of a givenstructural equation model M and 0 be the associated parameter space representinga proper subset ofR* (the set of all s-tuples of real numbers). The null hypothesisthat has been traditionally tested in SEM is that, for the population covariance ma-trix E of say p observed variables, there exists a 80 in 0 , such that Z = X(0O), whereZ(8) is the covariance matrix implied by the model (Steiger et al., 1985; see alsoBrowne & Cudeck, 1993). When fitting the model, a discrepancy function F(S,Z(8)) is minimized with respect to 9, where 5 is the empirical covariance matrixbased on a sample of N=n +1 subjects (e.g., Browne &Arminger, 1995). The func-tion F(.,.) satisfies the requirements of being nonnegative and twice continuouslydifferentiable in its arguments. Throughout this article, it is assumed that the pa-rameter vector 8 is identified by Z at 0O; that is, E(9O)=2(0) implies 8 = 80 (e.g., Sa-torra & Saris, 1985). A model Mk is called nested in model Mj, which will be de-noted later by M* < Mj, if the parameter space 0* of M* is obtainable from theparameter space ©,- of Mj after a certain number of constraints are imposed on pa-rameters of My. Formally, Mk is referred to as nested in M¡ if there exists a continu-ously differentiable (r x 1) vector-valued function h = h(Q), such that (a) its (s x r)Jacobian matrix L(0) = 3ft/38 has full column rank (at least in a neighborhood of 0O)and (b) 0* = {0 e 0,1 A(0)=0.}, where Q. is an (r x 1 ) vector consisting of zeroes only(e.g., Satorra & Saris, 1985). For simplicity of reference, the restrictions A(0) = Q_will often be called nesting restrictions (nesting constraints).

The next proposition (see Steiger et al., 1985, Theorem 1, Statement [ii], pp.256-259, where its proof is also provided) plays amajorrole in the discussion of as-

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

NESTED MODELS AND NONCENTRALITY 2 3 1

ymptotic confidence intervals in the subsequent sections and is therefore restatedhere.

Proposition 1

Under regularity conditions specified in detail in Steiger et al. (1985, pp. 255-256)and assumed in the rest of this article (practically, they amount to normality and up tomild lack of model fit), the test statistics Tk = nFk and Tk - T¡ = nFk - nF¡ are mutuallyasymptotically independent, where Fk and F¡ are the attained minimums of the dis-crepancy function F when models Mk and M¡ (Mk < Mj) are fitted to the covariancematrix S. The statistics Tk and 7} have asymptotically noncentral chi-square distribu-tions with dk and dk - d¡ degrees of freedom, respectively, and corresponding noncen-trality parameters 5* and 8* - 5,, where 8* and 5, are the large-sample limits of n timesthe minimum values of the discrepancy function for Mk and Mj, respectively.

ASYMPTOTIC CONFIDENCE INTERVALS FOR THEDIFFERENCE IN NONCENTRALITY PARAMETERS OF

TWO NESTED STRUCTURAL EQUATION MODELS

This section focuses on asymptotic confidence intervals for (a) the population dif-ference Sy=8* - 5,- in the noncentrality parameters 8* and 8/ of two nested models Mk

< Mj and (b) its normed version. For simplicity of reference, 5y is called the noncen-trality difference parameter (NDP).

Asymptotic Confidence Intervals

Let C(x\d,8) denote the cumulative distribution function of the noncentral chi-square distribution with ¿/degrees of freedom and a noncentrality parameter 8 (e.g.,Johnson & Kotz, 1970, chap. 28). Given the sample difference x = Tk - 7} in the teststatistics associated with Mk and Mj and the validity of Proposition 1, as lower andupper limits of an asymptotic, say 90%, confidence interval of 8*,-, following, for ex-ample, Browne and Mels (1994, p. 143), the solutions / and u for the parameter 8 inEquations 1 and 2, respectively, can be taken:

C(x\d,&) = .95 (1)C(x\d,8) = .05 (2)

To eliminate the direct dependence of the noncentrality parameter on samplesize, the normed NDP 8*/« = 8*/« - 5/n for two nested models Mk < Mj can be con-

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

2 3 2 RAYKOV AND PENEV

sidered (cf. McDonald & Marsh, 1990). For simplicity of reference, 8*/« is calledthe normed noncentrality difference parameter (NNDP). Because the NDP is obvi-ously estimable by 8*; = 8* - 5j, where 8* and 8 ̂ are estimates of the noncentralityparameters of Mt and Mj, respectively (e.g., McDonald & Marsh, 1990, p. 248), theNNDP is estimable by (8*// n) = 8* / n - 8/ / n} An asymptotic 90% confidence in-terval for the NNDP, Skjn, is given by the interval {Un, u/ri), whereby (/, u) is the as-ymptotic 90% confidence interval for NDP. The NNDP is closely related to the so-called Root Deterioration per Restriction Index (RDR) developed by Browne andDu Toit (1992, p. 284). Under the regularity conditions of Theorem 1 in Steiger etal. (1985; see Proposition 1 mentioned earlier), to approximate the NNDP, one cansquare the RDR and multiply the result by the difference in degrees of freedom, dk-d], of the models A/* < Mj.

The left and right endpoints of the asymptotic confidence interval for NDP, thatis, the solutions / and u for 8 of Equations 1 and 2, as well as the limits of the asymp-totic confidence interval for its normed version, NNDP, can be practically deter-mined as follows: The confidence interval for the RDR can be obtained using themodule FITMOD (Browne, 1991). Use of the previously indicated straightforwardalgebra on each of the confidence limits of RDR yields these for the NNDP, andtheir multiplication by n yields the confidence limits of the corresponding asymp-totic confidence interval for NDP. With no access to FITMOD, however, to obtainthe confidence limits of NDP and, from them, those of NNDP via division by n, aFORTRAN program utilizing a Wiener germ approximation to the noncentral chi-square distribution function (Dinges, 1989) and its quantiles may be applied, whichuses some NAG (1993) subroutines (Penev & Raykov, 1998). (These can be ob-tained from us on request.) Alternatively, a SAS program, which is provided forcompleteness in the Appendix, can be utilized for the same purposes.2

Use of Confidence Intervals for Assessing Plausibility ofNesting Restrictions in SEM

Once the previously mentioned asymptotic confidence intervals are worked out,the next question is how to make conclusions about their corresponding parameters

1 Given that for Mk < Mj both δk δ, and dk > dj hold (e.g., Bentler, 1990, p. 240, Equation 8b), δkj 0 fol-lows. If in empirical work, it happens that δ < δ j,δkj = 0 should be taken (cf. McDonald & Marsh, 1990, p.246; see this article's Appendix). Relatedly, if Fok and Foj denote the minimum discrepancy function valueif Mk and Mj, respectively, were fitted to the population covariance matrix Σ, Fok > Foj holds true.

2 As seen from the Appendix, the program is based on repeated calls of the built-in SAS subroutineCINV(.,.,.) for inversion of the noncentral chi-square distribution function. As mentioned in the SASLanguage Guide manual (see SAS Institute, 1991, p. 51 ), a limitation of CINV(.,.) is that it may fail for alarge noncentrality parameter. (This may happen, e.g., with some models with large sample sizes.)

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

NESTED MODELS AND NONCENTRALITY 2 3 3

by evaluating the endpoints of the intervals (and their length). If for two nestedmodels Mk < M¡ the lower limit of the confidence interval for their NDP is close tozero (and, preferably, the interval is not too long), there may be little difference intheir noncentrality parameters. Then, the hypothesis asserting that the additionalconstraints imposed in the more restricted model are meaningful in the populationmay be considered plausible. Conversely, if the lower limit of this interval is appre-ciably larger than zero, the models are likely to differ more than trivially, and hence,the hypothesis of valid restrictions is rejectable. Similarly interpreted is the leftendpoint of that interval for NNDP.

Information about the difference in GOF of two nested models is also obtainedby examining the significance of the difference in their inferential GOF indexes, Tk

- Tj, as well as the significance of its asymptotically equivalent Wald multiplier teststatistic, Lagrange multiplier test statistic, or both. (These are output by EQS;Bentler, 1993.) In case, however, at least the more restricted model Mt is not statis-tically acceptable, the significance or lack of significance of the widely used chi-square values difference Tk - T¡—as well as that of the Wald and Lagrange multi-plier test statistics—may be misleading as a means of evaluation of plausibility ofthe nesting constraints (e.g., Jöreskog & Sörbom, 1988). It is in these cases wherethe asymptotic confidence intervals for NDP and NNDP seem likely to be most use-ful, even though no precise norms of "closeness to zero" implied in the precedingdiscussion are transparent (e.g., McDonald & Marsh, 1990, p. 254). Specifically,these confidence intervals are likely to be most informative in empirical settingswhere both nested models are associated with high chi-square indexes relative totheir degrees of freedom. (For example, this may be due to large samples and exces-sive statistical power.) In such cases, and when the considered models are notgrossly misspecified (see also the regularity conditions underlying Theorem 1 inSteiger et al., 1985, p. 255, or the previously mentioned Proposition 1), it seems notunlikely that plausible nesting constraints may be associated with a significant dif-ference in the chi-square indexes (or significant Wald multiplier test statistics, La-grange multiplier test statistics, or both). This difference, if used to test the null hy-pothesis of valid constraints in the population—as frequently done in appliedresearch when evaluating the significance of 7* — 7} against the central chi-squaredistribution—will be suggesting that the constraints are meaningless in the popula-tion. At the same time, however, the left boundary of the previously mentioned con-fidence interval for the difference in noncentrality parameters—that is, NDP,NNDP, or both—may suggest a different conclusion.

We emphasize that the present asymptotic confidence intervals for NDP andNNDP are useful for examining plausibility of the nesting restrictions per se and, ingeneral, are not informative with respect to plausibility, or lack thereof, of either ofthe nested models.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

2 3 4 RAYKOV AND PENEV

Relation of NDPs to RMSEA

A main consideration when using the confidence intervals in practice is the in-volved decision of "closeness to zero" for their left limits. It is therefore helpful toobtain means of gauging the latter's proximity to zero. For this purpose, a link willbe used between the present NNDP, dk/n, and other GOF indexes for which cutoffshave been proposed. Such an index is the recently developed Root Mean Square Er-ror of Approximation (RMSEA; Steiger & Lind, 1980). Based on a substantialamount of experience, it has been suggested independently by Steiger (1989) andBrowne and Cudeck ( 1993) that an RMSEA less than .05 can be considered indica-tive of close model fit, that is, model fit being reasonably good. Similarly, anRMSEA of .08 and higher suggests mediocre or bad fit. In terms of confidence in-tervals, it has been suggested that, if the lower limit of the 90% confidence intervalfor the RMSEA is less than .05 (and in particular if the interval is not very wide), thefit of the model may be considered reasonable for practical purposes (e.g., Browne& Cudeck, 1993, pp. 146-147; Steiger, 1989, pp. 77-81).

The link between the NNDP, 5*/«, and the RMSEA index (and consequently be-tween NDP and RMSEA) is established using the RDR. For two nested models M*< Mj, RDR can be defined as follows (cf. Browne & Du Toit, 1992, p. 284):

(3)

with •/(. ) denoting the positive square root. Because for Ait < M¡, both 8* > 6,- and dk> dj hold (see footnote 1), RDR is well defined in the population and can be esti-mated by the following:

Using this link, an asymptotic 90% confidence interval for RDR is obtained as(•>](!/ n)/(dk-dj); ^(u I n) I {dk - dj)) where / and u are the lower and upper con-fidence limits of NDP (see Browne, 1991, p. 19; Browne & Du Toit, 1992, p. 284).Designating by Fok and Foj the minimum discrepancy function value if M* and M¡,respectively, were fitted to the population covariance matrix E, under the regularityconditions of Theorem 1 of Steiger et al. (1985), from Equation 4 one obtains theoriginal definition for ykj (Browne & Du Toit, 1992, p. 284):

dj) (3')

The chain of Equations 4,3, and 3' was presented here in order to illuminate thefact that, if throughout the previously presented reasoning one substitutes the satu-

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

NESTED MODELS AND NONCENTRALITY 2 3 5

rated model M, for the model M¡, because ôs = d, = 0, one obtains from Equation 3the single-model RMSEA (for its definition, see Browne & Cudeck, 1993, pp.142-145). Hence, capitalizing on the complete analogy between (a) the definitionof the RDR index y« and its point and interval estimates and (b) the definition of thesingle-model RMSEA index and its estimates, one could conjecture that thesame cutoff of .05 may as well be used with the estimates of RDR, as in Browne(1991, p. 19).

This conjecture now provides the earlier required means of gauging proximity tozero of the left endpoints of the confidence intervals for the NDP and NNDP. Thatis, given two nested models Mk < M¡ (and the validity of all previously discussed as-sumptions underlying the construction of the asymptotic confidence interval forNDP and its normed version), at the 90% confidence level, one may consider thenesting constraints imposed in Mb h(Q) = 0, plausible if the left endpoint of the 90%confidence interval for NDP (NNDP) is so close to zero that the left endpoint of thesame-level confidence interval for ykj is less than .05. Otherwise, the credibility ofthese constraints in the population may be questioned.

The SAS program in the Appendix of this article outputs the limits of the asymp-totic 90% confidence interval for the RDR, y#. Such intervals at other confidencelevels for NDP, NNDP, and RDR are obtainable from the program by changing thecorresponding probability cutoff (e.g., .975 instead of .95 and .025 instead of .05 inits 23rd and 34th lines, if 95% confidence intervals are required) for the inverse ofthe cumulative distribution function of the same chi-square distribution,CINV(.,DDF,NC). (See SAS Institute, 1991.)

Quantitative Rationale of the NDPs

This subsection establishes the quantitative rationale underlying the defini-tion of the NDPô*y, and consequently of its normed version 5#/n and the RDRindex y*;. To this end, consider again testing the null hypothesisHo: Bo e 0* = {9 e &j\ h(B) = 0} against the alternative Hi: 0 6 ©j. This null hypothe-sis asserts the validity of the nesting restrictions through which model Mk is ob-tained from model M¡. The likelihood ratio test rejects Ho if

} = n-Tj (5)

exceeds a certain critical point ca (for a given significance level a). In Equation 5,

F(S, 1(0)) = log| Z(9)| + tr {S(I(9)) "'} - log| S\-p (6)

is the commonly used maximum likelihood fit function, 0* is the vector minimizingF(S, E(6)) over 0*, and 9/ is the vector minimizing F(S, 2(9)) over 0/. As is well

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

2 3 6 RAYKOV AND PENEV

known (e.g., Satorra & Saris, 1985), if Ho is true, the test statistic 7*- 7} has asymp-totically a central chi-square distribution with r degrees of freedom. For local alter-natives, however, that is, for a sequence of parameter values {Q0,m}mn outside, yetnear, 0* (for a precise definition, see Steigeret al., 1985, p. 255), the asymptotic dis-tribution of Tk - Tj is noncentral chi-square with r degrees of freedom and the fol-lowing noncentrality parameter (see Satorra & Saris, 1985, p. 85):

X = q'(L'B-lL)-lq (7)

In Equation 7, B is (1/n) times the Fisher information matrix—the expectation of.5d2F(S, Z(0))/ae 39'—evaluated at 9» L = L(0o), and q is the limit of[m"2A(Goln)] as m increases unboundedly; thereby the sequence {Qo,m]mi>\ is as-sumed to converge to 0O in such a way that the limit of [minh(Oo,m)] exists and isequal to q.

Thus, because of Proposition 1, Equation 7 suggests that the NDP 8*, for twonested models Mt and M¡ (M* < M¡) can be approximated by a weighted sum ofsquared discrepancies. The discrepancies can be thought of as limits of the restric-tion values at local alternatives. The weights are the corresponding elements of thenormed Fisher information matrix (evaluated at the true parameter) premultipliedand postmultiplied by the Jacobian of the restrictions (also evaluated at the true pa-rameter).

Relation Between RDR and the RMSEA Indexesof Nested Models

The RDR index, and consequently the difference in noncentrality parameter and itsnormed version, is related to the difference in RMSEA indexes associated with thenested models M* and M-¡ (Mt < MJ). To explore this interrelation, let Y* and Y; denotetheir RMSEA indexes, respectively. Thus, y* = •JF^kldk and Y/ = TJFOJIdj (e.g.,Browne & Cudeck, 1993; given this definition of the single-model RMSEA index,the earlier Equations 3 and 3' suggest that the RDR index Yy might be viewed as an"RMSEA index for two nested models"). Obviously, if My is saturated y, = Fo¡ = d¡ =0, and thus Y*, = y* = y* - 0 = y* - y, (see Equation 3')- Similarly, if Fok = Foj, that is, ifthe population fit of Mk and AÍ, is identical, y*,-=y* - "ft=0. That is, the RDR index y*,equals the difference between the single-model RMSEA indexes y* and y¡ when (a)the less restrictive model is saturated or (b) both models are associated with identi-cal population fit.

However, if neither of these two cases hold, and for models with small misspeci-fications, large samples, and correctly chosen fit function (e.g., Browne & Mels,1994, p. 86), the following lower bound for y*y is obtained (note that then Fok > Fo¡, dk

> dj, and yt > y, hold):

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

NESTED MODELS AND NONCENTRALITY 2 3 7

H) (8)

That is,

S 2 2 d j ) (9)

Hence, apart from the cases where M¡ is saturated or the population fit of the twomodels is identical—in which event the difference in RMSEA indexes equals theRDR index ytj—the square of the latter is bounded from below by the difference inthe squares of the single-model RMSEA indexes, multiplied by djl(dk - dj). Theamount of discrepancy between the squared RDR index and the difference of thesquared single-model RMSEA indexes depends then on model complexity and thatof the nesting restrictions, as reflected in associated degrees of freedom. This dis-crepancy is multiplicatively represented by the ratio of degrees of freedom of theless restrictive model to those of the nesting constraints.

ESTIMATION OF POWER OF THE TEST OFNESTING RESTRICTIONS IN SEM

In the behavioral, educational, and social sciences, researchers interested in assess-ing substantively meaningful restriction(s) on parameters in initially consideredstructural equation models typically impose the constraint(s) in the latter and sub-sequently consult as a test statistic the difference in chi-square values, Tk - 7}, for apair of nested models M* < Mj. The question that seems to have attracted limited at-tention in practice so far, however, is that of power of the associated statistical testof validity of the constraints in the studied population. This issue is of high empiri-cal importance (e.g., Cohen, 1988), owing to the relevance of evaluating the likeli-hood of sensing invalid parameter restrictions. This section addresses this questionby extending the preceding discussion, utilizing the RDR index, and capitalizing onthe method of MacCallum et al. (1996). According to the approach of MacCallumet al., when the test of close fit (e.g., Browne & Cudeck, 1993) is focused on, twodistributions are compared: One is that of a noncentral chi-square variable with themodel degrees of freedom and a noncentrality parameter corresponding to anRMSEA of .05 (indicating close fit); the other is the chi-square distribution with the

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

238 RAYKOV AND PENEV

same degrees of freedom but with a noncentrality parameter corresponding to anRMSEA of .08 (indicating mediocre fit).

In this context of comparing fit of two nested models, in particular when powerof the nesting restriction test is of concern, because of the validity of Proposition 1,one can directly utilize the idea of the MacCallum et al. (1996) method. This possi-bility is given by the previously presented complete analogy of the definition of theRDR index, ykJ, and the single-model RMSEA that can be obtained from ykj whenthe less restrictive model is saturated. Thus, following the aforementioned recom-mendations about interpretation of values of the single-model RMSEA index, onemay assume (see Browne, 1991, p. 19) that values of y*, in excess of .08 may be in-dicative of the nesting restrictions being seriously violated. Similarly, values of y*yless than .05 may be interpreted as suggesting that these constraints are approxi-mately fulfilled. Obviously, ykJ = 0 is the case (only) when the restrictions are per-fectly fulfilled.

Therefore, because of Proposition 1, under the hypothesis y*,- < .05 (or yk¡ = 0,which is currently routinely tested in applied research), the differencex=Tk-T¡will be asymptotically following the chi-square distribution with degrees of free-dom dk - dj and a noncentrality parameter of up to .0025n(dk - dj). (See MacCallumet al., 1996, Equation 8.) Similarly, under the hypothesis ykj > .08, x will be asymp-totically following the same distribution yet with a noncentrality parameter of atleast .0Q64n(dk - dj). Thus, if the nesting restrictions are seriously violated, that is,ykJ > .08 holds true, a lower bound of the likelihood of sensing it with the differencein chi-square values test statistic x by rejecting the hypothesis of the restrictions be-ing approximately fulfilled (i.e., ykJ< .05) can be worked out using the SAS programby MacCallum et al. (see their Appendix). The power associated with testing per-fect validity of the restrictions against the hypothesis of seriously violated restric-tions or approximately fulfilled ones can be similarly estimated. The only changeneeded in the parameters input into that program is the corresponding values of ykj,namely, 0 and .08 or .05, respectively. This method of power evaluation for the testof nesting restrictions can also be used for computing sample size necessary forachieving a prespecified power with this test, by following the counterpart stepswith the approach by MacCallum et al. when used for sample size calculations withregard to the overall model test (see also their SAS program for this purpose).

ILLUSTRATION ON DATA FROM A TWO-GROUP STUDY

Raykov (1995) recently presented results of SEM of data from a longitudinal studyof plasticity in fluid intelligence of older adults by Baltes, Dittmann-Kohli, andKliegl (1986). A main finding was that of identity of the control and experimentalgroups in the latent correlations in Model 2 fitted there. This result was interpreted

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

NESTED MODELS AND NONCENTRALITY 2 3 9

as suggesting group-invariant interrelations among initial ability status andpretest-to-first-posttest fluid ability change as addressed by the analyzed repeatedtests of the fluid intelligence subabilities induction and figurai relations.

In the nesting restriction context of this article, it is of interest to estimate (a) dif-ferences in noncentrality parameters associated with the nested model versions withand without the group-invariance constraint in these latent correlations, as well as (b)statistical power associated with the test of these restrictions. To this end, one needsto focus on versions Vi and V4 of Model 2 in Raykov (1995, Table 4, p. 275). Vt as-sumes only group invariance in initial means, variances, and covariances, whichwere expected not to differ systematically across groups owing to random group as-signment taking place after pretest. Alternatively, V4 imposes the constraint of all la-tent correlations being the same in the experimental and control groups.

First, to examine differences in noncentrality of the two models under consid-eration here, the SAS program provided in the Appendix is used. The numbers en-tered in its initial part are sample size and fit indexes of models V\ and V4 (seeRaykov, 1995, pp. 274-276). With them, the program yields the following esti-mates (90% confidence intervals are provided in parentheses following the pointestimate):

1. NDP: 8*/= 9.02 (.0, 26.490).2. NNDP: &/ / n = .036 (.0, .107).3. RDR: % = .049 (.0, .084).3

These results, in particular the zero left endpoints of the three confidence inter-vals, suggest that the restrictions of equal across-groups latent correlations can beconsidered plausible.

In a second step, the power associated with the restrictions of interest here,namely, group identity in the previously mentioned latent correlations, is evaluatedutilizing the approach discussed in this article. To this end, the program for powerestimation by MacCallum et al. (1996, see their Appendix) is used. Thereby, oneneeds to enter into their program only the difference in degrees of freedom and sam-ple size (apart from the RDR values associated with the null and alternative hy-potheses). The program then yields .90 as an estimate of the probability of sensingserious discrepancies in the latent correlation pattern across the two groups (i.e., jkj

> .08) by rejecting the hypothesis of identity in it (i.e., y*, = 0). Thus, statisticalpower for the test of identity in the correlative pattern among initial ability statusand ability change in the experimental and control groups, as tapped by the used

3 Identical results are obtained for the Root Deterioration per Restriction Index and normed noncen-trality difference parameter with FITMOD (Browne, 1991, pp. 18-19). Direct algebraic rearrangements(see earlier) yield then are identical to the earlier presented estimate and confidence interval limits forthe noncentrality difference parameter.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

2 4 0 RAYKOV AND PENEV

fluid intelligence tests, has been satisfactorily high. Given this and the restrictionplausibility result, considerable trust may be placed in the finding of group identityin the focused latent correlations, which has been reported by Raykov (1995).

DISCUSSION AND CONCLUSION

GOF indexes for structural equation models have received a large amount of atten-tion during the past 15 years. Their estimated values have been used on numerous oc-casions in social, behavioral, and educational studies to argue for or against consid-ered models. The lack of precision in these indexes in applied research, however, isnot apparent when their sample values are cited. Analysts may obtain widely differ-ing values of these indexes with different samples for identical research questionsand settings. It is, therefore, recommendable to use confidence intervals for thoseGOF indexes for which such intervals can be obtained (e.g., Steiger, 1990) becausethe precision of the sample estimates of the indexes becomes clear from the length ofthe corresponding confidence intervals. At the same time, researchers can assess theplausibility of statements about alternative population values of these indexes bychecking if the respective confidence interval covers these values.

During the past few years, noncentrality parameters associated with structuralequation models have been focused on by developers of covariance structure analy-sis (Bentler, 1990; Browne & Cudeck, 1993; Browne & Du Toit, 1992; McDonald,1989; McDonald & Marsh, 1990; Steiger, 1989, 1990). Following their lead, thisarticle was concerned with the difference in noncentrality parameters of two nestedmodels as an indicator of plausibility of the nesting constraints imposed in the morerestrictive model. The confidence intervals discussed in this article combine andextend ideas by Steiger et al. (1985) and by Browne and Du Toit (1992). The inter-vals seem particularly informative in situations where at least one of the models'chi-square indexes is significant. These confidence intervals have asymptotic na-tures. Thus, similar to many GOF indexes used in SEM, they are most trustworthywith large samples. With less than optimal samples, their utility may be considera-bly lessened, which will also be the case with violations of the regularity assump-tions mentioned in Proposition 1 (see Steiger et al., 1985, Theorem 1). These confi-dence intervals remain useful, however, in cases with high power caused by verylarge samples (e.g., Browne & Cudeck, 1993; Steiger, 1989).

The approach of this article is not confined to situations where the more restric-tive model Mk is the model of variable independence, the so-called null model (e.g.,Bentler & Bonett, 1980). Use of this model in empirical research has these limita-tions: (a) Its choice may sometimes be ambiguous (e.g., Browne & Cudeck, 1993);(b) it may be of no interest in a particular situation; (c) the analyst may be interestedin the comparison of particular nested models that may be appreciably more generalthan the null model (e.g., Sobel & Bohrnstedt, 1985); (d) in certain cases, the null

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

NESTED MODELS AND NONCENTRALITY 2 4 1

model may not satisfy the previously mentioned assumption of approximability ofthe chi-square index distribution because of serious misspecifications possiblystemming from the underlying assumption of variable independence; or (e) all ofthese. The discussion in the first section of this article, however, will remain validalso in the case where A/* is the independence model. The following discussion ofconstructing asymptotic confidence intervals will also remain valid as long as theregularity conditions of Proposition 1 are tenable.

Theorem 1 in Steiger et al. (1985), in conjunction with a recently developedmethod by MacCallum et al. (1996) for evaluation of statistical power associatedwith the overall model test, is also useful for estimation of power with the test ofnesting restrictions in structural equation models (as well as, conversely, samplesize determination to achieve given power with the latter test). Given the currentextensive use of the difference in chi-square values test in social, behavioral, andeducational research when testing restrictions imposed in nested models, this arti-cle discussed a way of evaluating its power that seems to be of considerable practi-cal utility. It is thereby hoped that this article may contribute to emphasizing therelevance of evaluating power associated with specific restrictions subsequentlyimposed in an initially considered model, which are associated with multiple de-grees of freedom (cf. Satorra & Saris, 1985).

This approach to estimation of power of the restriction test, as well as that byMacCallum et al. (1996), is not insensitive to specification error because the ap-proach does not account for misspecification(s). As demonstrated by Kaplan andWenger (1993), propagation of such error into various parts of considered struc-tural equation models, as well as statistical power, depends on the pattern of zeroand nonzero elements in the parameter estimates' covariance matrix. It is thereforelikely, in these authors' view, that depending (at least) on the part of the modelwhere a misspecification is present, the magnitude of the error, whether the pa-rameters included in the constraint are affected by that specification error, and, re-latedly, the pattern of zero and nonzero elements in the parameter estimator covari-ance matrix of each of two nested models, estimates of power of the restriction testobtained with this approach are differentially affected (as well as estimates of theoverall model test with the MacCallum et al., 1996, method). More generally, it islikely that statistical power itself (regardless of method of its evaluation) is then dif-ferentially affected.

The normed noncentrality difference and (nonnormed) NDPs discussed herehave a clear intuitive meaning that suggests their utility, in particular that of theirconfidence intervals, in empirical social, behavioral, and educational researchaimed at assessing plausibility of nesting restrictions within structural equationmodels. As is well known, each model's noncentrality parameter is interpretable asan index of "badness of fit" (e.g., Browne & Cudeck, 1993), in the sense of reflect-ing the extent to which the model fails to satisfy the traditionally tested null hy-pothesis of there being a vector in its parameter space with which the model per-

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

2 4 2 RAYKOV AND PENEV

fectly reproduces the population covariance matrix. Thus, the NDP and NNDP inthis article quantify the relative lack of fit of two nested models under considera-tion, which is a direct empirical interpretation. The RDR of Browne and Du Toit(1992; see also Browne, 1991) in addition takes into account the complexity of theconstraints, and by square rooting the ratio of difference in discrepancy functionvalues of the two models to difference in degrees of freedom (see Equation 3').achieves a different metric that nonlinearly depends on that underlying the NDPs.Whereas the latter metric can be considered using squared discrepancy units (oflack of fit between model implied, at solution, and empirical covariance matrices),that of RDR is in terms of discrepancy units (per restriction). A positive feature ofRDR not shared with NDP or NNDP is that RDR accounts for complexity of thenesting restriction and uses a metric in terms of discrepancy, rather than squareddiscrepancy, units. Conversely, in our view, a positive feature of the NDP andNNDP is their direct and intuitive interpretation as indexes of relative lack of fit oftwo nested models, which generally capitalizes to a more pronounced extent on dis-crepancies between reproduced (at solution) and empirical covariance matrices.

This article focused on the difference in noncentrality parameters of nestedmodels, and functions of it, as an index of plausibility of the nesting restrictions. Al-ternatively, another approach might be based on the possibility to relate the twomodels' RMSEA indexes. As Equation 8 and Inequality 9 show, however, the rela-tion between RDR, NNDP, and NDP—for which confidence intervals have beenderived using the chi-square distribution—and the two models' RMSEA indexesseems more complex than what could permit a derivation, with not too much diffi-culties, of a confidence interval for the difference in RMSEA indexes. A conse-quential reason for these complications is the nonlinearity transformation underly-ing the relation between the noncentrality parameter and RMSEA. We doencourage further research into this issue, however, because a confidence intervalfor the difference in two RMSEA indexes, due to the fact that the RMSEA takes themodel complexities into account, is likely to be more informative about validity ofnesting restrictions. Whereas such research would have to deal with the implica-tions of the nonlinearity transformation of the population discrepancy functionvalue into RMSEA, the confidence intervals mentioned here seem to be a usefulchoice that have direct empirical meaning as well as utility in evaluating plausibil-ity of nesting restrictions and power of the associated test, as previously demon-strated.

In conclusion, we wish to emphasize that a main message of this article is the ap-propriateness, meaningfulness, and benefit of shifting attention away from the ac-tual value of the difference in chi-square values' statistic 7* - 7} for two nested mod-els My < Mj and toward the noncentrality parameter associated with this statistic. Indoing so, the article follows Bentler (1990), Browne (1991), Browne and Cudeck(1993), Browne and Du Toit (1992), McDonald (1989), McDonald and Marsh(1990), Steiger ( 1989,1990), and Steiger and Lind ( 1980). In particular, for applied

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

NESTED MODELS AND NONCENTRALITY 2 4 3

research with plausible assumptions underlying the approach discussed here (seeSteiger et al., 1985, pp. 255-256, or Proposition 1), the proposed asymptotic confi-dence intervals appear to be useful adjuncts to widely available relative fit indexesand test statistics of nesting constraints. The intervals are informative with respectto (a) the empirical extent of violation of the pertinent hypothesis in the populationrestrictions; (b) the range of plausible population values of differences in noncen-trality parameters; and (c) statistical power associated with the hypothesis in thefirst point, as judged informally by their length. In addition, estimation of statisticalpower associated with the test of the nesting restrictions is facilitated, as is samplesize determination to achieve given power with this test. In our opinion, power ofthis test should be of greater interest in social, behavioral, and educational research(e.g., see Cohen, 1988).

ACKNOWLEDGMENTS

This research was supported by grants from the Australian Research Council andthe Max Planck Society for Advancement of Science to Tenko Raykov and by agrant from the University of Melbourne to Tenko Raykov and Spiridon Penev.

We are indebted to J. H. Steiger, M. W. Browne, P. M. Bentler, A. Satorra, andR. C. MacCallum for valuable discussions on hypothesis testing and power in co-variance structure modeling and to the editor and four anonymous referees for valu-able comments and criticism on an earlier draft of the article that contributed con-siderably to its improvement. We are grateful to M. W. Browne for permission touse his program FITMOD (Browne, 1991).

REFERENCES

Baltes, P. B., Dittmann-Kohli, F., & Kliegl, R. (1986). Reserve capacity of the elderly in aging-sensitivetasks of fluid intelligence: Replication and extension. Psychology and Aging, 1, 172-177.

Bentler, P. M. (1990). Comparative fit indices in structural model. Psychological Bulletin, 107,238-246.

Bentler, P. M. (1993). EQS: Structural equations program manual Los Angeles: BMDP StatisticalSoftware.

Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in covariance structuremodels. Psychological Bulletin, 88, 588-606.

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.Browne, M. W. (1991). MUTMUM: User's guide. Columbus: Ohio State University, Department of

Psychology.Browne, M. W., & Arminger, G. ( 1995). Specification and estimation of mean- and covariance structure

models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of statistical modeling for thesocial and behavioral sciences (pp. 185-249). New York; Plenum.

Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S.Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

2 4 4 RAYKOV AND PENEV

Browne, M. W., & Du Toit, S. (1992). Automated fitting of nonstandard models. Multivariate Behav-ioral Research, 27, 269-300.

Browne, M. W., & Mels, G. (1994). RAMONA PC: User's guide. Columbus: Ohio State University, De-partment of Psychology.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erl-baum Associates, Inc.

Dinges, H. (1989). Special cases of second order Wiener germ approximations. Probability Theory andRelated Fields, 83, 5-57.

Johnson, N. L., & Kotz, S. (1970). Continuous univariate distributions (Vol. 2). New York: Wiley.Jöreskog, K. G., & Sörbom, D. (1988). LISREL7: A guide to the program and its applications. Moores-

ville, IN: Scientific Software International.Jöreskog, K. G., & Sörbom, D. (1993). LISREL8: User's reference guide. Chicago: Scientific Software

International.Kaplan, D., & Wenger, R. N. ( 1993). Asymptotic independence and separability in covariance structure

models: Implications for specification error, power, and model modification. Multivariate Behav-ioral Research, 28, 467-482.

MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination ofsample size for covariance structure modeling. Psychological Methods, 1, 130-149.

McDonald, R. P. (1989). An index of goodness-of-fit based on noncentrality. Journal of Classification,6, 97-103.

McDonald, R. F., & Marsh, H. W. (1990). Choosing a multivariate model: Noncentrality and goodnessof fit. Psychological Bulletin, 107, 247-255.

NAG. (1993). FORTRAN library manual (Mark 16). Oxford, England: Author.Penev, S., & Raykov, T. (1998). A Wiener germ approximation of the noncentral chi-square distribution

and of its quantiles. Manuscript submitted for publication.Raykov, T. (1995). Multivariate structural modeling of plasticity in fluid intelligence of aged adults.

Multivariate Behavioral Research, 30, 255-287.SAS Institute. (1990). SAS/STAT software: CALIS and LOGISTIC procedures (SAS Tech. Rep. No.

P-200). Cary, NC: Author.SAS Institute. (1991). SAS language guide (Release 6.03). Cary, NC: Author.Satorra, A., & Saris, W. E. (1985). Power of the likelihood ratio test in covariance structure analysis.

Psychometrika, 50, 83-90.Sobel, M. E., & Bohrnstedt, G. W. (1985). Use of null models in evaluating the fit of covariance struc-

ture models. In N. B. Tuma (Ed.), Sociological methodology 1985 (pp. 152-178). San Francisco:Jossey-Bass.

Steiger, J. H. (1989). EzPATH: A supplementary module for SYSTAT and SYGRAPH. Evanston, IL:SYSTAT.

Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach.Multivariate Behavioral Research, 25, 173-180.

Steiger, J. H., & Lind, J. C. (1980, MONTH). Statistically-based tests for the number of common factors.Paper presented at the meeting of the Psychometric Society, Iowa City, IA.

Steiger, J. H., Shapiro, A., & Browne, M. W. (1985). On the multivariate asymptotic distribution of se-quential chi-square statistics. Psychometrika, 50, 253-264.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

APPENDIXSAS Program for Computing Asymptotic Confidence Intervals for Noncentrality Difference

Parameters of Nested Structural Equation Models

In this program, the user supplies the sample size and fit indexes (chi-square values, degrees of freedom,and noncentrality parameters). These are obtained from the output of the nested models fitted to datausing, for example, LISREL8 (Jöreskog & Sörbom, 1993), EzPATH (Steiger, 1989), RAMONA(Browne & Mels, 1994), or S AS PROC CALIS (SAS Institute, 1990). The numbers entered here are theones associated with the models focused on in this article's section, "Illustration on Data From aTwo-Group Study" (see also Raykov, 1995). The meaning of each variable is given in the subsequentnote. The inserted comments, preceded by an asterisk, explain the purpose of subsequent program lines.

DATA NDP;•SAMPLE SIZE AND FIT INDICES ARE FIRST ENTERED;N=248;DELTAI= 31.78;DELTA2 = 22.76;CS1 = 163.78;CS2 = 139.76;DF1 = 132;DF2 = 117;•NESTED MODEL FIT INDICES ARE THEN COMPUTED;DCS = CS1-CS2;DDF = DF1-DF2;•DIFFERENCE IN NONCENTRALITY PARAMETERS ARE OBTAINED;NDP = DELTA1-DELTA2;IF DELTA1<DELTA2 THEN NDP=0;NNDP = NDP/(N-1);GAMMA=SQRT(NNDP/DDF);»PREPARATION FOR A LOOP LOCATING UPPER 90%-CI LIMIT FOR NDP;NC=NDP-.001; :

•LOOP STARTS;DO UNTIL (Q>DCS);NC=NC+.001;Q = CINV(.05,DDF,NC);END; •LOOP ENDS;•UPPER CONFIDENCE INTERVAL LIMITS ARE NOW OBTAINED;U_NDP=NC;U_NNDP = U_NDP/(N-1);U_GAMMA = SQRT(U_NNDP/DDF);•PREPARATION FOR A LOOP LOCATING LOWER 90%-CI LIMIT FOR NDP;NC=NDP+.001;•LOOP STARTS;DO UNTIL (Q<DCS);NC=NC-.001;IF N O 0 THEN Q = CINV(.95,DDF,NC); ELSE Q=0;END; •LOOP ENDS;•LOWER CONFIDENCE INTERVAL LIMITS ARE NOW OBTAINED;IF NC>0 THEN L_NDP=NC; ELSE L_NDP=0;

(Continued)

245

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014

APPENDIX(Continued)

L_NNDP = L_NDP/(N-1);L_GAMMA = SQRT(L_NNDP/DDF);*FINAL RESULTS ARE NEXT OUTPUT;PROC PRINT;TITLE 'LOWER AND UPPER 90%-CI LIMITS:';VAR NDP L_NDP U_NDP NNDP L_NNDP U.NNDP GAMMA L_GAMMA U_GAMMA;RUN;

Note. This program is to be used with nested models only, whereby the indices of the morerestrictive model are DELTA 1, CS1, and DFL N= sample size (to be supplied by the user); NDP =difference in noncentrality parameter, with lower and upper limits of the asymptotic 90% confidenceinterval denoted by L_NDP and U_NDP, respectively; NNDP = normed NDP, with lower and upperconfidence interval limits L_NNDP and U_NNDP; GAMMA = NNDP per degree of freedom (rootdeterioration per restriction), with lower and upper confidence limits L_GAMMA and U_GAMMA;DELTA 1 and DELTA2 = noncentrality parameters of the nested models M¡ and Mh respectively (Mi <Mf, to be supplied by the user; obtained from theLISREL8, EzPATH, RAMONA, or SAS PROC CALISoutputs); CS1 and CS2 = chi-square goodness-of-fit indexes of Mt and M¡, respectively, withcorresponding degrees of freedom DFl and DF2 (to be supplied by the user; obtained from the usedsoftware outputs); CINV(p,d,8) = inverse function of the cumulative distribution function of thenoncentral chi-square distribution with d degrees of freedom and a noncentrality parameter 8, at aprobability cutoff/?; NC = temporary (running) noncentrality parameter needed at each loop-step for theinverse function CINV.

246

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

11:

00 2

1 N

ovem

ber

2014