This article was downloaded by: [Central Michigan University]On: 20 October 2014, At: 06:39Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Multivariate BehavioralResearchPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/hmbr20

A Direct Method for ObtainingApproximate Standard Errorand Confidence Intervalof Maximal Reliability forComposites With CongenericMeasuresTenko Raykov & Spiridon PenevPublished online: 10 Jun 2010.

To cite this article: Tenko Raykov & Spiridon Penev (2006) A Direct Method forObtaining Approximate Standard Error and Confidence Interval of Maximal Reliabilityfor Composites With Congeneric Measures, Multivariate Behavioral Research, 41:1,15-28, DOI: 10.1207/s15327906mbr4101_2

To link to this article: http://dx.doi.org/10.1207/s15327906mbr4101_2

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly or

indirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

A Direct Method for ObtainingApproximate Standard Error andConfidence Interval of MaximalReliability for Composites With

Congeneric Measures

Tenko RaykovMichigan State University

Spiridon PenevUniversity of New South Wales

Unlike a substantial part of reliability literature in the past, this article is concernedwith weighted combinations of a given set of congeneric measures with uncorrelatederrors. The relationship between maximal coefficient alpha and maximal reliabilityfor such composites is initially dealt with, and it is shown that the former is a lowerbound of the latter. A direct method for obtaining approximate standard error andconfidence interval for maximal reliability is then outlined. The procedure is basedon a second-order Taylor series approximation and is readily and widely applicablein empirical research via use of covariance structure modeling. The describedmethod is illustrated with a numerical example.

The topic of scale reliability and its estimation has been the subject of an impres-sive amount of research in the behavioral disciplines over the past several decades(e.g., Bentler, 2004b). Unlike the case of simple sum score (e.g., Bollen, 1989;

MULTIVARIATE BEHAVIORAL RESEARCH, 41(1), 15–28Copyright © 2006, Lawrence Erlbaum Associates, Inc.

We are indebted to the Editor and two anonymous Referees for valuable and critical comments onan earlier draft that have contributed considerably to its improvement, as well as to P. M. Bentler, M. W.Browne, G. R. Hancock, and H. Li for valuable discussions on maximal reliability.

Correspondence concerning this article should be addressed to Tenko Raykov, Measurement andQuantitative Methods, Michigan State University, 443A Erickson Hall, East Lansing, MI 48824.E-mail: raykov@msu.edu

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

Raykov, 2001), the problem of how to choose weights in a linear combination of agiven set of measures so as to obtain maximal reliability or maximal internal con-sistency has met with considerably less attention and in particular rather limitedempirical utilization. After a few initial insightful treatments (e.g., Bentler, 1968;Green, 1952; Thomson, 1940), it has been only more recently that researchers haverevisited this topic with renewed interest (e.g., Bartholomew, 1996; Bentler,2004a; Conger, 1980; Hancock & Mueller, 2001; Li, 1997; Li, Rosenthal, & Ru-bin, 1996; Raykov, 2004; Yuan & Bentler, 2002). While point estimation of maxi-mal reliability is well documented in the literature (e.g., Bentler, 2004a; Conger,1980; Li, 1997), the issue of its interval estimation has received much less attention(Yuan & Bentler, 2002), especially as regards ready and wide applicability of cor-responding procedures in behavioral research.

The present article contributes to bridging this gap. An initial discussion ofthe relationship between maximal alpha and maximal reliability coefficientsshows that as in the unweighted scale case the former is a lower bound of thelatter; hence, when interested in finding that linear combination of a pre-speci-fied homogeneous set of measures which possesses highest measurement con-sistency (“precision”) one only needs to be concerned with maximal reliability.A direct method for obtaining approximate standard error and confidence inter-val for maximal reliability is then outlined that is based on a second-order Tay-lor series approximation of it as a function of model parameters and is readilyand widely applicable in empirical research. A related intention of the article isto draw further attention to the theoretically and empirically relevant concept ofmaximal reliability by providing a straightforwardly utilizable procedure of itsinterval estimation that is no less important for behavioral research than its pointestimation.

NOTATION AND BACKGROUND

Throughout this article, we assume that a set of congeneric measures is given, de-noted X1, X2, …, Xk (k > 1; Jöreskog, 1971); that is, the k components X1, X2, …, Xk

assess the same underlying latent dimension, denoted �, with possibly differentunits and origins of measurement as well as error variances. Hence (e.g., Lord &Novick, 1968),

Xi = Ti + �i = �i + �i� + �i , (1)

holds (i = 1, 2, …, k), where T1, T2, …, Tk and �1, �2, …, �k are correspondinglythe true and error scores of the consecutive measures, with �i assumed in the se-

16 RAYKOV AND PENEV

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

quel uncorrelated and with positive variances designated �i(i = 1, …, k).1 Foridentifiability reasons and without limitation of generality we also assume thatVar(�) = 1, where Var(.) denotes variance in the studied subject population.2

A voluminous body of literature exists on point and interval estimation ofsimple, that is, unweighted sum scores, Z = X1 + X2 + … + Xk, also frequentlyreferred to as unweighted composites/scales (e.g., Bollen, 1989; McDonald,1999; see also Raykov, 2001). By comparison, much less attention—especiallyin empirical studies—has been paid to weighted sum scores, that is, scales thatresult after measures are possibly differentially weighted in the overall sum. Un-like the former voluminous literature, we will be concerned with that of infi-nitely many possible weighted composites X(w) = w1X1 + w2X2 + … + wkXk =w�X, which is associated with highest measurement consistency across the entireset of linear combinations of X1, X2, …, Xk . (In this article, underlining will de-note vector and priming transposition; note that X(1k) is the unweighted sumscore, where 1k denotes the vector of size k consisting only of ones.)

Before focusing on such optimal linear combinations, we mention by way ofbackground for the following developments that as is well known (e.g., Bollen,1989) the reliability coefficient �Z of the unweighted scale score Z = X1 + X2 + … +Xk is

where T and � are the vectors of individual component true scores and error scores,respectively. For the reliability of a weighted scale, X(w) = w�X, due to X(w) = w�T+ w�� it follows that (e.g., Bollen, 1989)

INTERVAL ESTIMATION OF MAXIMAL RELIABILITY 17

� �� �

� � � �

11 , (2)

1 1k

Z kk k

Var T

Var T Var� �

�

�� �

� ��

1If k = 2, additional identifying restrictions are needed, such as indicator loading equality (tau-equiva-lent measures) and/or error variance equality (e.g., parallel measures; Lord & Novick, 1968). Since thetrue scores of congeneric measures are linearly related, T2 through Tk are linear combinations of T1; forsimplicity of notation, in the sequel � = T1 is set, which is only a symbolism rather than an additional re-striction in the model.

2Since the location parameters �1, �2, …, �k are not consequential for reliability (see Equation 2that is unaffected by the mean structure of the original set of measures), without loss of generality wewill assume them all—as usual—equal to zero (e.g., McDonald, 1999); further is assumed that �1, �2,…, �k do not vanish simultaneously, a condition readily fulfilled in empirical behavioral research, andthat as typical in reliability discussions all variables appearing in Equation 1 possess zero means (e.g.,Bollen, 1989; Lord & Novick, 1968).

� �� �

� � � �. (3)

Var w Tw

Var w T Var w�

�

��

� ��

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

In the next section, we explicate the relationship between two arguably immediatecandidates for fulfilling the criterion of optimal measurement consistency (“preci-sion”) and show that we only need to be concerned with one of them when search-ing for a linear combination of the given measures that is associated with least (rel-ative) error variance.

THE RELATIONSHIP BETWEEN MAXIMAL ALPHA ANDMAXIMAL RELIABILITY COEFFICIENTS

Coefficient alpha (�) is still one of the most frequently used psychometric indi-ces in empirical research, partly due to a strong tradition going back at least toGuttman (1945). As shown however by Novick and Lewis (1967), in the currentsetting � is a lower bound of the reliability �Z of the unit-weighted composite Z;thereby, � = �Z if and only if X1, X2, …, Xk are (essentially) tau-equivalent, thatis, evaluate the same true score in the same units of measurement (but with pos-sibly different scale origins and/or error variances; below, for convenience, werefer to this statement in Novick & Lewis, 1967, as “lower bound theorem”). Inthe context of maximal measurement “precision,” it may be tempting to hypoth-esize that an appropriate linear combination of the measures X1, X2, …, Xk,which would be associated with the highest possible coefficient alpha, might“compensate” for this general deficiency of �. This hypothesis turns out to beuntrue.

Indeed, consider an arbitrary linear combination of the original measures inEquation 1, X(u) = u1X1 + u2X2 + … + ukXk , and its associated coefficient al-pha, designated �X(u). Denote Yi = uiXi (i = 1, …, k). Since Yi (i = 1, …, k) arescalar multiples of congeneric measures, it follows that (a) Y1, Y2, …, Yk arethemselves congeneric measures that (b) assess the same construct underlyingthe original measures X1, X2, …, Xk (i.e., � in Equation 1), and (c) their errorscores are uiEi and uncorrelated among themselves (like those of X1, X2, …,Xk). Thus, X(u) = Y1 + Y2 + … + Yk is a unit-weighted composite of the conge-neric measures Y1, Y2, …, Yk with uncorrelated errors, and hence the lowerbound theorem in Novick and Lewis (1967) is applicable for X(u). Therefore, itfollows that �X(u) does not exceed the reliability coefficient of X(u), denoted�X(u), that is,

�X(u) ≤ �X(u). (4)

However, X(u) = Y1 + Y2 + … + Yk = u1X1 + u2X2 + … + ukXk is a linear combinationof the initial measures X1, X2, …, Xk , and as such possesses reliability that is no

18 RAYKOV AND PENEV

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

higher than the maximal reliability achievable with the same measures, denoted*�, that is,

�X(u) ≤ *�. (5)

From Inequalities 4 and 5 evidently follows

�X(u) ≤ *�. (6)

Inequality 6 states that for an arbitrary linear combination of the initial set of mea-sures—and hence for any linear combination of them—the corresponding � doesnot exceed the maximal reliability coefficient for this set (which coefficient is aconstant). Hence, the maximal coefficient alpha, denoted *�, that is achievablewith a linear combination of a given set of congeneric measures with unrelated er-rors is a lower bound of the maximal reliability coefficient for a linear combinationof these measures, that is,

*� ≤ *�. (7)

The developments in this section and in particular Inequality 7 show that whenconcerned with point and interval estimation of maximal measurement consis-tency one need only be concerned with the maximal reliability coefficient, as wewill be in the rest of this article. In the next section dealing with maximal reliabilityin more detail, we will address the question of when in Inequality 7 equality ob-tains and when strict inequality holds in it.

A DIRECT METHOD FOR EVALUATION OF ANAPPROXIMATE STANDARD ERROR AND CONFIDENCE

INTERVAL FOR MAXIMAL RELIABILITY

The issue of how to point estimate the weights in the optimal linear combinationrendering the maximal reliability coefficient, as well as the latter itself, has longbeen resolved (e.g., Conger, 1980; Green, 1952; Li, 1997; Thomson, 1940); a rela-tively straightforward one-step procedure accomplishing it, along with interval es-timation of these optimal weights, has been also recently proposed within theframework of covariance structure analysis in Raykov (2004). How to evaluate anindex of stability of maximal reliability, however, that is, how to interval estimatethe maximal possible reliability coefficient, has received markedly less attention.The procedure resolving it (Yuan & Bentler, 2002) may be viewed based on a

INTERVAL ESTIMATION OF MAXIMAL RELIABILITY 19

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

first-order approximation of the maximum reliability coefficient as a function ofmodel parameters, and in addition does not seem to be in a form allowing readyand wide utilization in empirical behavioral research. The present section respondsto these concerns.

First, as shown for instance in Conger (1980), the linear combination renderingmaximal reliability is obtained with the following weights:

That is, the optimal linear combination of X1, X2, …, Xk in Equation 1, which is as-sociated with the highest degree of measurement consistency, is

Before proceeding with interval estimation of maximal reliability, Equation 9allows us to answer the question when equality holds in Inequality 7 and when itis a strict inequality. Specifically, Equations 9, 1 and the lower bound theorem inNovick and Lewis (1967) show, in a way similar to the reasoning in the preced-ing section, that in the present setting (congeneric measures with uncorrelatederrors) maximal reliability equals maximal coefficient alpha if and only if themeasures Vi = �iXi / �i = �iTi / �i + �i�i / �i = + �i�i / �i (i = 1, …, k; seeEquation 1) are (essentially) tau-equivalent. That is, *� = *� when and onlywhen

One obvious example when Equation 10 holds, and hence maximal alpha equalsmaximal reliability, is when the measures X1, …, Xk are parallel (i.e., measure thesame construct in the same units of measurement and with the same error vari-ances; e.g., Jöreskog, 1971). However, there are other cases when Equation 10 istrue and the maximal alpha and maximal reliability coefficients are identical. Inparticular, as can be readily seen with simple algebra from Equation 1, Equation 10is equivalent to

�1 = �2 = … = �k , (11)

20 RAYKOV AND PENEV

� �1

* * . (9)k

ii

ii

X X w X�

��

� � �

2 /i ii T� �

22 21 2

1 2

. (10)k

k

�� �

� � �� � ��

� �* 1, 2, ..., . (8)ii

i

w i k�

�� �

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

with �i denoting the reliability coefficient of Xi (i = 1, …, k). In any other case, that is,when at least two of the original measures in Equation 1 have distinct reliabilities,maximal alpha is lower than maximal reliability, that is, *� < *� holds.

The large-sample distribution of the maximal reliability coefficient hasbeen derived recently in Yuan and Bentler (2002, pp. 253–254). In particular,they have shown that the asymptotic variance of the maximal reliability esti-mator is

where J is the vector of partial derivatives of the maximal reliability coefficientwith regard to all model parameters (see, e.g., Equation 14, in the following), isthe vector of partial derivatives of all elements of the model implied covariancematrix � with regard to these parameters,

with Dp being the duplication matrix rendering the stacked vector of all col-umns of � from the stacked vector of only its nonredundant elements, and� is the Kronecker product symbol (e.g., Magnus & Neudecker, 1999). Em-pirical use of the expressions in Equations 12 and 13 is however (a) ratherlaborious and tedious in routine behavioral research in need of interval esti-mation of maximal reliability, (b) involves taking by the researcher of multi-ple partial derivatives of this reliability coefficient with respect to model pa-rameters, (c) has the inconvenient property that the number of thesederivatives increases with increasing length k of the initial composite of in-terest (as could be repeatedly the case when one is involved in scale devel-opment and revision), and (d) can be viewed as based on a first-order ap-proximation of maximal reliability as a function of model parameters. Thisis the motivation for presenting next a straightforward approach that (a) issubstantially more easily and widely applicable, (b) does not require thecomputational activities involved in an utilization of Equations 12 and 13 forpurposes of interval estimation of maximal reliability, (c) does not becomemore laborious in the above sense with increasing scale length and/orchange in the composition of the original scale, and (d) is based on a sec-ond-order Taylor series approximation of maximal reliability as a function ofmodel parameters.

INTERVAL ESTIMATION OF MAXIMAL RELIABILITY 21

� �1 1.5 , (13)p pW � �� D D� ��

��

� � 12 , (12)W��

� � �� J J� �� �

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

To this end, we first note that substitution of Equation 8 into Equation 3 yieldsthe following expression for the maximal reliability coefficient in terms of modelparameters:

Based on Equation 1, Equation 14 leads to:

(see also Footnote 2). Denoting for simplicity of notation and later convenience

from Equation 15 it follows that the maximal reliability coefficient is

22 RAYKOV AND PENEV

2

1

, (16)k

i

ii

�

��

� �

� �

� � � �

2

1

2

1 12

2

12

2 2

1 12

12

1

/

*

/ /

, (15)

1

k

i ii

k k

i i i i ii i

ki

ii

k ki i

i ii ik

i

iik

i

ii

Var

Var Var

� � �

�

� � � � � �

�

�

� �

� �

�

�

�

�

�

� �

�

� �

�

�

� � � �� �

� � � �� � � � �� � � ��� �� �� ����� �

�� ��� �� ��� ����� �

�

�

�

� �

�

� �

�

�

� �

� � � �

1

1 1

/

* . (14)

/ /

k

i i ii

k k

i i i i i ii i

Var T

Var T Var

� �

�

� � � � �

�

� �

� � � �� �

� � � �� � � � �� �

�

� �

* . (17)1

�

�

�

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

Thus, if

symbolizes the population value of , with �0,i and �0,i being the population valuesfor the loadings and error variances, respectively, a second-order Taylor expansionof *� in the vicinity of 0 yields from Equation 17 (e.g., Rao, 1973):

where ‘≈’ denotes “approximately equal.”Hence, if s symbolizes the standard error associated with an optimal estimator

of , say a maximum likelihood estimator with a large sample (Bollen, 1989), thenan approximate standard error for maximal reliability would be obtained by apply-ing the variance operator on both sides of Equation 19 and taking (positive) squareroot; this leads to the following estimate of a large-sample standard error of themaximal reliability coefficient:

where a caret denotes estimate.3

Since Equation 1 can obviously be considered defining a covariance structuremodel (e.g., Jöreskog, 1971), with the large-sample normality of the parameter esti-mator in covariance structure modeling (e.g., Bollen, 1989) a large-sample 100(1 –)%-confidence interval formaximal reliabilitywouldresult fromEquation20as

where z1-/2 is the 1 – /2th quantile of the standard normal distribution (0 < < 1).

INTERVAL ESTIMATION OF MAXIMAL RELIABILITY 23

� �� � � �

2

2 2

2ˆ. . * 1 , (20)ˆ ˆ1 1

s sS E

�

� �� �

� � � � � � � � � �2 2 3

0 0 0 0 0* / 1 / 1 , (19)� � � � � � � �≈

3Strictly speaking, formal application of Equation 20 would require one to assume that convergencein law of to the limiting normal distribution occurs simultaneously with that of the firstfour moments to those of the pertinent normal (N stands for sample size); if the distribution of the origi-nal measures is not too heavy tailed, this convergence would usually hold in empirical research employ-ing covariance structure models.

� �0ˆN �

20,

00,1

, (18)k

i

ii

�

��

� �

� � � �� �1 / 2 1 / 2ˆ ˆˆ ˆmax 0,* . . * ,min 1,* . . * , (21)z S E z S E � � � �� �

� � � �� �

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

To obtain a large-sample standard error for an estimator of the parametric ex-pression in Equation 20, no additional effort is required on part of the researcher,since the popular covariance structure modeling program LISREL (Jöreskog &Sörbom, 1996) can estimate and provide a standard error for it. This will be ac-complished if is defined following Equation 18 as an additional (auxiliary) pa-rameter of the covariance structure model in Equation 1, which does not have anyimplications for the covariance structure and thus does not affect model fit (see fol-lowing section and Appendix). In this way, when using the presently proposed in-terval estimation approach no further activities are required by the analyst when in-creasing or alternatively decreasing the length of the scale under consideration(i.e., changing the number k of initial congeneric measures considered, X1, …, Xk)or when altering its composition. This is because the (structure of the) critical stan-dard error formula in Equation 20 is unchanged then—as long as the definition of in Equation 18 accounts for the changed length k or scale composition in the modelfitting/analytic session—and one only needs to substitute in it the standard error of that is obtained for the new scale in the same manner as for the old one. Thisproperty implies a considerable convenience feature of the interval estimation pro-cedure outlined in this article, which can be readily capitalized on in the process ofscale development and revision that is a frequent activity behavioral scientists be-come engaged in empirical research. The next section illustrates the described ap-proach with a numerical example.

ILLUSTRATION ON DATA

To demonstrate the utility and applicability of the outlined method for interval esti-mation of maximal reliability, we employ simulated multinormal zero mean datageneratedfork=4componentsonN=500casesaccording to thefollowingmodel:

where � was a standard normal variable while �1 through �4 were independentzero-mean normal variables with variances .4 each; the covariance matrix of thesimulated data is given in Table 1. Following Equation 9, the linear combination ofX1 through X4 with maximal reliability in the population is determined from Equa-tion 22 as:

*X = 2X1 + 2.25X2 + 1.5X3 + 2X4. (23)

24 RAYKOV AND PENEV

1 1

2 2

3 3

4 4

.8

.9

.6

.8 , (22)

XXXX

� �� �� �� �

� �� �� �� �D

ownl

oade

d by

[C

entr

al M

ichi

gan

Uni

vers

ity]

at 0

6:39

20

Oct

ober

201

4

The reliability of *X, that is, the population maximal reliability coefficient, is thendetermined from Equation 15 as

To utilize the procedure for interval estimation of maximal reliability proposedin this article, we first fit the congeneric model defined in Equation 1 to the data inTable 1 using the maximum likelihood method (e.g., Jöreskog & Sörbom, 1996),with an added parameter for the quantity that is defined to equal the expression inthe right-hand side of Equation 18 (see Appendix for the LISREL source code andspecifically its COnstraint line accomplishing the definition of ). This model isassociated with acceptable goodness-of-fit indices: chi-square = 1.01, degrees offreedom = 2, p-value = .60, and root mean square error of approximation = 0 with a90% confidence interval (0, .072). The quantity is thereby estimated at 6.34, witha standard error of .55. Using this estimate, via Equation 16 the maximum likeli-hood estimate of maximal reliability results as:

that is identical, up to round-off error, with the true value of the maximal reliabilitycoefficient in Equation 24. Employing now Equation 20, an approximate standarderror of maximal reliability is furnished as

With Equation 26, an approximate 95% confidence interval for maximal reliabilityis now rendered as

(.84, .88). (27)

INTERVAL ESTIMATION OF MAXIMAL RELIABILITY 25

TABLE 1Covariance Matrix of Simulated Data (N = 500)

Variable X1 X2 X3 X4

X1 0.98X2 0.71 1.18X3 0.49 0.51 0.78X4 0.63 0.70 0.48 1.01

6.34ˆ* .86, (25)7.34

� � �

� �� �

� �2 22

.55ˆ. . * 1 2 .55 / 7.34 .01. (26)1 6.34

S E � � � ��

� �

� �

.64 / .4 .81/ .4 .36 / .4 .64 / .4* .86. (24)

1 .64 / .4 .81/ .4 .36 / .4 .64 / .4�

� � �� �

� � � �

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

We conclude that the population maximal reliability possible to obtain with alinear combination of the four components X1 through X4 defined in Equation 22(a) could be expected with high confidence to lie between .84 and .88 (an inter-val that indeed covers the population maximal reliability in Equation 24), (b) isbest estimated at .86, and (c) is associated with the weighted scale score *Xgiven in Equation 23.

CONCLUSION

This article proposed a direct procedure for interval estimation of maximal reli-ability, a theoretically and empirically relevant concept that has been notably un-dervalued in empirical behavioral research over the past 50 years or so. Initially,the discussion showed that for pre-specified congeneric measures withuncorrelated errors maximal coefficient alpha is a lower bound of maximal reli-ability obtainable with a linear combination of these measures. Hence, when in-terested in optimal measurement consistency (“precision”) with a given set ofunidimensional components one needs to be concerned only with their corre-sponding maximal reliability coefficient. A readily applicable method was out-lined then for obtaining an approximate standard error and confidence interval ofmaximal reliability for such measures. The proposed approach complements al-ready available and well-documented methods for point estimation of maximalreliability (e.g., Bartholomew, 1996; Bentler, 2004b; Conger, 1980; Li, 1997;Raykov, 2004). In addition to the general method in Yuan and Bentler (2002) forobtaining the asymptotic distribution of maximal reliability, the method in thisarticle represents a straightforwardly and widely utilizable procedure in behav-ioral research that is based on a second-order approximation of maximal reliabil-ity as a function of model parameters. Further, the present procedure is em-ployed with little added effort on part of the researcher when the original scale islengthened/shortened or its composition changed. This feature becomes of par-ticular empirical convenience when one is involved in scale development and re-vision activities, as is frequently the case in empirical work. Last but not least,being developed within the framework of the covariance structure modelingmethodology whose foundation is an asymptotic statistical theory, the describedmethod is best employed with large samples and similarly requires (approxi-mately) continuous components (e.g., Raykov, 1997) as well as a tenable conge-neric model for them. In such settings, this approach is also applicable with datathat deviate from normal by exhibiting not too heavy tails (Footnote 3), as longas the fit function appropriate for the distribution of observed measures is usedfor purposes of fitting the covariance structure model Equation 1 (Bollen, 1989;Bentler, 2004a).

26 RAYKOV AND PENEV

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

REFERENCES

Bartholomew, D. J. (1996). The statistical approach to social measurement. London: Academic Press.Bentler, P. M. (1968). Alpha-maximized factor analysis (alphamax): Its relation to alpha and canonical

factor analysis. Psychometrika, 33, 335–345.Bentler, P. M. (2004a). EQS structural equation program manual. Encino, CA: Multivariate Software.Bentler, P. M. (2004b, July). Should coefficient alpha be replaced by model-based reliability coeffi-

cients? Keynote address at the Biannual Meeting of the Society for Multivariate Analyses in the Be-havioral Sciences, Jena, Germany.

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.Conger, A. (1980). Maximally reliable composites for unidimensional measures. Educational and Psy-

chological Measurement, 40, 367–375.Green, B. F. (1952). A note on the calculation of weights for maximum battery reliability. Psychometrika,

17, 57–61.Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10, 255–282.Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable sys-

tems. In R. Cudeck, S. H. C. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Past andpresent. A festschrift in honor of Karl G. Jöreskog (pp. 195–221). Chicago: Scientific Software Inter-national.

Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109–133.Jöreskog, K. G., & Sörbom, D. (1996). LISREL8: User’s guide. Chicago: Scientific Software Interna-

tional.Li, H. (1997). A unifying expression for the maximal reliability of a linear composite. Psychometrika,

62, 245–249.Li, H., Rosenthal, R., & Rubin, D. (1996). Reliability of measurement in psychology: From

Spearman-Brown to maximal reliability. Psychological Methods, 1, 98–107.Lord, F. M., & Novick, M. (1968). Statistical theories of mental test scores. Readings, MA: Addi-

son-Wesley.Magnus, J. R., & Neudecker, H. (1999). Matrix differential calculus. New York: Wiley.McDonald, R. P. (1999). Test theory. A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates,

Inc.Novick, M., & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements.

Psychometrika, 32, 1–13.Rao, C. R. (1973). Linear statistical inference and its applications. New York: Wiley.Raykov, T. (1997). Scale reliability, Cronbach’s coefficient alpha, and violations of essential tau-equiv-

alence for fixed congeneric components. Multivariate Behavioral Research, 32, 329–354.Raykov, T. (2001). Estimation of congeneric scale reliability via covariance structure analysis with

nonlinear constraints. British Journal of Mathematical and Statistical Psychology, 54, 315–323.Raykov, T. (2004). Estimation of maximal reliability: A note on a covariance structure modeling ap-

proach. British Journal of Mathematical and Statistical Psychology, 57, 21–27.Thomson, G. H. (1940). Weighting for battery reliability and prediction. British Journal of Psychology,

30, 357–360.Yuan, K.-H., & Bentler, P. M. (2002). On robustness of the normal theory based asymptotic distribu-

tions of three reliability coefficient estimates. Psychometrika, 67, 251–259.

Accepted August 2004

INTERVAL ESTIMATION OF MAXIMAL RELIABILITY 27

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014

APPENDIXLISREL Source Code for IntervalEstimation of Maximal Reliability

STANDARD ERROR FOR MAXIMAL RELIABILITY (ILLUSTRATION EXAMPLE)

DA NI=4 NO=500

CM = <SEE TABLE 1>

MO NY=4 NE=2 PS=SY,FI

LA

X_1 X_2 X_3 X_4

LE

ETA AUXILVAR ! PS(2,2) = (SEE CONSTRAINT BELOW)

VA 1 PS 1 1

FR LY 1 1 LY 2 1 LY 3 1 LY 4 1

CO PS(2,2)=LY(1,1)**2*TE(1,1)**-1+LY(2,1)**2*TE(2,2)**-1+C

LY(3,1)**2*TE(3,3)**-1+LY(4,1)**2*TE(4,4)**-1

! THIS IS THE RHS OF EQ. (16)

OU

Note. Use (a) standard error of PS(2,2) in place of s and (b) estimate ofPS(2,2) in place of in Equation 20 to obtain approximate standard error formaximal reliability; with this standard error, Equation 21 furnishes approximateconfidence interval for maximal reliability at a given confidence level (RHS =right-hand side; AUXILVAR = auxiliary variable whose variance is constrained toequal ). With a different number of measures, modify correspondingly 2nd, 6th,and 10th through 12th lines (see Equation 8; Jöreskog & Sörbom, 1996); with a(partly) different set of measures, change correspondingly the input data (e.g., se-lected variables or covariance matrix) in 3rd line, and if necessary account for dif-ferent number of measures as mentioned.

28 RAYKOV AND PENEV

ˆ,

Dow

nloa

ded

by [

Cen

tral

Mic

higa

n U

nive

rsity

] at

06:

39 2

0 O

ctob

er 2

014