guttman’s model abilityguttman’s model of ability tests aharon tziner, tel-aviv university...

59

Examination of an Extension of

Guttman’s Model of Ability Tests

Aharon Tziner,Tel-Aviv University

Avigdor Rimmer,Pliat, Ltd., Tel-Aviv

An extension of Guttman’s structural model of abil-

ity tests was devised and investigated with two sam-ples consisting, respectively, of 335 and 225 males.The examinees in the first sample came for vocationalguidance after their military service and were adminis-tered a 17-test battery. The second sample consisted ofapplicants for various jobs in an organization and wereadministered a 14-test battery. For each sample, a ma-trix of intercorrelations between scores was obtainedbased on the number of correct responses. The mat-rices were submitted to Guttman-Lingoes SmallestSpace Analysis. The two-dimensional structure foundwas a radex in which (1) the facet of the language ofpresentation radially divided the space and (2) thefacet of mental operation formed concentric rings. Thesignificance of these findings for theoretical and ap-plied problems relating to ability tests is discussed.

The underlying structure of ability tests is a topicof considerable interest (cf., Cunningham, 1981;Kelderman, Mellenbergh, & Elshout, 19~1). There

appear to be two primary reasons for this. First,from a general perspective of ability testing it is

important to have a coherent structural model thatreasonably organizes the tests into meaningful fac-ets. Such a model could aid in deciding, in a sys-tematic and rational manner, what new tests shouldbe developed in order to measure facets previouslyonly partially assessed, or not assessed at all. Sec-ond, researchers are interested in determining

whether there is a universal structural model of

general applicability that does not vary even whenused for different testing objectives (e.g., voca-tional counseling or selection for job openings inorganizations) or when used on individuals withvarying levels of education, age, and so forth.

Previous studies that were aimed at examiningthe foregoing questions were mainly based on fac-tor analysis (see, for instance, Guilford, 1980, 19~1).This method, however, has been sharply criticizedon several grounds. Basic assumptions of its sta-tistical model, as, for instance, linearity, are oftenviolated (Keldenman et ~l. , 1981). Apart from this,factors with only one or two salient loadings arefrequently found when using factor analysis to re-veal the structure of ability tests (Cunningham,1981). In such a context, the decision regardingthe appropriate number of factors becomes unu-sually difficult, thus leading to solutions that arenot parsimonious and giving rise to an inconsis-tency across investigators (Schlesinger & Guttman,1969). Furthermore, although the factor analyticstrategy of research has often been used to test

hypotheses of structure, it has also been shown tobe inadequate for this purpose (Guttman, 1958;Shapira & Zevulun, 1979; Shye, 1978).A better approach to the structural configuration

issue might emerge from Guttman’s facet theory(for a detailed exposition see Shirom, 1982; Shye,1978). Relying on this approach, Schlesinger andGuttman (1969) classified ability tests into two fac-

Downloaded from the Digital Conservancy at the University of Minnesota, http://purl.umn.edu/93227. May be reproduced with no cost by students and faculty for academic use. Non-academic reproduction

requires payment of royalties through the Copyright Clearance Center, http://www.copyright.com/

60

ets : (1) the language of presentation of the test

items, which may be verbal, ngural, or numerical;and (2) the mental operation required by the test,ely, rule inference or rule application.They thendefined a mapping sentence to corroborate thestructural configuration based on the foregoing fac-ets. The empirical investigation, using high schoolstudents as subjects, firmly supported the hypoth-esized structural configuration: a radex yielded bya radial arrangement of tests according to the lan-guage of presentation facet, together with a patternof concentric circles formed by the mental opera-tion facet.

Nevertheless, it is important to outline a weak-ness inherent in one of the facets-that of mental

operation. This facet did not take into account theimportant differentiation in mental operations in-dicated by Piaget (1956; and also in Brainerd, 1978).According to Piaget, the rule of application oper-ation entails three levels of cognitive complexityorganized hierarchically from the lowest to the

highest:1. Ability to successfully handle concrete items

(designated here concrete rule application),2. Ability to successfully handle signs (clerical

rule application), and3. Ability to successfully handle cognitive con-

cepts (abstract- cognitive rule of application).For this reason the mental operation was res-

tructured in the present study so that it would be

composed of four elements instead of two:1. Rule inference,2. Abstract-cognitive rule application,3. Clerical rule application, and4. Concrete rule application.

Using this extension, a mapping sentence was de-vised similar to that formulated and examined bySchlesinger and Guttman ( 1969) in order to extractthe maximum from the operation facet. The aimwas to find out empirically whether or not the radexstructural configuration is still preserved after theintroduction of the above-mentioned extension. Two

replications were carried out: one on a sample ofindividuals applying for vocational counseling, theother on a sample of selected applicants for jobpositions in organizations. It was then decided to

test the mapping sentence, employing two repli-cations, in order to ascertain whether the objectivesof testing in any way affected the structural con-figuration of ability tests that emerged.

Research Mapping Sentence -

The mapping sentence tested in the current studyis shown in Figure 1. The aim was to examine

empirically whether the radex would still be an

appropriate structural configuration to portray geo-metrically the interrelationships among the ele-

ments of facets, even when using this elaboratemapping sentence.

Method

Subjects and Procedure

The first sample consisted of 335 males, mostlyaged 21 to 22, selected at random from applicantsfor vocational guidance after completion of theirmilitary service. The second sample consisted of225 males aged 2 1 t® 35, also selected at random, 9from applicants for job positions in an organiza-tion. Both groups completed ability tests that wereadministered and scored by professional psychol-ogists-17 tests for those in the first sample and14 for those in the second. Examinees in the first

sample were tested at the Vocational Branch of theIsraeli Ministry of Labor and Welfare in Tel Aviv,and those in the second sample were tested at aprivate psychological testing company providingselection and placement services.

Variables

The combination of elements of the two facets, 9A and B, forms profiles ~stru~ta~pdes in facet theoryterminology) represented here by the ability tests.For instance, the Verbal Analogies test representsthe ajbi structuple. The detailed list of tests em-ployed, along with the structuples to which theycorrespond, is presented in Table 1.

Data Analysis Procedure

The investigation of the empirical structure ofthe ability tests was carried out using Guttman-



61

Figure 1The Mapping Sentence for Ability Tests

The performance of subject (x) on a psychometric ability test item presentedin

Lingoes Smallest Space Analysis (SSA; see, e.g.,Guttman, 1968; Lingoes, 1973). This is a non-

metric multidimensional scaling procedure that

geometrically represents the correlation matrix basedon the order of magnitude of the intercorrelationsamong the variables-in this case, the tests. SSA

portrays the tests in the current study as points inan Euclidean space. The more the correlation be-tween the corresponding tests increases, the nearerthe two points are to each other. More explicitly,if rij is the observed correlation between tests i and

j, and if dij is the calculated distance between thesetwo tests for the smallest space, then SSA attemptsto find out the space with the minimal number of

dimensions in which the following inequalities arepreserved:

Order among the correlations is thus preserved in

the smallest possible Euclidean space, and the ab-solute sizes of the input correlation coefficients canbe reproduced from the scattergram of the rii onthe dij (Shepard diagram) printed by the computer.Such a chart indicates how well ~;~ = f(dr~)9 wheref is a monotonically decreasing function.

The goodness of fit between the smallest possibleEuclidean space and the correlation matrix is nu-

merically measured by a coefficient of alienation(1 _ ~.z)vz9 where r is a rank-order correlation be-tween the intercorrelations of the tests and their

corresponding geometric distances. The smaller thecoefficient of alienation, the better the fit. Zero

represents a perfect fit. In practice, however, acoefficient of alienation smaller than .15 is con-

sidered a good fit (see Guttman, 1968).To find evidence regarding the structural config-

uration in the SSA output, it is necessary to applythe principles of contiguity regions (Elizur, 1981;Shapira & Gev~l~n9 1978). In this connection, and



62

Table 1

Psychometric Ability Tests and the Structuples they Representin the Two Replications

in order to detect a radex structure, one of the facetsin the mapping sentence of this study (A) shouldhave acted as a polarizing facet, while the other(B) should have acted as a modulating facet. Afacet is designated as modulating when it dividesthe space into concentric circles and as polarizingwhen three or more lines originating at the samepoint (or pole) separate the space by diverging intodifferent directions (Schlesinger & Guttman, 1969).Additional relevant details pertaining to the prin-cipl~s ®f contiguity in facet analysis can be foundelsewhere (Gratch, 1973; Shye, 1978).

Results

First computed among the number of correct re-sponses to the tests were Pearson intercorrelations. eTables 2 and 3 display the results for the first andsecond replication, respectively. The first point ofinterest that can be seen in both tables is that all

the correlations are zero or positive, fulfilling oneof the requirements set up by Guttman for facetanalysis (for more details refer to Gratch, 1973). eIn addition, the correlations in the first applicationare much higher than in the second application, a



63

itII III IuIIIII I

II 11

L1’) II u u

~ !II G ~ II II

II

II !!, I: II

II 11

’M .¡.J C(j II II

II

, p _ ~ ~ il n ll n

’M U II II

~ P< II II

u II II

.¡.J II II

’M

~ 0)

II II II fl

w II fl

v II II ii 11

..c .¡.J II II

~.5 ~ II II II

~ &dquo; ida 0) II II

E-iE-i1l C(j Q) II ii II~ !

II

II ii, _ _ II ii II

....... II II

, ~ < II )) n u

~ ! II II

G II II

QJ j II n u

~! II II

çQ Q)

II II u

0) II II

0 ~ II II II

il n.¡.J C(j II II

II n ii

....... Q) II II

il n n

~ H II II

o ! ii ii il ii

U II II

IIIIIIIIIIIIIIIIIIIIIIIIIIIII11

&dquo;0~i

coo

If&dquo;Bo

V

<1J

n]CD

~<1J()

&dquo;&dquo;&dquo;

M

OJ

c¡¡

m

0

<1J

~tBI

4-4o

Co

<1J

OJ

o()!-<

,...

Q)

W



64

+

««

11

G

E0

4.JMu

~C-

w«

bx0uNm

N

CT ~r

N .11 rl

2 W

11 1-1

p WNH

I.1

.1<

r:NN3

NSS

ro

Ijlio

(L)~4ou

’1j

0:1

0:3:

o

v

nj0:10:1

4.J

e0:1U

....

c/;)J]

I])I-<0:1

I])

o..c0:1

’1j

0:1

;J

11

....o

UJ¡:;o

0:1......

I])

o(j

I])W¡:;

,....

tt)Wo



65

fact that can be explained by the first sample beingmore heterogeneous than the second, as dictatedby the objective of testing (vocational guidanceversus selection and placement for a particular or-ganization).

These matrices were then subjected to the SSA-I computer program, which maps the tests as pointson the Euclidean space of two dimensions. Thesetest configurations are shown in Figures 2 and 3,corresponding to the matrices in Tables 2 and 3 9respectively. The coefficients of alienation ob-

tained were . 11 and .095, respectively, for the con-figurations in Figures 2 and 39 which can be con-

sidered as indicating good and very good fit of thetwo-dimensional plots to the original intereorrela-tion matrices (as references for goodness of fit, seeGuttman, 1968; Kenny & Canter, 1981). Both fig-ures show that every test is contiguous with othersthat share the same facet elements. This is con-

sistent with the principle of contiguity in facet the-ory (see Shye, 1978), which explicitly states thatitems having the same facet elements should bemore highly correlated and thus closer together inthe configurative plot than items that do not. Thisis illustrated here, for example, by Verbal Analysisand ~i~dir~~mth~-®dd-~J®rd tests pertaining to the

Figure2SSA Map for the 17 Tests in the First Group

(Vocational Guidance Situation)



66

Figure 3SSA for the 14 Tests in the Second Group

(Selection Situation)

same structuple <2~j in Figure 2. These are clearlycloser to each other than to the Identifying Figuresin Small Print test, which represents structuplea3b3 .

Furthermore, both figures have a similar config-urative structure, as is expected according to themapping sentence: a radex configuration formed byjoint action of the language presentation facet (A), 9which divides the space radially (polarizing facet),and the mental operation facet (B), which dividesthe space from the center to the outside in concen-

tric rings (modulating facet). Certain implications

are evident concerning the order of elements inFacet B. This facet divides the space so that (1)the rule-inferring ability tests occupy the centralregion, (2) encircled contiguously by the cognitive-abstract rule applying test region, (3) next by theclerical rule-applying tests regions, and (4) furthestby the concrete rule-applying tests region. Hence,the tests in each region are much closer to eachother than to tests outside the region. For instance,Mathematical Progressions tests and Raven Pro-gressive Matrices tests are much closer to eachother than to Addition Exercises tests (Figure 3),



67

since both pertain to the rule-inference region, whilethe latter pertains to the clerical rule-applying re-gion.

There is, however, one noticeable exception tothe above conclusion: In general, the ability testsrepresenting structuple ~3~ ~ Figure 3 cluster to-gether in the figural area within the concrete rulecircle. This is correct for all of them, except forKoh’s cubes test, which is unexpectedly situatedin the clerical circle. It may be reasonable to sup-pose that this seemingly odd finding could be ac-counted for by the fact that it presents stimuli in aconcrete form, although the type of mental oper-ation required by this test is the rule of inference.Thus, this test is positioned somewhere in the midarea, as is indeed shown by its place in Figure 3.

Discussion

The implications of the results in this study aretwo-fold. First, although only a relatively smallnumber of tests were involved, of which not all

the structuples were equally represented (if at all)by tests, and although a new elaborated mentaloperation facet (instead pf the original one gener-ated by Schlesinger & Guttman, 1969) was used,the radex proved again to be an appropriate con-figuration of the structure of ability tests. This con-clusion is further supported by the fact that theradex structure did not evolve by chance, but ratherthat the empirical SSA results confirmed what hadpreviously been predicted by the mapping sentenceof the current study. Second, the configuration ofpoints representing the ability tests exhibited thesame structure (the radex) in both replications, al-beit practically the tests administered to the testeesin the two samples differed greatly, as dictated bythe objective of testing (i.e., vocational guidanceversus selection and placement). Consequently, itappears plausible to conclude that the radex struc-ture can be expected to reproduce itself even if

other tests are utilized, given that the two facets-mental operation and language of presentation-are represented.Of additional interest is the fact that those few

tests employed in both replications (c.~., Raven’sProgressive Matrices) retained location in the same

regions of the radex in Figure 2 as in Figure 3,thus indicating that the placement of a test in theradex is apparently invariant with respect to othertests in the same battery; rather, the location is

predictable by the mapping sentence. In contrast,the structural invariance is clearly opposed to thesalient difference in patterns of intercorrelation be-tween tests, notably upon scrutiny of Tables 1 and2. This probably demonstrates that intercorrela-tions are to a large extent affected by various ex-traneous factors (e.g., characteristics of exami-

nees) unlike the remarkable stability of theconfigurative structure-the radex. In this regardthe authors are, in essence, consistent with Elizur

(1982), who arrived at a similar conclusion in hisinvestigation of work values structure. In order tofurther support the conclusions concerning this

structure, more studies across various sociode-

mographic characteristics of examinees (e.g., age,level of education, type of occupation) are certainlyneeded; they should, in addition, deal with the ob-jectives of testing (e.g., assessment of managerialpotential, assessment for promotion and for place-ment in organizations).A further implication emerged from the findings

concerning the radex structure. As mentioned pre-viously, the mental operation facet divided the spaceinto four concentric regions. These regions may beconceived as being arranged in a gradient orderformed according to mental abstractness when theconcrete rule-application is the lowest level andrule inference is the highest. This implies that ataxonomy of ability tests might be devised accord-ing to the above-mentioned four levels of abstract-ness. Further, this taxonomy would be maintained,no matter in what language test items are presented-verbal, numerical, or figural.

If the veracity of this conclusion is proved infurther similar replications, a practical contribu-tion, in addition to the theoretical contribution, couldbe made to the understanding of the underlyingstructure of ability tests. For instance, in order todesign a battery of ability tests for selection, it

would only be necessary to obtain information-via a job analysis procedure-about the mentaloperation and the language of presentation that pre-dominantly accounts for successful performance in



68

that job. Then, all that would be required wouldbe to select or to construct tests corresponding tothe type of mental operation and the chosen lan-of presentation revealed in the job analysisprocess.Some readers may consider this suggestion to be

oversimplified. The authors trust, however, that asit is theoretically reasonable, it offers an approachfor devising test batteries from a minimal numberof components that would still contain all the rel-evant core elements. Some redundancy among pre-dictors would exist, even if the proposed test bat-tery procedure i followed, since, as has been pointedout, the tests tend to relate to one another. Yet,this redundancy would be substantially smaller thanthat expected in a test battery structured in a dif-ferent manner. Some overlap among predictors,however, would not be detrimental as long as it isnot too large and it increases the overall validityof the test battery as a whole (cf. Tziner & Dolan,1982).To conclude, the authors agree with those re-

searchers who maintain that the ultimate test forthe validity of findings is their recurrence in nu-merous studies (cf. James, I~~l~ilc9 ~ Brett, 1982).For this reason additional research is suggested inorder to accurately assess the extent of recurrenceof the findings, in this study. This is particularlythe case when comparing results from facet anal-ysis, employed here with those emanating fromother methods of structure exposure, such as Krus-kaFs multidimensional scaling analysis (l~r~sl~~l &

Wish, 1978).

References

Brainerd, C. J. (1978). Piaget’s theory of intelligence.Englewood Cliffs NJ: Prentice-Hall.

Cunningham, W. R. (1981). Ability factor structure dif-ferences in adulthood and old age. Multivariate Be-havioral Research, 16, 3-22.

Elizur, D. (1981). The importance and structure of joboutcomes. Unpublished manuscript, Bar-Ilan Univer-sity.

Elizur, D. (1982). Facets of work values: A structuralanalysis of work outcomes. Unpublished manuscript,Bar-Ilan University.

Gratch, H. (1973). Twenty-five years of social research:

The story of the Israel Institute of Applied Social Re-search. Jerusalem: Jerusalem Academic Press.

Guilford, J. P. (1980). Fluid and crystallized intelli-

gences : Two fanciful concepts. Psychological Bulle-tin, 88, 406-412.

Guilford, J. P. (1981). Higher-order structure-of-intel-lect abilities. Multivariate Behavioral Research, 16,411-435.

Guttman, L. (1958). What lies ahead for factor analysis?Educational and Psychological Measurement, 18, 497-515.

Guttman, L. (1968). A general nonmetric technique forfinding the smallest coordinate space for configurationof points. Psychometrika, 33, 465-506.

James, L. R., Mulaik, S. A., & Brett, J. M. (1982).Conditions for confirmatory analysis and causal in-ference. San Francisco: Sage Publications.

Kelderman, H., Mellenbergh, G. J., & Elshout, J. J.

(1981). Guilford’s facet theory of intelligence: Anempirical comparison of models. Multivariate Behav-ioral Research, 16, 37-61.

Kenny, C., & Canter, D. (1981). A facet structure fornurses’ evaluation of ward designs. Journal of Oc-cupational Psychology, 54, 93-108.

Kruskal, J. B., & Wish, M. (1978). Multidimensionalscaling. London/Beverly Hills CA: Sage Publications.

Lingoes, J. C. (1973). The Guttman-Lingoes nonmetricprogram series. Ann Arbor: Mathesis Press.

Piaget, J. (1956). The child’.s conception of space. Lon-don: Routledge & Kegan Paul.

Shapira, Z., & Zevulun, E. (1979). On the use of facetanalysis in organizational behavior research: Someconceptual considerations and an example. Organi-zational Behavior and Human Performance, 23, 411-428.

Shirom, A. (1982). What is organizational stress? A

facet analytic conceptualization. Journal of Occupa-tional Behavior, 3, 21-38.

Schlesinger, I. M., & Guttman, L. (1969). Smallest

space analysis of intelligence and achievement tests.Psychological Bulletin, 71, 95-100.

Shye, S. (Ed.) (1978). Theory construction and dataanalysis in the behavioral sciences. San Francisco:Jossey-Bass.

Tziner, A., & Dolan, S. (1982). Validity of an assess-ment center for identifying future female officers inthe military. Journal of Applied Psychology, 67, 728-736.

Acknowledgments

Both authors contributed equ~lly t® this articles. Theauthors th~~ak L. Guttman for his helpful comments onan earlier drc~f’t of this article.



69

Author’s Address

Send requests for reprints or further information to AharonTziner, Department of Labor Studies, Faculty of SocialSciences, Tel Aviv University, Ramat Aviv, Tel Aviv69978, Israel.



guttman’s model abilityguttman’s model of ability tests aharon tziner, tel-aviv university...

Documents