factor structure and validity of the macarthur … · stand trial vs. competency to plead guilty;...

Factor Structure and Validity of the MacArthur Competence AssessmentTool—Criminal Adjudication

Patricia A. ZapfThe City University of New York

Jennifer L. SkeemUniversity of California, Irvine

Stephen L. GoldingUniversity of Utah

Examination of the available literature regarding the development of the MacArthur CompetenceAssessment Tool—Criminal Adjudication (MacCAT–CA) reveals 2 theoretical factor structures onwhich the MacCAT–CA was based: one in which 3 lower-order constructs are proposed (under-standing, reasoning, appreciation) and one in which 2 higher-order constructs are proposed (com-petence to assist counsel and decisional competence). Confirmatory factor analyses were conductedwith the MacCAT–CA’s original normative sample (N � 729) to test both the relative fit ofthese 2 theoretical factor structures and models that combine the 2 factor structures. Analyses werealso completed to examine the convergent and discriminant validity of the MacCAT–CA. Resultsare discussed in terms of the strengths and weaknesses of the nomothetic nature of theMacCAT–CA.

Keywords: MacCAT–CA, competency to stand trial, forensic assessment, factor structure, validity

Adjudicative competence, traditionally labeled competencyto stand trial, is a jurisprudential concept with roots in Englishcommon law that allows for the postponement of criminalproceedings for those individuals who, because of mental im-pairment, are not able to take part in their own defense. Theevaluation of adjudicative competence is arguably the singlemost significant mental health inquiry pursued in criminal law(Nicholson & Kugler, 1991), in part because “more defendantsare evaluated for competency and more financial resources areexpended for their evaluation, adjudication, and treatment thanfor any other class of forensic activities” (Golding, 1992, p. 77).Mirroring the significance of this issue, several groups havedeveloped various specialized instruments to aid in the assess-ment of competency to stand trial over the past three decades(e.g., Golding, 1993; Golding, Roesch, & Schreiber, 1984;Lipsitt, Lelos, & McGarry, 1971; McGarry, 1965; McGarry &Curran, 1973; Roesch, Webster, & Eaves, 1984; Roesch, Zapf,Eaves, & Webster, 1998; Rogers, Tillbrook, & Sewell, 2004;Wildman et al., 1978).

Aspects of the Competency Construct That InformInstrument Evaluation

Three key aspects of the adjudicative competency construct arerelevant to the development and evaluation of competency assess-ment instruments. First, competency is a different construct thaneither intelligence or mental disorder. The presence of cognitivedisability or mental disorder is merely a threshold issue that mustbe established to “get one’s foot in the incompetency door” (Bon-nie, 1992; Golding & Roesch, 1988; Grisso, Appelbaum, Mulvey,& Fletcher, 1995; Skeem, Golding, Cohn, & Berge, 1998). Symp-toms are relevant only when linked with impairment in the abilitiesnecessary to understand legal proceedings and participate in one’sdefense. Based on this principle, scores on competency assessmentmeasures should be associated with measures of intelligence andpsychopathology only to the extent that the latter themselves arenomologically related to the competency construct. Because sub-groups of individuals with different levels of intelligence or psy-chopathology will produce different average levels of psycholegalabilities, there will be some valid covariation across these sub-groups (Meehl, 1995), as depicted in Areas 4 and 5 of Figure 1. Anideal measure of competency, however, would perfectly overlapthe variability in the psycholegal abilities associated with adjudi-cated competent or incompetent individuals (i.e., Areas 2 and 3 ofFigure 1 would be zero, Area 6 would be isomorphic with Area 7,and Area 4 would be isomorphic with Area 5). In the language oftest construction, “good” items on competency assessment instru-ments (Campbell, 1960; Krause, 1972) covary with intelligence ormental disorder only to the extent that such constructs covary withcompetency. In the language of Baron and Kenny (1986), thereshould exist an indirect path between measures of competency andIQ/psychopathology, mediated by the competency status of indi-

Patricia A. Zapf, Department of Psychology, John Jay College ofCriminal Justice, The City University of New York; Jennifer L. Skeem,Department of Psychology and Social Behavior, University of California,Irvine; Stephen L. Golding, Department of Psychology, University of Utah.

We thank Norman Poythress for providing the data set from which theseanalyses were run and for assisting us in terms of answering and clarifyingquestions that we had while writing this article.

Correspondence concerning this article should be addressed to PatriciaA. Zapf, Department of Psychology, John Jay College of Criminal Justice,The City University of New York, 445 West 59th Street, New York City,NY, 10019-1128. E-mail: [email protected]

Psychological Assessment Copyright 2005 by the American Psychological Association2005, Vol. 17, No. 4, 433–445 1040-3590/05/$12.00 DOI: 10.1037/1040-3590.17.4.433

433

viduals, but there should be no direct path between a measure ofcompetency and measures of IQ/psychopathology.

The second relevant aspect of the competency construct is thatno fixed set of psycholegal abilities can define it (Drope v. Mis-souri, 1975; Poythress et al., 1999; Roesch & Golding, 1980). Theabilities required to competently stand trial vary, depending on thefunctional context of the case (e.g., the defendant, the charges, theevidence, available trial strategies; see Golding, 1993; Skeem,Golding, & Emke-Francis, 2004). The open-textured nature of thecompetency construct raises questions about highly structuredcompetency assessment instruments (e.g., Lipsitt et al., 1971;Wildman et al., 1978). Some competency assessment instruments(e.g., Golding, 1993; Roesch et al., 1998; Rogers et al., 2004)adopt a structured clinical judgment approach, directing evalua-tors’ attention to the range of psycholegal competence abilities(e.g., appreciation of charges, understanding and reasoned choiceof legal options) established by case law and the case context.Though structured, these instruments prescribe case-specific in-quiries with defendants, based on their charges and likely legalstrategies, and yield idiographic scores based on subjective ratings.

One of the most recently developed instruments, the MacArthurCompetence Assessment Tool-Criminal Adjudication (MacCAT–CA; Hoge, Bonnie, Poythress, & Monahan, 1999; Poythress et al.,1999) adopts a newer, standardized approach to assessment inwhich evaluators ask uniform questions of each defendant thatlargely reference a hypothetical case vignette. This instrumentyields nomothetic, criterion-based scores that may be comparedwith normative data. The MacCAT–CA developers, however, em-phasize that the instrument was designed for use as a tool ratherthan a test of competence. Scores are meant to be interpretedwithin the context of the defendant’s case and integrated with morecase-specific inquiries.

The third key principle is that standards of competence are onearea of inquiry, whereas competence conceptualizations that un-derlie assessment tools are another. There is no constitutionalrequirement for a standard of competency to differ across suchdifferent adjudicative competency contexts (e.g., competency tostand trial vs. competency to plead guilty; Godinez v. Moran,1993; see also Cooper v. Oklahoma, 1996). Although the

MacCAT–CA, for example, separates competence abilities intotwo (competence to assist counsel [CAC] and decisional compe-tence [DC]) or three (understanding, reasoning, and appreciation)domains, this is not inconsistent with current competence stan-dards. An important corollary of this principle is that empiricalsupport for the MacCAT–CA’s two- or three-domain conceptual-ization of competency bears strongly on the instrument’s validitybut cannot impart the “truth” about the construct that underliescompetency standards (Roesch, Hart, & Zapf, 1996). Tests thatoperationalize specific conceptualizations of competency assume aparticular pattern of relations among items across individuals andare not immune to examination based on the extent of empiricalsupport for their hypothesized factor structure. However, there areclear limits to what that factor structure imparts about the truenature of the competency standard.

The MacCAT–CA

In this paper, these three nomological aspects of the competencyconstruct are considered in light of the results of an analysis of thefactor structure and validity of the MacCAT–CA. TheMacCAT–CA is a second-generation instrument that was devel-oped to address a number of the limitations of other, first-generation instruments. It was, in part, a response to Grisso’s(1992) call for an “instrument offering standardized administrationand scoring (as contrasted with CST interview guides and subjec-tive ratings) to assess the domain of CST-related abilities for thegeneral population of defendants who are referred for CST eval-uations” (pp. 366–367). Unlike past measures, the MacCAT–CAincludes standardized administration, criterion-based scoring, anda systematic distinction between defendants’ existing legal knowl-edge and capacity to attain such knowledge. Moreover, theMacCAT–CA was developed on the basis of a strong theoreticalfoundation (Hoge, Bonnie, et al., 1997).

The MacCAT–CA is based on a comprehensive research instru-ment (Hoge, Poythress, et al., 1997) that was developed, pilottested, and refined prior to administration in a large field study(Bonnie et al., 1997; Hoge, Bonnie, et al., 1997; Hoge, Poythress,et al., 1997; Otto et al., 1998). This field study yielded theMacCAT–CA, which consists of 22 items grouped into threesubscales of psycholegal abilities: Understanding, Reasoning, andAppreciation. An evaluator begins the MacCAT–CA by reading ahypothetical vignette to the defendant. The first subscale (eightitems) assesses the defendant’s ability to understand informationabout the legal system and the process. For each item, the defen-dant is asked a question grounded in the vignette (e.g., “What is thejob of the attorney for the defense”) and is awarded 2 points (itemsare rated 0, 1, 2) if he or she demonstrates full understanding. If thedefendant earns less than 2 points, the examiner discloses theanswer and asks the defendant to repeat the disclosure in his or herown words, enabling separate assessment of the defendant’s ca-pacity to understand and his or her preexisting understanding.

The second section (eight items) assesses the defendant’s abilityto reason. The first five items in this section assess the defendant’sability to choose from among two options the most legally relevantpiece of information that the hypothetical defendant in the vignetteshould disclose to his lawyer. The last three items require thedefendant to evaluate mock legal options for the hypotheticaldefendant.

Figure 1. Conceptual analysis of the overlap of measures. Numbersdesignate overlapping and nonoverlapping areas. MacCAT–CA �MacArthur Competence Assessment Tool—Criminal Adjudication.

434 ZAPF, SKEEM, AND GOLDING

The third section (six items) assesses the defendant’s ability toappreciate his or her own legal situation. This section departs fromthe hypothetical vignette to explore, in a standardized manner, thedefendant’s appraisal of how he or she is likely to function and tobe treated during the course of adjudication, relative to otherdefendants (e.g., whether he or she is likely to be treated more,less, or about as fairly as other defendants). These items are scoredon the basis of the plausibility (vs. delusionality) of the grounds forthe defendant’s judgment.

The psychometric properties of the MacCAT–CA were exam-ined based on a sample of 729 felony defendants in eight differentstates (Otto et al., 1998; see also Rogers, Grandjean, Tillbrook,Vitacco, & Sewell, 2001). The results indicated that theMacCAT–CA demonstrated good reliability. For each of the threesections, internal consistency ranged from .81 to .88 (� � .81 forReasoning, .85 for Understanding, and .88 for Appreciation), andinterrater reliability ranged from very good to excellent (intraclassR � .75 for Appreciation, .85 for Reasoning, and .90 for Under-standing). Otto and colleagues (1998) report that support for theconstruct validity of the MacCAT–CA was “found in the pattern ofcorrelations between the MacCAT–CA measures and select clin-ical variables” (p. 439). MacCAT–CA Understanding, Reasoning,and Appreciation scores correlated .41, .34, and .14, respectively,with estimated Wechsler Adult Intelligence Scale—Revised(WAIS–R; Wechsler, 1981) full scale IQ, and �.23, �.29, and�.36, respectively, with Brief Psychiatric Rating Scale (BPRS;Overall & Gorham, 1962) total scores. Whether such relationshipswith measures of intelligence and psychopathology are theoreti-cally desirable characteristics for a measure of competency isdiscussed later. The three MacCAT–CA scales correlated moder-ately with clinicians’ global ratings of competency (r � .36, .42,and .49, respectively) for a subset of defendants who had beenhospitalized as incompetent.

Theoretical Underpinnings of the MacCAT–CA

Three factors. The MacCAT–CA’s three-scale structure ispartially rooted in the MacArthur group’s review of the literatureon competency to consent to treatment. Despite differences interminology, four commonly accepted “prongs” of the generalconstruct of competence are apparent (see Appelbaum & Grisso,1988, 1995; Appelbaum, Lidz, & Meisel, 1987; Appelbaum &Roth, 1982; Roth, Meisel, & Lidz, 1977), including the ability to(a) express a choice, (b) understand relevant information, (c)appreciate the personal importance of the situation, and (d) ratio-nally manipulate information. The MacCAT–CA assesses the threelatter prongs of competence—understanding, appreciation, andreasoning.

Two factors. Bonnie (1992, 1993) expanded on Winick’s(1987) work to draw parallels between standards of treatmentcompetency and criminal competency. Specifically, he conceptu-alized adjudicative competence in terms of a foundational con-struct (CAC) and a related contextualized construct (DC). Thefoundational construct of CAC refers to “the minimum conditionsrequired for participating in one’s own defense” (p. 297) to pre-serve the dignity and reliability of the criminal process. It includesthe individual’s:

(i) capacity to understand the charges, the purpose of the criminalprocess and the adversary system, especially the role of the defense

counsel; (ii) capacity to appreciate one’s situation as a defendant inthe criminal prosecution; and (iii) ability to recognize and relatepertinent information to counsel concerning the facts of the case(Winick, p. 243–287).

In contrast with this foundational construct, the contextualizedconstruct of DC encompasses the defendant’s ability to makedecisions and behave appropriately at a trial. According to Bonnie(1992), decisional competence is separable from competence toassist counsel: “although the relevant psychological capacities mayoverlap, decision-making about defense strategy encompasses con-ceptual abilities, cognitive skills, and capacities for rational think-ing that are not required for assisting counsel” (p. 305). Tests ofDC should be contextualized or tailored to the specific situationand the decision that is required. Bonnie’s reformulation of theconcept of adjudicative competency was in large part the founda-tion for the MacCAT–CA. Each section of the MacCAT–CAcontains items that tap both CAC and DC.

Competing or complementary models? Given the develop-ment and theoretical underpinnings of the MacCAT–CA, thereappear to be two distinct, perhaps competing, theoretical factorstructures for the instrument: the three-factor structure (composedof understanding, appreciation, and reasoning) that corresponds tothe MacCAT–CA designed scales, and Bonnie’s two-factor struc-ture (comprising CAC and DC). Although these two models wereexplicitly viewed as complementary in an early version of theinstrument, the relation between them became blurred with thecreation of the final MacCAT–CA. The forerunner of theMacCAT–CA blended the three-factor model with Bonnie’s two-factor model. This crossing of three functional ability domains(understanding, reasoning, and appreciation) with two legal do-mains or levels of competency (CAC, DC) produced six scales(Hoge, Bonnie et al., 1997). Notably, however, an exploratoryfactor analysis suggested this forerunner instrument was unidimen-sional (see Cruise & Rogers, 1998). In contrast, the finalMacCAT–CA was simply offered as a three-factor instrument. Aclose examination of the instrument, however, reveals vestiges ofthe two-factor model. In some way not described in publishedmaterials, the MacCAT–CA’s authors appear to have sampled bothCAC and DC items from the three functional domains in a mannerthat conflates the three-factor model with Bonnie’s two-factormodel. The authors note only that “the . . . MacCAT–CA willemploy items derived from the [research instrument] and [will] bemuch shorter and user-friendly” (Hoge, Bonnie, et al., 1997; seealso Otto et al., 1998).

In summary, during the development of the MacCAT–CA, theauthors strongly emphasized the importance of producing a mea-sure with a clear conceptual structure (e.g., Hoge, Bonnie et al.,1997; Otto et al., 1998). Although the research instrument explic-itly crossed three functional abilities and two legal domains, therewas a conceptual leap to the MacCAT–CA, which is offered astapping three functional abilities (without the relation of theseabilities to the two legal domains). The MacCAT–CA may have athree-factor structure, a two-factor structure, or a structure thatcrosses both models. One such possible “crossed” structure in-volves construing understanding, appreciation, and reasoning astypes of abilities that may each be exercised within two contentdomains—competency to assist counsel and decision competence(see Bonnie & Grisso, 2000).

435FACTOR STRUCTURE OF THE MACCAT–CA

Investigating the MacCAT–CA’s Factor Structure

Rogers and his colleagues (2001) completed exploratory factoranalyses of the MacCAT–CA, based on 149 mentally disorderedoffenders, and found a two- or three-factor solution. The three-factor solution grossly conformed to the designed three-factormodel (understanding, reasoning, and appreciation), whereas thetwo-factor solution combined the instrument’s understanding andreasoning scales. The authors raised the possibility that the latterfinding was based on method variance, given that the understand-ing and reasoning items are based on the hypothetical case vi-gnette, whereas the appreciation scale is more case specific. Al-though this study is a welcome investigation of the factor structureof the MacCAT–CA, it is based on two small samples of conve-nience and does not apply confirmatory factor analytic techniques,which are most appropriate given the MacCAT–CA’s theoreticalbasis.

The chief purpose of this article is to examine the relative fit oftwo theoretical factor structures (separately and in combination) tothe MacCAT–CA normative data. Our secondary aim is to assessaspects of the MacCAT–CA’s construct validity by estimating theextent to which it taps aspects of severe mental illness and dis-ability that do not overlap with competence deficits per se.

Method

The dataset used in this study was obtained from the researchers whocompleted the original MacCAT–CA validation and normative study. Themethod of the original MacCAT–CA validation study is described in detailelsewhere (Otto et al., 1998; Poythress et al., 1999). Briefly, theMacCAT–CA was normed on a sample of 729 felony defendants in eightdifferent states. Three subgroups of defendants were recruited, includingJail Unscreened (JU), Jail Treated (JT), and Hospitalized Incompetent (HI)groups. The JU subgroup consisted of 197 randomly selected jail inmateswho were presumed to be competent to proceed (in that the issue ofadjudicative competency had not been raised in their cases). The JailTreated (JT) subgroup consisted of 249 jail inmates presumed to becompetent but who were receiving mental health services for reasonsunrelated to competency. These two groups of jail inmate participants (JUand JT) were recruited from lists of defendants provided by the publicdefenders’ offices in each of the eight states. The refusal rate was 9% foreach of the two jail inmate subgroups. The HI subgroup consisted of a totalof 283 defendants who had been admitted to forensic psychiatric units forcompetency restoration after having been adjudicated incompetent to pro-ceed. Potential HI participants were identified as they entered the facilityfor restoration and were recruited within 14 days of admission. The refusalrate was 22% for the HI subgroup.

The HI group was significantly older than the participants in the JU andJT samples and Caucasian and non-Caucasian participants were “some-what disproportionately distributed across the three samples” (p. 437). Inaddition, the HI group obtained higher scores on the Minnesota Multipha-sic Personality Inventory—2 Psychoticism scale and on the BPRS. Thethree groups were similar in gender, education, socioeconomic status,amount and type of involvement with the criminal justice system, andestimated current cognitive–intellectual functioning.

The study procedure consisted of a one-time interview in which defen-dants completed a variety of measures that included the MacCAT–CA, theBPRS (Overall & Gorham, 1962), and the Information and Picture Com-pletion subtests of the WAIS–R (Wechsler, 1981). These subtests wereused to estimate full scale WAIS–R intelligence scores.

The MacCAT–CA consists of 22 items with each item being scored as0, 1, or 2. The scores for the subscales, therefore, range from 0–16 forunderstanding, from 0–16 for reasoning, and from 0–12 for appreciation.

Results

The aims of this study were addressed in three stages. First,confirmatory factor analyses (CFAs, [maximum likelihood estima-tion1], performed in LISREL 8.30) were completed to test therelative fit of the theoretical three-factor model, two-factor model,and combined two- and three-factor models. Second, modificationindices from the CFAs were used to determine the best fitting,theoretically logical model for the data. Third, supplementary scaleanalyses were performed to examine the construct validity of theMacCAT–CA, including the nature and degree of associationbetween MacCAT–CA scale scores and measures of intelligenceand psychopathology. The results of these three stages of analysesare presented below. In each section, the results of the analyses forthe combined sample are presented first, followed by the results forthe JU, the JT, and HI subgroups.

Confirmatory Factor Analyses (A Priori Models)

First, we tested the two-factor, three-factor, and combined two-and three-factor models using CFA. For these analyses, we reportseveral relevant indices of goodness of fit for each model becauseeach index addresses a slightly different issue, and good-fittingmodels will produce consistent results across many indices (Ull-man, 2001). Specifically, for each model, we report (a) absolute fitindices (�2, �2/df, Goodness of Fit Index; GFI), (b) relative fitindices (Comparative Fit Index [CFI], Normed Fit Index [NFI]),(c) parsimoniousness fit indices (Akaike Information Criterion[AIC]; Consistent Akaike Information Criterion [CAIC]), and (d)non-centrality-based indices (root-mean-square error of approxi-mation; RMSEA—for overviews, see Bentler, 1995; Church &Burke, 1993; Kline, 1998; MacCallum & Austin, 2000; Ullman,2001). Because the CFI and NFI indices are not easy to interpretwe note “models with overall fit indices of less than .90 canusually be improved substantially” (Bentler & Bonett, 1980, p.600). In other words, minimum values of .90 for CFI and NFIstatistics are conventionally required to indicate even “adequate,”“satisfactory,” or “acceptable” fit (Byrne, 1994; Kline, 1998; Ul-lman, 2001). In fact, Hu and Bentler (1999) recently analyzed theability of various statistics to distinguish between true populationand misspecified models at different thresholds and recommendeda higher cutoff of .95 for the CFI. To ensure consistency in thisarticle, we used the following heuristic labels to describe fit:inadequate when most values of the GFI, CFI, and NFI are below.90 (RMSEA � .10), adequate when most values are between .90and .93 (RMSEA � .05–.10), and reasonably good when mostvalues are at or near .95 (RMSEA � .05).

Three-factor model. First, we tested the fit of the three-factormodel, given that this model is explicitly set forth for theMacCAT–CA in publications (Otto et al., 1998; Poythress et al.,1999). We tested a correlated three-factor model composed of

1 Given the ordinal nature of the items, we attempted to determinewhether maximum likelihood estimation would provide estimates thatwould differ significantly from more robust methods. We ran the boot-strapping technique available in Amos 5.0 (Assessment Systems; St. Paul,MN) on these data, with 1,000 samples, in an attempt to correct fornon-normally distributed data. With this procedure, we found the exactsame results as reported using maximum likelihood estimation.


three latent variables that directly corresponded to the three sec-tions of the MacCAT–CA. The first latent variable (understanding)consisted of Items 1–8, the second (reasoning) consisted of Items9–16, and the third (appreciation) consisted of Items 17–22. Thisthree-factor model appeared to be a reasonably good fit of the datafor the full sample (see Figure 2; Table 1). When this same modelwas tested using each of the three subsamples, its fit was found tobe inadequate for the JU and JT subgroups but adequate for the HIsubgroup.

An examination of the correlations between the three factorsreveals strong relationships between understanding and reasoning(r � .92), reasoning and appreciation (r � .73), and understandingand appreciation (r � .62). This suggests the presence of a strongsecond-order factor that runs though the entire instrument. Giventhat respecifying the model in terms of three first-order factors thatload on a single second-order factor would result in identical fitindices, we instead tested for the presence of a single second-orderfactor by testing the difference between a correlated-factors modeland an orthogonal-factors model (Gorsuch, 1983). Specifically, wespecified the three first-order factors in two models—one in whichthe correlations between the first-order factors are left free to varyand one in which the correlations are set to zero. The correlated

factors model fit significantly better than the orthogonal factorsmodel, suggesting the existence of a higher order structure, �diff

2

(3) � 876.94, p � .0001 (see Table 1). In addition, we also testedthree variations of a correlated-factors model (wherein we sequen-tially set to 1 the correlation between each of the three possiblepairs of first-order factors) to test for the independent presence ofeach of the three first-order factors. The results suggest that eachfirst-order factor is necessary to avoid a decrement in fit, differ-ence in �2(1) � 1449.75, 57.29, and 606.27, all ps � .001. Takentogether, these results suggest the presence of three distinct first-order factors and an overarching second-order factor.2

Bonnie’s two-factor model. The MacCAT–CA’s primary au-thor provided us with a description of the MacCAT–CA items thatrelate to Bonnie’s DC and CAC factors (Norman Poythress, per-sonal communication, December 5, 2001). This two-factor model(see Figure 3) was composed of two latent variables. The first

2 Although comparison of an oblique solution with an orthogonal onedoes not technically prove the existence of a single higher order factor, wewill use the terms first-order and second-order factors as a heuristicconvenience in the remainder of this article.

Figure 2. Loadings for the correlated three-factor model. MC � MacArthur Competence Assessment Tool—Criminal Adjudication item number; RMSEA � root-mean-square error of approximation.


variable (DC) was composed of six items—two (Items 7 and 8)from the Understanding section, three (Items 14, 15, and 16) fromthe Reasoning section, and one (Item 22) from the Appreciationsection. The second factor (CAC) consisted of all 16 remainingitems on the MacCAT–CA. This two-factor model inadequately fitthe data for the full sample (both when the correlation between thefactors was left free to vary as well as when it was set to zero) aswell as data for each of the three subgroups (see Figure 3 andTable 1).

Combined two- and three-factor models. Given that the fore-runner to the MacCAT–CA explicitly crossed two- and three-factor models, we also wished to simultaneously test the fit ofthese two models. Thus, we did this in three different ways. First,we partially crossed the three functional abilities (understanding,reasoning, appreciation; see Figure 2) with the legal domains(CAC, DC; see Figure 3) to simultaneously test the fit of thesemodels (in a correlated five-factor model). That is, we allowedeach of the 22 items to cross-load on two factors—one of the threefunctional abilities (understanding, appreciation, reasoning) andone of the two legal domains (CAC, DC). This model is consistentwith construing understanding, appreciation, and reasoning astypes of abilities that may each be exercised within the contentdomains of competency to assist counsel and decision competence(see Bonnie & Grisso, 2000).

Second, we allowed each of the 22 items to load onto one of sixfirst-order factors (understanding CAC, understanding DC, appre-ciation CAC, appreciation DC, reasoning CAC, reasoning DC) andthen each of these first-order factors was allowed to load on one ofthe three second-order functional ability factors (understanding,appreciation, or reasoning) and one of the two second-order legaldomains (CAC, DC). Finally, given that item-level analyses cansometimes be problematic, we formed six scales from the 22 itemsas per the direction of the developers of the MacCAT–CA(UCAC � Items 1–6, UDC � Items 7–8, RCAC � Items 9–13,RDC � Items 14–16, ACAC � Items 17–21, ADC � Item 22)and then allowed each of these six scales to load on one of thethree functional abilities and one of the two legal domains. Noneof these three different models fit the data. That is, there was eithera problem with the identification of the model and thus it did notconverge, or if the number of iterations was increased to beyond

what is considered appropriate so as to allow for convergence (seeJoreskog & Sorbom, 1993), the solutions did not make sense asevidenced by loadings greater than 1.

Comparing the relative fit of the theoretical models. Giventhese problems with the combined two- and three-factor modelspecifications, we were unable to compare them with the individ-ual two- and three-factor models. The difference in fit between thethree-factor model and Bonnie’s two-factor model suggested thatthe three-factor model fit the data better for both the full sample(an adequate vs. inadequate fit) and the three subgroups. Unfortu-nately, because these models were not nested, we could not sta-tistically compare their relative fit.

Notably, although the three-factor model adequately fit the datafor the HI subgroup, even this model inadequately fitted the JU andJT subgroups. This difference does not appear attributable tosample size differences, given that each subgroup consisted ofapproximately 200 participants (HI � 283, JU � 197, JT � 249),which is sufficient for small to medium models (Ullman, 2001).This difference in fit may, however, be attributable to differencesin the distributions of MacCAT–CA scores across subgroups.Specifically, there was more variability in MacCAT–CA scores forthe HI subgroup (i.e., the “incompetent” or “more severely ill”group) than for the JT and JU subgroups (i.e., the “presumedcompetent” samples). MacCAT–CA scores approximated a normaldistribution in the HI subgroup, but were substantially more neg-atively skewed and kurtotic in the JT and JU subgroups.3 The HIsubgroup’s closer approximation to a normal distribution than theJU and JT subgroups mainly reflects the fact that a greater numberof individuals in the former group scored at the lower end of thedistribution, although the overall range of scores for each of thethree groups did not appear to be glaringly restricted.

3 Means: JT � 36.6 (SD � 6.6), JU � 37.2 (SD � 5.7), HI � 26.5(SD � 10.5). Kurtosis: JT � 1.83, JU � 3.38, HI � �.55. Skewness: JT ��1.48, JU � �1.56, HI � �.50.

Table 1Goodness of Fit (GFI) Indices for Models Tested

Model �2 df �2/df CFI NFI GFI AIC CAIC RMSEA

Three-factor model (correlated) 472.6 206 2.29 .96 .93 .94 566.6 829.4 .042Three-factor model (orthogonal) 1349.6 209 6.46 .80 .78 .86 1437.6 1683.6 .087Bonnie’s two-factor model (correlated) 2041.5 208 9.81 .83 .80 .80 2131.5 2383.2 .110Bonnie’s two-factor model (orthogonal) 2426.7 209 11.61 .69 .67 .77 2514.7 2760.7 .120Three-factor model (JU) 520.76 206 2.53 .63 .54 .81 614.8 816.1 .088Bonnie’s two-factor model (JU) 544.47 208 2.62 .62 .52 .80 634.5 827.2 .091Three-factor model (JT) 348.77 206 1.69 .88 .77 .89 442.8 655.1 .053Bonnie’s two-factor model (JT) 414.89 208 1.99 .85 .74 .87 504.9 708.2 .063Three-factor model (HI) 301.73 206 1.46 .95 .88 .91 395.7 614.1 .041Bonnie’s two-factor model (HI) 1165.3 208 5.60 .76 .70 .73 1255.3 1464.4 .128Modified models

Collapsed two-factor model 529.9 207 2.56 .95 .92 .94 621.90 879.12 .046Modified three-Factor model 348.37 204 1.71 .98 .95 .96 446.37 720.36 .031Three-factor model with error terms correlated 389.74 204 1.91 .97 .94 .95 487.71 761.72 .035

Note. CFI � Comparative Fit Index; NFI � Normed Fit Index; AIC � Akaike Information Criterion; CAIC � Consistant Akaike Information Criterion;RMSEA � root-mean-square error of approximation; JU � jail unscreened group; JT � jail treated group; HI � hospitalized incompetent group.


Confirmatory Factor Analyses & Schmid–LeimanTransformation (Modified Models)

In summary, the first stage of analysis suggested that a hierar-chical three-factor model fit the data better than the two-factormodel. To determine whether a particularly good fit could beobtained with alternative models, the second stage of analysisinvolved examining modification indices from the three-factorCFA in an attempt to develop a better-fitting model. The modelswere developed and tested in a stepwise manner.

Collapsed two-factor model. The results of the correlatedthree-factor CFA suggested that the understanding and reasoningfactors were very strongly correlated (r � .92). Because these twofactors appeared virtually redundant, we combined them into asingle factor (i.e., fixed the correlation between factors to 1) andtested the fit of this new two-factor model. Although the results(see Table 1) indicated that this model (understanding/reasoningand appreciation) was a reasonably good fit to the data, a chi-square comparison of this model’s fit with that of the originalthree-factor model suggested that combining the understandingand reasoning factors significantly reduced the model’s fit,��2(2) � 57.29, p � .001.

Modified three-factor model. An examination of the items’factor loadings suggested that Items 9–13 tended to load bythemselves and apart from the other items on the understanding/reasoning factor. Thus, we constructed a modified three-factormodel: the first factor included Items 1–8 and 14–16 (i.e., theoriginal Understanding subscale plus Items 14–16), the secondfactor included Items 9–13, and the third factor included Items17–22 (i.e., the original Appreciation subscale). In this model, weallowed the error terms for Items 2 and 3 and Items 15 and 16 tocorrelate with each other, given that these two item pairs werehighly similar in nature and format to one other. Specifically,Items 2 and 3 ask about charges (i.e., aggravated and simpleassault, respectively), and Items 15 and 16 ask about advantagesand disadvantages for pleading one way or another. Given ourearlier finding that there did appear to be evidence to suggest asecond-order factor (in addition to the three first-order factors), weincluded this second-order factor in the representation of thismodel (see Figure 4 and Table 1). This modified three-factormodel provided the best overall fit to the data.

This modified three-factor model differs from the theoreticalthree-factor model in two respects. First, the model collapses items

Figure 3. Loadings for Bonnie’s two-factor model. MC � MacArthur Competence Assessment Tool—Criminal Adjudication item number; CAC � competence to assist counsel; DC � decisional competence;RMSEA � root-mean-square error of approximation.


from the Understanding section and Items 14–16 from the Rea-soning section (i.e., the items that tap decisional competence) intoone factor, places Items 9–13 from the Reasoning section into asecond factor, and keeps all items from the Appreciation section asthe third factor. Second, the model allows error terms for two itempairs to correlate. To determine the extent to which the better fit ofthe modified than the theoretical three-factor model was attribut-able to substantive item changes or to permitting error terms tocorrelate, we conducted a CFA of the theoretical three factormodel in which we permitted error terms for Items 2 and 3 andItems 15 and 16 to correlate. The results (see Table 1; three-factormodel with error terms correlated) suggest that the fit of thetheoretical three-factor model (e.g., CFI � .96) manifests modestimprovement when error terms are permitted to correlate (e.g.,CFI � .97) and additional modest improvement if substantive itemchanges are made (e.g., CFI � .98). In short, both aspects of thenew model contribute modestly to improved fit.

Schmid–Leiman transformation. Given that a hierarchicalthree-factor structure appeared to best fit the data, an exploratoryhigher order factor analysis4 was performed using the Schmid andLeiman (1957) orthogonalization procedure to more closely ex-

plore the measure’s hierarchical factor structure. The results pro-vide additional evidence suggestive of a strong second-order factor(accounting for 72.3% of the variance) and three first-order fac-tors, as well as the best fitting modified three-factor model. Withrespect to the latter point, Items 14–16 loaded on the Understand-ing subscale of the MacCAT–CA in this analysis (factor loadingsare provided in Table 2).

Association of the MacCAT–CA With Measures ofIntelligence and Psychopathology

In summary, the second stage of analysis involved developing amodified three-factor model and determining that it, like the the-

4 A program called Secondor, developed by Bruce Thompson, ResearchProfessor of Education, University of New Orleans, was used for thishigher order factor analysis and Schmid–Leiman orthogonalization proce-dure. This program first computes first-order principal components rotatedto the promax criterion, then computes second-order principal componentsrotated to varimax criterion, then computes uncentered factor scores fromthe product matrix and the Schmid–Leiman solution.

Figure 4. Loadings for modified three-factor model. MacCompe � overarching competence construct mea-sured by the MacArthur Competence Assessment Tool—Criminal Adjudication (MacCAT–Ca); MC �MacCAT–CA item number; RMSEA � root-mean-square error of approximation.


oretical three factor model, provided a reasonably good fit to thedata. In a third stage of analysis, we performed supplementaryanalyses to determine the degree of association between scores onscales constructed from the theoretical three-factor model andmeasures of intelligence and psychopathology. As explained in theintroduction, the degree of association between the MacCAT–CAand measures of IQ and psychopathology are high enough tojustify further investigation of the scale’s discriminant validity.

Basic relation to intelligence and psychopathology. To esti-mate the basic relations between the MacCAT–CA factors on onehand and intelligence and psychopathology on the other, tworegression analyses were performed. First, all three MacCAT–CAsubscales were entered in a single step as predictors of WAIS IQscores. The squared semipartial correlation for each subscale wasused to estimate the variance uniquely accounted for by eachfirst-order factor (understanding, reasoning, and appreciation), andthe difference between the sum of these correlations and theincrement in variance accounted for on that step of the regressionwas used to estimate the variance uniquely accounted for by thesecond-order factor.5 Second, this process was repeated, usingBPRS total scores as the criterion. The results, shown as semipar-tial correlations in Table 3, suggest a moderate relation betweenthe MacCAT–CA and intelligence and psychopathology.

Relation to competence-relevant intelligence and psychopathol-ogy. Examining correlations between the MacCAT–CA andmeasures of psychopathology (BPRS total scores) and intelligence(WAIS IQ estimate) estimates Areas 2 and 5 of Figure 1 and thusconfounds covariation that reflects valid covariance (Area 5) with

covariation that is undesirable (Area 2). For this reason, we appliedBaron and Kenny’s (1986) framework to examine the size of Area2. Again, there should be an indirect path between theMacCAT–CA and intelligence/psychopathology, mediated by thecompetency status of individuals. There should, however, be nodirect path between the MacCAT–CA and measures of intelli-gence/psychopathology. Area 2 represents the direct path betweenthe MacCAT–CA and measures of IQ/psychopathology, whereasArea 5 represents the indirect path between these measures medi-ated by competency status. We used group status6 (HI vs. JT/JU)as an indicator of differences in psycholegal abilities related tocompetency status.

The mediation strategy was applied as a method for estimatingthe direct and indirect relation between the MacCAT–CA andintelligence or psychopathology rather than as a test of a causalmodel. A series of six regression analyses were performed (seeBaron & Kenny, 1986) to yield the path coefficients (a, b, c, andc�) necessary to test MacCAT–CA (total scores) mediation modelsfor intelligence (as indexed by WAIS IQ estimates) and psycho-pathology (as indexed by BPRS total scores). Where theMacCAT–CA is X, competency is M, and intelligence or psycho-pathology is Y, the traditional paths are a (relation between X andM), b (relation between M and Y, controlling for X), c (directrelation between X and Y), and c� (relation between X and Y,controlling for M). These coefficients were checked against theresults of a model assessed with Amos 5.0.

Intelligence (WAIS IQ). MacCAT–CA total scores and scoresof intelligence were moderately related (path c � .35). Thisrelationship was weakly, but significantly, mediated by compe-tence status (c-c� � .08, p� .001; Sobel, 1982). As shown inFigure 5, the mediated effect was small (path b � �.15), partic-ularly relative to the direct relation between the MacCAT–CA andintelligence (path c� � .43). Once competency status is taken into

5 We remind the reader here that we are using the term second-orderfactor as a heuristic convenience, as we have not technically proven theexistence of a single, second-order factor but the existence of a higher orderfactor structure has been suggested by the results of our analyses.

6 Competence status (HI v. JU/JT) is an imperfect measure of truecompetence. It was, however, the best measure available for this study andrepresents judicial decisions made (vs. not raised) about these defendants.

Table 2Results of the Schmid and Leiman (1957) OrthogonalizedHigher Order Analysis

MacCAT–CAitem

Second-orderfactor solution

First-order factor solution

Understanding Appreciation Reasoning

1 .558 .340 �.025 .0232 .545 .376 �.045 �.0063 .464 .457 �.056 �.1344 .645 .337 �.031 .0895 .610 .331 �.017 .0596 .669 .341 .017 .0567 .610 .257 .024 .1018 .680 .351 .026 .0469 .683 .131 �.010 .313

10 .690 .136 .059 .25111 .684 .098 .071 .27612 .654 .016 �.024 .42813 .562 �.069 �.012 .44814 .445 .265 .042 �.03115 .638 .278 .137 �.00316 .591 .252 .120 .00817 .638 .015 .462 �.01418 .627 �.004 .413 .04219 .598 .017 .453 �.03420 .655 �.024 .501 .00421 .629 �.033 .495 .00122 .687 .058 .415 .014

% varianceexplained 72.3 10.8 11.2 5.7

Note. Bold numbers indicate items with the highest loadings on eachfactor. MacCAT–CA � MacArthur Competence Assessment Tool—Crim-inal Adjudication.

Table 3Relation of MacCAT–CA Scales and General Factor toIntelligence and Psychopathology

MacCAT–CA indicator

Intelligence(WAIS-IQ)

Psychopathology(BPRS total)

r semi-partial r semi-partial

First-order factors (scales)Understanding .41 .26 �.23 .01Reasoning .34 .09 �.29 �.07Appreciation .14 �.12 �.29 �.24

Second-order factor — .30 — �.26Total association .35 .42 �.33 �.36

Note. MacCAT–CA � MacArthur Competence Assessment Tool—Criminal Adjudication; WAIS � Wechsler Adult Intelligence Scale;BPRS � Brief Psychiatric Rating Scale.


account, there is a sizable independent relation between theMacCAT–CA and WAIS-IQ scores.

Psychopathology (BPRS total). MacCAT–CA total scores andscores of psychopathology were moderately and inversely related(path c � �.24). This relationship was weakly, but significantly,mediated by competence status (c-c� � .09, p� .001; Sobel, 1982).As shown in Figure 6, the mediated effect (path b � .17) wasweaker than the direct relation between the MacCAT–CA andpsychopathology (path c� � �.24). Again, once competency statusis taken into account, there is a significant independent relationbetween the MacCAT–CA and psychopathology.

Discussion

This study was designed to examine the factor structure of theMacCAT–CA to determine which theoretical or empirically de-rived model best fit its normative data and to explore this mea-sure’s construct validity by assessing the extent to which it tapssevere mental illness and disability rather than competence deficitsper se. Each of these purposes is discussed in turn below, followedby an analysis of adjudicative competence as a nomotheticconstruct.

Which Factor Structure Best Fits the Data?

With respect to the question regarding which of the theoreticalfactor structures best fit the MacCAT–CA normative data, it ap-pears that the three-factor model better fits these data than Bon-nie’s two-factor model. In addition, the results of both exploratoryfactor analyses (Schmid–Leiman transformation) and confirmatoryfactor analyses (comparison of correlated and orthogonal three-factor models) indicate the possibility of a second-order generalfactor running through the MacCAT–CA such that its structure ishierarchical.

Although the original three-factor model that reflects the threesections of the MacCAT–CA fits the data reasonably well, wefound the best overall fit in a modified three-factor model in whichall items from the Understanding section and Items 14–16 fromthe Reasoning section are collapsed into a first factor, Items 9–13from the Reasoning section compose the second factor, and allitems from the Appreciation section make up the third factor. TheSchmid–Leiman results suggested a similar solution. Careful ex-amination of the items composing each of the factors indicates thatMacCAT–CA items that are similar in nature and format appear to

cluster together, suggesting that the method of measurement issomewhat confounded with the facet of competency measured.Specifically, the first factor in the modified model was composedof items that addressed vignette-based understanding (includinglegal options); the second factor, case-specific items that addressedspecific appreciation; and the third factor, vignette-based itemsthat required identification of the more relevant of two pieces ofinformation. Our finding that method variance may be confoundedwith construct variance is consistent with that of Rogers andcolleagues’ (2001) finding that the MacCAT–CA items based onthe hypothetical case vignette tended to load separately from thosethat involved more case-specific inquiries.

As discussed later, if the MacCAT–CA confounds constructvariance with method variance, this may have important implica-tions. However, the modifications made to arrive at the modifiedthree factor model (and the exploratory Schmid–Leiman results)risk capitalizing on chance fluctuations that are specific to thenormative sample and may not be replicable. The original threefactor model, having been specified prior to examining the data,does not capitalize on sampling error. Future research with anindependent sample is needed to test the modified model in an apriori manner to truly evaluate its fit.

Bonnie’s two-factor model did not adequately fit the data. Thismay be attributable partially to the way in which items for theMacCAT–CA were chosen from the research instrument. That is,although Bonnie’s model apparently provided strong theoreticalguidance for developing items for the research instrument, thistheoretical grounding may have become somewhat diluted asdecisions were made about which items to keep as part of the morepractical MacCAT–CA. The exact method for item selection hasnot been addressed in published materials. Although the principleof assessing decisional competence separate from foundationalcompetence is fine in theory (and doesn’t necessarily contradictGodinez v. Moran, 1993), the MacCAT–CA does not appear toachieve this. Notably, the authors of the MacCAT–CA may nothave intended to assess decisional competence separately giventhat they abandoned this idea somewhere between the forerunnerto the MacCAT–CA and the actual MacCAT–CA. Nevertheless, itappears that there is no discernable distinction between CAC andDC in the MacCAT–CA (i.e., r � 1.00 between CAC and DC inFigure 3).

Figure 5. Mediational model: MacArthur Competence AssessmentTool—Criminal Adjudication (MacCAT–CA) scores and Wechsler AdultIntelligence Scale (WAIS) IQ scores. ***p � .001.

Figure 6. Mediational model: MacArthur Competence AssessmentTool—Criminal Adjudication (MacCAT–CA) scores and Brief PsychiatricRating Scale (BPRS) scores. ***p � .001.


Distinguishing Competency From Other Constructs

The MacCAT–CA (scales/higher order factor) was moderatelyrelated to intelligence (r � .42) and psychopathology (r � �.36)as seen in Table 3. Mediation analyses of the relation betweenMacCAT–CA total scores and measures of IQ and psychopathol-ogy indicate a direct association between these measures indepen-dent of whether a defendant is competent. Although a defendant’scompetence status partially mediates the association between thesemeasures, the indirect effect is weak. Thus, the association be-tween the MacCAT–CA and intelligence/psychopathology is notan artifact of the sampling procedure used in the normative study,as an adjudicated incompetent group (HI) was mixed with groupsassumed competent (JT and JU). As noted by Meehl (1995),mixing such groups, when they differ on mean levels of the IQ andpsychopathology, can create covariation between MacCAT–CAscores and measures of IQ/psychopathology that are ambiguous intheir meaning. Here, by distinguishing direct and indirect paths, wewere able to show that the direct paths between the MacCAT–CAand psychopathology and intelligence are nontrivial (see Area 2;Figure 1). This is particularly the case for intelligence, which wasmoderately (path � .43) associated with MacCAT–CA scores,independent of whether a defendant was incompetent. TheMacCAT–CA may test intellectual abilities (e.g., abstract reason-ing) that are not essential to the functional capacities needed forcompetence.

Assuming that the moderate to strong covariation with intelli-gence and psychopathology is replicable, an issue of the constructvalidity of the MacCAT–CA potentially arises. Although Otto andhis colleagues (Otto et al., 1998) may be correct in noting that suchcovariation supports the construct validity of the MacCAT–CA,we read the theoretical, empirical and jurisprudential literature inthe opposite fashion. Throughout the past 3 decades of researchand scholarship on the competency construct, a common theme hasbeen the inappropriateness of confounding psychopathology orretardation with incompetency. The level of direct relation be-tween the MacCAT–CA and measures of psychopathology andintelligence approach the range of the empirical correlations be-tween the MacCAT–CA subscales and clinician global ratings ofcompetency previously reported by Otto and colleagues (1998).We believe that this pattern of covariation has important implica-tions for the discriminant validity of the MacCAT–CA. This is inpart because clinical judgments have historically been quitestrongly associated with psychopathology and intelligence (seeNicholson, Briggs, & Robertson, 1988; Nicholson & Kugler,1991). We recognize that these data may reasonably be construeddifferently by others. In any event, an informed debate about thenomological aspects of the competency construct seems warranted.

Valuing Adjudicative Competence an IdiographicConstruct

The results of this study raise a second issue about the extent towhich norm-referenced or nomothetic assessment approaches ap-propriately capture an individual’s adjudicative competence. Assuggested earlier, our findings and those of Rogers and his col-leagues (2001) suggest that MacCAT–CA items that reference thehypothetical case vignette tend to load separately from those thataddress the defendant’s own case. Hypothetical inquiries mayreveal different response patterns than case-specific ones. Al-

though the MacCAT–CA represents a significant advance in stan-dardized administration, criterion-based scoring, and normativedata, it seems worth exploring in future research whether theseadvances come at the cost of diluting focus on the defendant’sparticular case and competency.

Less than one third of the MacCAT–CA items are case specific,and these items merely ask the defendant to make judgments abouthow equitably he will be treated in the criminal justice system,relative to other defendants. Although it is an open empiricalquestion, it seems unlikely that this defendant’s rational and fac-tual understanding of the proceedings against him and an ability toconsult with his own attorney (Rogers et al., 2001) can be assessedfully by the MacCAT–CA’s six relativistic judgments about, forexample, how fairly he will be treated compared with other de-fendants. Although the MacCAT–CA’s operational definition ofcompetency abilities is nomothetic, the measure is intended toinform idiographic decisions about a particular defendant. Theextent to which this nomothetic-idiographic translation is success-ful is unclear.

It may be that case-specific inquiries are needed to assessaccurately some of the general psycholegal abilities encompassedby the MacCAT–CA. An open-ended conversation with a defen-dant about his or her case, recent experiences with the criminaljustice system, and planned legal strategy may reveal irrational ordelusional thought processes that are not apparent during brief,highly structured inquiries about relative judgments and theirbases. A defendant may answer the appreciation section items wellin an abstract sense but hold specific delusions about whether ornot he will even be tried that become evident after theMacCAT–CA booklet is put away. The incremental utility of theMacCAT–CA to structured, case-specific inquiries (and viceversa) is an important issue to address in future research.

In a related sense, future work must establish how examinersshould integrate MacCAT–CA normative scores with contextualissues involved in the defendant’s specific case. The MacCAT–CA’s authors explicitly acknowledge that competency cannot bereduced to an inflexible list of requisite abilities (Poythress et al.,1999) and recommend that the MacCAT–CA be supplementedwith case-specific inquiries and consideration of the case context(Otto et al., 1998). It remains for future research, however, toprovide examiners with clear guidance about how best to integratethe results of the MacCAT–CA with the rest of the competencyassessment. In the absence of such guidance, decision makers maybe apt to rely heavily on MacCAT–CA scores, which may be quitecompelling when considered alongside less scientific-looking id-iographic considerations. Just as the presence of mental illness isonly a starting point in assessing competency, a high or low scoreon the MacCAT–CA does not resolve the issue.

Final Conclusions

Despite some of the structural weaknesses identified in thisstudy, the MacCAT–CA appears to represent a psychometricallysound normative approach to measuring the generalized aspects ofunderstanding, reasoning, and appreciation—as well as an overar-ching component—as applied to adjudicative competence (repre-sented by the suggestion of a strong second-order factor, in addi-tion to three distinct first-order factors). The MacCAT–CA isunique in its attempt to assess systematically reasoning and deci-sional capacities relevant to competency. As a normative tool, the


measure’s primary conceptual weakness concerns its inability tolink normative scores to an individual, contextualized decision.Paradoxically, alternative devices, including structured interviews,being developed under a different conceptual approach have, astheir strength, a direct focus on the linkage of an individual’spsycholegal abilities to a contextualized decision and have, as theirweakness, an inability to normatively reference an individual’spsycholegal abilities. It would thus seem obvious that future re-search should focus on the comparative validity, reliability, andqualitative strengths and weaknesses of different assessment ap-proaches (see generally, Hase & Goldberg, 1967; Knudson &Golding, 1974). To our knowledge, only one study has everexamined the comparative validity of comprehensive interview-based competency assessment approaches in the United States(Golding, Roesch, & Schreiber, 1984; Schreiber, Roesch, & Gold-ing, 1987). A few others have examined the comparative validityof competency assessment instruments in Canada (Zapf & Roesch,2001) or of noncomprehensive interview-based instruments suchas the Georgia Court Competency Test and the CompetencyScreening Test (Nicholson, Briggs, & Robertson, 1988; Ustad,Rogers, Sewell, & Guarnaccia, 1996).

An additional important direction would involve comparing thedifferent approaches in different contexts. There are both rationaland empirical reasons to believe that different approaches to com-petency assessment, in routine practice, will often exhibit approx-imately equal validities (Golding, Roesch, & Schreiber, 1984). Theprimary reason for such approximate equivalency is that mostjurisdictions have few controls over the granting of petitionsevaluate defendants’ competency, and hence the referral processoften results in upward of 70% of defendants recommended ascompetent (Roesch & Golding, 1987; Roesch, Zapf, Golding, &Skeem, 1999). In addition, many of the incompetent defendants aregrossly psychotic and cognitively disorganized. Thus, in a typicalreferral stream, most assessment approaches may easily reachroughly equivalent results. A more strenuous testing ground wouldinvolve so-called “gray area” cases in which there is dispute,disagreement, and/or controversy surrounding the defendants’competencies.

References

Appelbaum, P. S., & Grisso, T. (1988). Assessing patients’ capacities toconsent to treatment. The New England Journal of Medicine, 319,1635–1638.

Appelbaum, P. S., & Grisso, T. (1995). The MacArthur treatment compe-tence study: I. Mental illness and competence to consent to treatment.Law and Human Behavior, 19, 105–126.

Appelbaum, P. S., Lidz, C. W., & Meisel, A. (1987). Informed consent:Legal theory and clinical practice. New York: Oxford University Press.

Appelbaum, P. S., & Roth, L. H. (1982). Competency to consent toresearch: A psychiatric overview. Archives of General Psychiatry, 39,951–958.

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variabledistinction in social psychological research: Conceptual, strategic, andstatistical considerations. Journal of Personality and Social Psychology,51, 1173–1182.

Bentler, P. (1995). EQS structural equation program model. Encino, CA:Multivariate Software, Inc.

Bentler, P., & Bonett, D. (1980). Significance tests and goodness of fit inthe analysis of covariance structures. Psychological Bulletin, 88, 588–606.

Bonnie, R. J. (1992). The competence of criminal defendants: A theoreticalreformulation. Behavioral Sciences and the Law, 10, 291–316.

Bonnie, R. J. (1993). The competence of criminal defendants: BeyondDusky and Drope. University of Miami Law Review, 47, 539–601.

Bonnie, R. J., & Grisso, T. (2000). Adjudicative competence and youthfuloffenders. In T. Grisso & R. G. Schwartz (Eds.), Youth on trial: Adevelopmental perspective on juvenile justice. Chicago: University ofChicago Press.

Bonnie, R. J., Hoge, S. K., Monahan, J., Poythress, N., Eisenberg, M., &Feucht-Haviar, T. (1997). The MacArthur adjudicative competencestudy: A comparison of criteria for assessing the competence of criminaldefendants. Journal of the American Academy of Psychiatry and theLaw, 25, 249–259.

Byrne, B. (1994). Structural equation modeling with EQS and EQS/Windows: Basic concepts, applications, and programming. ThousandOaks, CA: Sage.

Campbell, D. T. (1960). Recommendations for APA test standards regard-ing construct, trait, or discriminant validity. American Psychologist, 15,546–553.

Church, A., & Burke, P. (1993). Exploratory and confirmatory tests of theBig Five and Tellegen’s Three- and Four-Dimensional models. Journalof Personality and Social Psychology, 66, 83–114.

Cooper v. Oklahoma, 116 S. Ct. 1373 (1996).Cruise, K. R., & Rogers, R. (1998). An analysis of competency to stand

trial: An integration of case law and clinical knowledge. BehavioralSciences and the Law, 16, 35–50.

Drope v. Missouri, 420 U. S. 162 (1975).Godinez v. Moran, 113 S. Ct. 2680 (1993).Golding, S. L. (1992). Studies of incompetent defendants: Research and

social policy implications. Forensic Reports, 5, 77–83.Golding, S. L. (1993). Interdisciplinary Fitness Interview—Revised: A

training manual. State of Utah Division of Mental Health.Golding, S. L., & Roesch, R. (1988). Competency for adjudication: An

international analysis. In D. N. Weisstub (Ed.), Law and mental health:International perspectives (Vol. 4, pp. 73–109). New York: PergamonPress.

Golding, S. L., Roesch, R., & Schreiber, J. (1984). Assessment andconceptualization of competency to stand trial: Preliminary data on theInterdisciplinary Fitness Interview. Law and Human Behavior, 8, 321–334.

Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum.Grisso, T. (1992). Five-year research update (1986–1990): Evaluations for

competence to stand trial. Behavioral Sciences and the Law, 10, 353–369.

Grisso, T., Appelbaum, P., Mulvey, E., & Fletcher, K. (1995). TheMacArthur treatment competence study: II. Measures of abilities relatedto competence to consent to treatment. Law & Human Behavior, 19,127–148.

Hase, H. D., & Goldberg, L. R. (1967). The comparative validity ondifferent strategies of deriving personality inventory scales. Psycholog-ical Bulletin, 67, 231–248.

Hoge, S. K., Bonnie, R. J., Poythress, N., & Monahan, J. (1999). TheMacArthur Competence Assessment Tool—Criminal Adjudication.Odessa, FL: Psychological Assessment Resources.

Hoge, S. K., Bonnie, R. J., Poythress, N., Monahan, J., Eisenberg, M., &Feucht-Haviar, T. (1997). The MacArthur adjudicative competencestudy: Development and validation of a research instrument. Law andHuman Behavior, 21, 141–179.

Hoge, S. K., Poythress, N., Bonnie, R. J., Monahan, J., Eisenberg, M., &Feucht-Haviar, T. (1997). The MacArthur competence study: Diagnosis,psychopathology, and competence-related abilities. Behavioral Sciencesand the Law, 15, 329–345.

Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariancestructure analysis: Conventional criteria versus new alternatives. Struc-tural Equation Modeling, 6, 1–55.


Joreskog, K. G., & Sorbom, D. (1993). LISREL 8: Structural equationmodeling with the SIMPLIS command language. Chicago: ScientificSoftware.

Kline, R. (1998). Principles and practice of structural equation modeling.New York: Guilford Press.

Knudson, R., & Golding, S. L. (1974). Comparative validity of “tradi-tional” versus “S-R format” inventories of interpersonal behavior. Jour-nal of Experimental Research in Personality, 8, 111–127.

Krause, M. S. (1972). The implications of convergent and discriminantvalidity data for instrument validation. Psychometrika, 37, 179–186.

Lipsitt, P. D., Lelos, D., & McGarry, A. L. (1971). Competency for trial:A screening instrument. American Journal of Psychiatry, 128, 105–109.

MacCallum, R., & Austin, J. (2000). Applications of structural equationmodeling in psychological research. Annual Review of Psychology, 51,201–226.

McGarry, A. L. (1965). Competency for trial and due process via the statehospital. American Journal of Psychiatry, 122, 623–631.

McGarry, A. L., & Curran, W. J. (1973). Competency to stand trial andmental illness. Rockville, MD: National Institute of Mental Health.

Meehl, P. E. (1995). Bootstraps taxometrics: Solving the classificationproblem in psychopathology. American Psychologist, 50, 266–275.

Nicholson, R. A., Briggs, S. R., & Robertson, H. C. (1988). Instruments forassessing competency to stand trial: How do they work? ProfessionalPsychology: Research and Practice, 19, 383–394.

Nicholson, R. A., & Kugler, K. E. (1991). Competent and incompetentcriminal defendants: A quantitative review of comparative research.Psychological Bulletin, 109, 355–370.

Otto, R. K., Poythress, N. G., Nicholson, R. A., Edens, J. F., Monahan, J.,Bonnie, R. J., et al. (1998). Psychometric properties of the MacArthurCompetence Assessment Tool—Criminal Adjudication (MacCAT–CA).Psychological Assessment, 10, 435–443.

Overall, J., & Gorham, D. (1962). The Brief Psychiatric Rating Scale.Psychological Reports, 10, 799–812.

Poythress, N., Nicholson, R., Otto, R. K., Edens, J. F., Bonnie, R. J.,Monahan, J., & Hoge, S. K. (1999). The MacArthur Competence As-sessment Tool—Criminal Adjudication: Professional manual. Odessa,FL: Psychological Assessment Resources.

Roesch, R., & Golding, S. L. (1980). Competency to stand trial. Urbana:University of Illinois Press.

Roesch, R., & Golding, S. L. (1987). Defining and assessing competencyto stand trial. In I. B. Weiner & A. K. Hess (Eds.), Handbook of forensicpsychology. New York: Wiley and Sons.

Roesch, R., Hart, S. D., & Zapf, P. A. (1996). Conceptualizing andassessing competency to stand trial: Implications and applications of theMacArthur treatment competence model. Psychology, Public Policy,and Law, 2, 96–113.

Roesch, R., Webster, C. D., & Eaves, D. (1984). The Fitness InterviewTest: A method for assessing fitness to stand trial. Toronto, Ontario,Canada: University of Toronto Centre of Criminology.

Roesch, R., Zapf, P. A., Eaves, D., & Webster, C. D. (1998). The FitnessInterview Test (Rev. ed.). Burnaby, British Columbia, Canada: MentalHealth, Law, & Policy Institute, Simon Fraser University.

Roesch, R., Zapf, P. A., Golding, S. L., & Skeem, J. L. (1999). Defining

and assessing competency to stand trial. In A. L. Hess & I. B. Weiner(Eds.), The handbook of forensic psychology (pp. 327–367). New York:Wiley.

Rogers, R., Grandjean, N., Tillbrook, C. E., Vitacco, M. J., & Sewell,K. W. (2001). Recent interview-based measures of competency to standtrial: A critical review augmented with research data. Behavioral Sci-ences and the Law, 19, 503–518.

Rogers, R., Tillbrook, C. E., & Sewell, K. W. (2004). Evaluation ofCompetency to Stand Trial (ECST-R). Lutz, FL: Psychological Assess-ment Resources.

Roth, L. H., Meisel, A., & Lidz, C. W. (1977). Tests of competency toconsent to treatment. American Journal of Psychiatry, 134, 279–284.

Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factorsolutions. Psychometrika, 22, 53–61.

Schreiber, J., Roesch, R., & Golding, S. (1987). An evaluation of proce-dures for assessing competency to stand trial. Bulletin of the AmericanAcademy of Psychiatry and Law, 15, 187–203.

Skeem, J. L., Golding, S. L., Cohn, N. B., & Berge, G. (1998). Logic andreliability of evaluations of competence to stand trial. Law and HumanBehavior, 22, 519–547.

Skeem, J., Golding, S. L., & Emke-Francis, P. (2004). Assessing adjudi-cative competency: Using legal and empirical principles to informpractice. In W. T. O’Donohue & E. R. W. R. Levensky’s (Eds.),Forensic psychology: A handbook for mental health and legal profes-sionals (pp. 175–211). New York: Academic Press.

Sobel, M. E. (1982). Asymptotic intervals for indirect effects in structuralequations models. In S. Leinhart (Ed.), Sociological methodology (pp.290–312). San Francisco: Jossey-Bass.

Ullman, J. (2001). Structural equation modeling. In B. G. Tabachnick &L. S. Fidell (Eds.), Using multivariate statistics (4th ed., pp. 653–771).Boston: Allyn & Bacon.

Ustad, K. L., Rogers, R., Sewell, K. W., & Guarnaccia, C. A. (1996).Restoration of competency to stand trial: Assessment with the GeorgiaCourt Competency Test and the Competency Screening Test. Law andHuman Behavior, 20, 131–146.

Wechsler, D., (1981). The Wechsler Adult Intelligence Scale—Revised:Manual. San Antonio, TX: Psychological Corporation.

Wildman, R. W., Batchelor, E. S., Thompson, L., Nelson, F. R., Moore,J. T., Patterson, M. E., & deLaosa, M. (1978). The Georgia CourtCompetency Test: An attempt to develop a rapid, quantitative measureof fitness for trial. Unpublished manuscript, Forensic Services Division,Central State Hospital, Milledgeville, GA.

Winick, B. J. (1987). Incompetency to stand trial: An assessment of costsand benefits, and a proposal for reform. Rutgers Law Review, 39,243–287.

Zapf, P. A., & Roesch, R. (2001). A comparison of the MacCAT–CA andthe FIT for making determinations of competency to stand trial. Inter-national Journal of Law and Psychiatry, 24, 81–92.

Received June 24, 2003Revision received May 26, 2005

Accepted June 1, 2005 �


factor structure and validity of the macarthur … · stand trial vs. competency to plead guilty;...

Documents