using the mmpi-2 to detect substance abuse in an outpatient mental health setting

7/28/2019 Using the MMPI-2 to Detect Substance Abuse in an Outpatient Mental Health Setting

1/7

Psychological Assessment1999, Vol. 11, No. 1,94-100Copyright 1999 by the American Psychological Association, Inc.1040-3590/99/S3.00

Using the MMPI-2 to Detect Substance Abuse in an OutpatientMental Health SettingL. A. R. Stein, John R. Graham, Yossef S. Ben-Porath, and John L. McNultyKent State University

This study examined the utility of the MMPI-2 in detecting substance-abuse problems in an outpatientmental health setting. Specifically, the utility of the Addiction Acknowledgment Scale (AAS; N. C.Weed, J. N. Butcher, T. McKenna, & Y. S. Ben-Porath, 1992), the Addiction Potential Scale (APS; N. C.Weed et al., 1992), and the MacAndrew Alcoholism ScaleRevised (MAC-R; J. N. Butcher, W. G.Dahlstrom, J. R. Graham, A. Tellegan, & B. Kaemmer, 1989) in the prediction of substance abuse wasevaluated. In addition, the incremental validity of the AAS and the APS in comparison to the MAC-Rscale was evaluated. The sample consisted of 500 women and 333 men from a large community mentalhealth center in Northeastern Ohio. Results indicated that the MAC-R scale, the AAS, and the APS wererelated to interviewer ratings of substance abuse in this outpatient treatment setting. Specifically, theresults pointed to the superiority of AAS over APS in substance-abuse identification and the significantcontribution of AAS to the information available from the MAC-R scale alone.

Alcohol and drag use and abuse at times are an integral part ofproblems presented by clients at outpatient mental health clinics.These difficulties may go undetected when they are not the pri-mary reason for referral, as might be the case in primary mentalhealth settings (Weed, Butcher, McKenna, & Ben-Porath, 1992).One of the most widely used instruments fo r assessing mentalhealth problems in such settings is the MMPI-2 (Butcher, Dahl-strom, Graham, Tellegen, & Kaemmer, 1989). This study exam-ined the utility of the MMPI-2 in detecting substance-abuse prob-lems in an outpatient setting where substance abuse is not theprimary focus of treatment.

The MacAndrew Alcoholism Scale (MAC; MacAndrew, 1965)of the original MMPI was developed to differentiate alcoholicfrom nonalcoholic psychiatric patients. The scale was constructedby contrasting the MMPI responses of 200 male alcoholics seekingtreatment at an outpatient clinic with 200 male nonalcoholic psy-chiatric patients from the same facility. MacAndrew (1981) re -viewed more than tw o dozen studies that demonstrated the abilityof the MAC scale to differentiate alcoholic from nonalcoholicparticipants. The scale seems to be effective with men and women,as well as inpatients and outpatients (Graham & Stronger, 1988).

Gottesman an d Prescott (1989) indicated that the routine use ofthe MAC scale to identify substance abusers is not as compellingas many users had assumed. They noted poor construct and pre-dictive validity. However, most of the studies reviewed by themused a cutoff score of 24 in discriminating abusers from nonabus-ers, whereas Graham (1993) recommended a cutoff of 28. Inaddition, as Graham (1993) pointed out, Gottesman and Prescott(1989) seemed to imply that professionals would decide whetheror not a person abuses substances on the basis of MMPI (or

L. A. R. Stein, John R. Graham, Yossef S. Ben-Porath, and John L.McNulty, Department of Psychology, Kent State University.Correspondence concerning this article should be addressed to L. A. R.Stein, who is now at the Center fo r Alcohol an d Addiction Studies, Bo xG-BH, Brown University, Providence, Rhode Island 02912.

MMPI-2) data alone. As Graham (1993) recommended, no deci-sions should be made on the basis of MMPI (or MMPI-2) dataalone, and high scores on MAC (or MAC-R) should alert clini-cians to obtain corroborating data regarding the possibility ofsubstance abuse.

With the development of the MMPI-2, four of the original MACitems were among those eliminated from the test because ofobjectionable content. These items were replaced with four ne witems selected because they differentiated alcoholic from nonalco-holic men and women. The revised scale is labeled MAC-R(Butcher, Dahlstrom, Graham, Tellegan, & Kaemmer, 1989). In aneffort to expand the assessment of alcohol and drug problems withthe broader item pool of the MMPI-2, the Addiction PotentialScale (APS) and the Addiction Acknowledgment Scale (AAS)were developed (Weed et al., 1992). The APS consists of 39 itemsthat were endorsed differently by substance abusers compared withboth nonclinical an d psychiatric inpatient samples. In an effort toavoid reliance on obvious content in abuse potential determination,items were eliminated from the APS if they contained obviousreference to substance abuse. The AAS, a 13-item content-basedscale, wa s constructed, using rational item selection augmented byinternal consistency procedures (Weed et al.). Items were selectedfor the AAS because of their obvious content relation to substanceabuse and, therefore, it is a more face-valid scale than the APS.

Weed et al. (1992) presented data showing that the APS andA A S discriminated well between substance-abuse, nonclinical,and psychiatric samples and did so substantially better than theMAC-R scale. Specifically, the AAS appeared to discriminatebetween the substance-abuse an d nonclinical samples better thanthe APS, whereas the APS appeared to distinguish betweensubstance-abuse and psychiatric samples better than the AAS.Weed et al. noted that their findings were limited because thesamples used in development an d cross-validation of the AAS andAPS came from the same settings. Greene, Weed, Butcher, Arren-dondo, an d Davis (1992) found that in a different setting the APSdiscriminated between psychiatric inpatients an d substance-abuse

94


2/7

SUBSTANCE ABUSE AND THE MMPI-2 95samples more effectively than the AAS and that both scalesdiscriminated better than the MAC-R scale. However, althoughWeed et al. found the MAC-R scale to be quite ineffective atdiscriminating between psychiatric and substance-abuse samples,results presented by Greene et al. (1992) indicated that theMAC-R scale discriminated between psychiatric and substance-abuse samples, effectively. In addition, the magnitude of discrim-ination by the APS in the study conducted by Greene et al. was lessthan in the study conducted by Weed et al.

No study to date has directly addressed the utility of the AASand APS in assessing substance abuse in an outpatient mentalhealth setting where substance abuse, although often of consider-able relevance, is not the primary focus of the assessment. It isimportant to continue efforts to validate the AAS and APS with avariety of samples, including clients in outpatient settings. Inaddition, as recommended by Butcher, Graham, and Ben-Porath(1995), new MMPI-2 scales should add significantly to predictionof relevant behaviors and characteristics beyond what is possibleusing existing scales. The purpose of this study was to explore theeffectiveness of the MAC-R scale, the APS, and the AAS in theidentification of substance abuse, and to examine the incrementalvalidity of the AAS and APS in comparison to the MAC-R scalein an outpatient mental health facility where substance abuse is notthe primary focus of treatment.

On the basis of the body of research available about the MAC/MAC-R scale (for a summary of this research, see Graham, 1993),and the research regarding the APS and AAS reviewed above, weexpected that the MAC-R scale, the APS, and the AAS would allbe effective in identifying substance abuse. This study was con-ducted in an effort to determine which scale or scales are mosteffective in an outpatient setting where substance abuse is not theprimary focus of treatment, and the extent to which the newMMPI-2 substance abuse scales add incrementally to the MAC-Rin identifying substance abusers in this setting.

MethodsParticipants

Data were collected at a large community mental health center (CMHC)in Northeast Ohio. A variety of treatment programs was available for theclients, inclu ding partial hospitalization and individual and group ou tpa-tient treatment. Clients with substance abuse as a primary problem werereferred elsewhere. However, the CMHC served dually diagnosed clientswho did not have a substance-abuse disorder as the primary diagnosis. Aspecialized program was available for these clients involving individualcounseling, psychoeducational group intervention, family therapy, supportgroups, and referral to Alcoholics Anon ymous, Narcotics Anonymou s, andAl-Anon. On the basis of an in-depth interview during intake, and wherepossible, previous treatment records, the determination was made as towhether substance abuse was a primary or secondary problem.The sample used in this study was a subset of a larger sample collectedby Graham, Ben-Porath, and McNulty (in press). Th e larger sample in-cluded all persons seeking services at the mental health center from April1991 through December 1992 and included 1,035 men and 1,447 women.Out of a total of 2,482 clients seeking services at the CMHC, 1,263participants did not receive services beyond intake and so did not completethe MMPI-2. Valid MMPI-2 profiles were defined as follows: Cannot sayraw score


3/7

96 STEIN, GRAHAM, BEN-PORATH, AND McNULTYInstruments

MMPl-2 . Th e MMPI-2 (Butcher et al., 1989), a revised and updatedversion of the original MMPI (H athaway & McK inley, 1940), is a 5 67-itempersonality inventory. The MMPI-2 includes the validity and clinical scalesof the original MMPI, as well as new content, supp lementary, and validityscales. Adequate internal consistencies and test-retest reliabilities of theMMPI-2 scales ar e reported in the test manual (Butcher et al., 1989) alongwith preliminary validity information.Internal-consistency coefficients on MAC-R are a = .45 for women anda = .56 for men,based on the normative sample (Butcher et al., 1989).Also on the basis of the normative sample, test-retest reliabilities for a1-week interval ar e r = .78 for women an d r = .62 for men on MAC-R(Butcher et al., 1989). Although Weed et al. (1992) reported no internal-consistency coefficient for APS on a combined sample of substance abus-ers, psychiatric patients, and normative subjects, a = .74 for AAS (Gra-ham, 1993). According to Weed et al., on the basis of the normativesample, test-retest reliabilities for a 1-week interval are r = .77 for womenand r = .69 for men on APS and r = .84 for women and r = .89 for menon AAS. Although they share no items, the Pearson product-momentcorrelation coefficient between AAS and APS is r = .33 for normativeparticipants, r = .33 for a psychiatric sample, an d r = .36 for a substance-abuse sample (Weed et al.).

Intake form. An intake form was designed for the larger study (Grahamet al., in press) and was completed by a trained intake worker on the basisof a personal interview with each client.' Intake workers consisted ofpsychologists, nurses, and social workers and were highly experienced ininterviewing. Data from the intake form included in this study were th edemographic information (e.g., age, education, marital status, employmentstatus, and race), selected information concerning mental health history(e.g., psychiatric hospitalizations and outpatient treatment), diagnosis, andsubstance-abuse history. Intake workers inquired regarding participants'past an d current experience with alcohol, marijuana, cocaine, heroin, an dother substances. For each substance, ratings were made on a 4-point Likertscale as follows: 1 (n o use), 2 (some use), 3 (possible abuse), and 4(definite abuse).

During several training sessions, intake workers were instructed to limitthe definite abuse category to those clients fo r whom they ha d greatconfidence that substance abuse was a problem. The possible abuse cate-gory was to be used with clients for whom there was strong indication ofsubstance abuse, but for whom intake workers were unsure as to whethera rating of definite abuse applied. The some-use rating was to be used whensome substances had been used but not to levels that had impacted theclient 's life adversely. Th e no-use category wa s reserved fo r clients whoabstained from substances. Before substance-abuse ratings were made onthe intake form, specific information regarding past an d present use ofspecific substances was obtained. Information was gained through anin-depth interview, and where possible, pre vious treatment records. Inquir-ies regarding substance abuse dealt with frequenc y, amount, and durationfor specific substances, as well as negative effects of use.

ProcedureIntake forms were completed following an extensive interview con-

ducted at the time clients requested services. Interviewers were instructedto structure the interview around the intake form. Each client completed theMMPI-2 shortly after an intake interview. The median n umber of daysbetween intake and completion of the MMPI-2 was 7. Ratings of each ofth e specific substances were combined into a single substance-abuse indexby assigning the maxim um rat ing for any substance as the overall indicatorof substance abuse. Fo r example, if a client ha d abuse ratings of 1 foralcohol, 3 for marijuana, 2 for cocaine, 1 for heroin, and 3 for othersubstances, an overall substance-abuse rating of 3 was recorded for thatclient. Participants were assigned to one of four substance use groups basedon this variable. For women, the average substance-abuse rating was

M = 2.3 and SD = 0.93 (with M dn = 2 and mode = 2), whereas, formen,M = 2.6 and SD = 0.99 (with Mdn = 2 and mode = 2).

ResultsTable 2 presents mean scores and standard deviations on the

MAC-R, AAS, and APS for participants grouped by substance-abuse rating. T scores were derived based on the normative sample(see Butcher et al., 1989). As reported in Table 2, analysis ofvariance (ANOVA) indicated, for both genders, that the fourgroups differed significantly for each substance-abuse scale. Ta-ble 2 also reports the results of post hoc t tests and their associatedeffect sizes. For both genders and all three scales, differencesbetween no use and all other substance-abuse groups were statis-tically significant. Similarly, for both genders and for all threescales, differences between some use and definite abuse werestatistically significant. Examining data for women and men, forboth the AAS and MAC-R scale but not for the APS, differencesbetween possible abuse and definite abuse were statistically sig-nificant. For both genders, only for the AAS was the differencebetween some use and possible abuse statistically significant. Inaddition, for both genders, effect sizes were generally largest forAAS.

Before conducting a series of hierarchical regressions, intercor-relations among MAC-R, AAS, and APS scores, and substance-abuse ratings were examined (see Table 3). The positive butmodest correlations between scales suggest that they are not com-pletely redundant. For men and women, the highest correlationswere between AAS and MAC-R (r = .49 for women; r = .45 formen) and between AAS and substance-abuse rating (r = .52 forwomen; r = .50 for men). These data indicate that although AASand MAC-R are moderately intercorrelated, the ability of AAS toadd to the identification of substance abuse beyond MAC-R is notattenuated substantially by shared variance among these scales.To test the incremental contribution of the APS and the AAS tothe identification of substance abuse, we conducted a series ofhierarchical regression analyses, with substance-abuse rating as thedependent variable. As reported in Table 4, the independent vari-ables were entered in three blocks. The MAC-R scale was enteredin the first block. AAS was entered in the second block to test itsincremental contribution in predicting substance abuse beyondMAC-R. APS was entered in the third block to test its incrementalcontribution beyond MAC-R and AAS. For men and women,AAS added significantly to the variance in substance abuse ac-counted for by the MAC-R scale alone, whereas the addition of theAPS produced a significant contribution for women only. In thesecond set of analyses presented in Table 5, the order of entry forAPS and AAS was reversed. For both genders, the addition of bothAPS and AAS contributed significantly to the proportion of vari-ance in rated substance abuse.

To evaluate the accuracy with which the substance-abusescales could classify participants as abusers or nonabusers, wecalculated the positive and negative predictive powers, sensi-tivity, specificity, and overall classification accuracy of the

1 A copy of the intake form is available, on request, from L. A. R. Stein,Center for Alcohol and Addiction Studies, Box G-BH,Brown University,Providence, Rhode Island 02912.


4/7

SUBSTANCE ABUSE AND TH E MMPI-2 97Table 2Substance-Abuse Scores for Four Substance-Abuse Groups

Substance-abuse groupNo use Some use Possible abuse Definite abuse

Scale" M SD M SD M SD M SD d fSignificant group

differences'"Women

MAC-R 18.80 3.32 85 20.34 4.18 267 21.27 4.03 67 23.06 4.73 81 15.95** 3,496 a (0.39), b (0.68),c (0.22), e (0.63),f (0.40)AAS 49.77 9.29 85 56.34 10.62 267 63.19 11.38 67 71.19 13.97 81 59.86** 3,496 a (0.64), b (1.31),c (1.81), d (0.64),e(1.29) , f (0.62)APS 47.28 9.69 85 51.91 10.05 267 55.09 9.61 67 56.10 10.73 81 12.77** 3,496 a (0.46), b (0.81),c (0.86), e (0.41)Men

MAC-R 19.41 3.86 32 22.54 4.11 153 23.50 4.00 58 25.84 4.92 90 20.87** 3,32 9 a (0.77), b (1.04),c (1.38), e (0.75),f (0.51)AAS 46.22 7.69 32 52.99 9.83 153 60.31 10.15 58 65.18 13.34 90 36.80** 3,329 a (0.71), b (1.51),c (1.56), d(0.74),e (1.08), f (0.40)APS 45.34 10.11 32 51.26 10.97 153 55.17 10.42 58 56.60 10.47 90 11.05** 3,329 a (0.55), b (0.95),c (1.09), e (0.50)Note. MAC-R = MacAndrew Alcoholism ScaleRevised; AAS = Addiction A cknowledgment Scale; APS = Addiction Potential Scale. " MAC-R isin raw score as is tradition ally the case (see G raham, 1993); AAS and APS are in linear T scores. b Significant group differences are as follows: a = meanMMPI-2 scale scores are significantly different ( p .022) between no use and some use grou ps; b = mean MMPI-2 scale scores are significantly different( p .001) between no use and possible abuse groups; c = mean MMPI-2 scale scores are significantly different (p .001) between no use and definiteabuse groups; d = mean MMPI-2 scale scores are significantly different (p .001) be tween some use and possible abuse group s; e = mean MMP I-2 scalescores are significantlydifferent (p ^ .006) between some use and definite abuse groups; f = mean MMPI-2 scale scores are significantly different (p &.042) between possible abuse and definite abuse groups. Numbers within parentheses are effect sizes for the post hoc tests and are expressed as d .** p < .001.

scales, using data from those participants about whom we haddefinitive data. As stated above, all participants were rated ona 4-point scale. Participants rated at the two extremes of thisscale (no use and definite abuse) were clearly nonabusers orabusers of substances. Because of ambiguity regarding thoserated at the two intermediate levels (some use and possibleabuse), they were not included in the classification analyses.Although this procedure reduced the sample size (N = 166 forwomen; N = 122 for men), it eliminated ambiguity resulting in

Table 3Pearson Product-Moment Correlation Coefficients BetweenSubstance-Abuse Scales and Substance-Abuse Rating

ScaleMAC-RAASAPSRating

MAC-R_

0.45***0.36***0.39***

AAS0.49***0.37***0.50***

AP S0.27***0.29***0.29***

Rating0.30***0.52***0.26***

Note. Correlations involving MacAndrew Alcoholism ScaleRevised(MAC-R) were computed using raw scores, those for Addiction Acknowl-edgment Scale (AAS ) and Addiction Potential Scale (APS) were computedusing linear T scores. Correlations above the diagonal refer to women,those below the diagonal refer to men.***/> < .0005.

a more precise evaluation of the classification accuracy of theMMPI-2 substance abuse scales.Table 6 shows the classification accuracy analyses using the

Table 4Predicting Substance Abuse From Hierarchical RegressionAnalyses: Addiction Potential Scale (APS) Entered Last

Scalesentered8

MAC-RAA SAPS

MAC-RAA SAPS

R

.30.52

.53

.39.53

.54

K 2

.09.27

.28

.15.28

.29

p2"adjustedWomen

.09.27

.27Men

.15.28

.28

ANOVA

F(l,498) = 47.31**F(2, 497) = 90.75**''change ~~ l.VJF(3, 496) = 63.87**Change = 7-68*

f(l, 331) = 58.94**F(2, 330) = 65.31**change =6094**F(3, 329) = 44.51**Change = 2.37

Note. ANOVA = analysis of variance. a MacAndrew AlcoholismScaleRevised (MAC-R) is in raw score; Addiction Acknowledgm entScale (AAS) and APS are in linear T score.* / > < . 0 0 6 . * * ; > < . 0 0 1 .


5/7

98 STEIN, GRAHAM, BEN-PORATH, AND McNULTYTable 5Predicting Substance Abuse From Hierarchical RegressionAnalyses: Addiction Acknowledgment Scale (AAS) Entered Last

Scaleentered"

MAC-RAPSAA S

MAC-RAPSAAS

R

.30.35

.53

.39.42

.54

R2

.09.12

.28

.15.18

.29

"adjusted

Women.09.12.27

Men.15.17.28

ANOVA

F(l, 498) = 47.31**F(2, 497) = 33.95**Change = 1890**F(3,496) = 63.87**Change = 10896**

F(l, 331) = 58.99**F(2, 330) = 35.65**F = 10 S9**change 1U.J7F(3, 329)= 44.51**Fchange = 51.36**

Note. AN OVA = analysis of variance. " MacAndrew AlcoholismScaleRevised (MAC-R) is in raw score; AAS and Addiction PotentialScale (APS) are in linear T score. ** p < .001.

resulting base rate of 49% for women and base rate = 74% formen. These percentages reflect the base rate of substance abusefound in the smaller sample of participants for whom substance-

abuse ratings were unambiguous (i.e., ratings of 1 or 4). Generally,overall hit rates for women and men were rather poor for theMAC-R and APS (ranging from 47% to 69%).For women, Tscore >60 on AAS yielded the best overall hit rate (86%) with asensitivity of 85%, a specificity of 87%, and positive and negativepredictive powers of 86% each. For men, T score a60 on AASyielded the best overall hit rate (76%) with a sensitivity of 71%, aspecificity of 91%, and positive and negative predictive powers of96% and 53%, respectively.

Because the actual base rates obtained in the narrower samplewere unrealistically high, analyses were again conducted on theparticipants for whom substance-abuse ratings were unambiguous;however, base rates were adjusted (Meehl & Rosen, 1955) toreflect the base rate of substance abuse found in the larger sampleof participants (N = 500 for women; N 333 for men). Theseresults are reported in Table 7. Generally, overall hit rates wereacceptable for both women and men on MAC-R and APS (rangingfrom 76% to 86%). For women, T score >65 on AAS yielded thebest overall hit rate (92%) with a sensitivity of 69%, a specificityof 96%, and positive and negative predictive powers of 79% and94%, respectively. For men, T score 5:60 on AAS yielded the bestoverall hit rate (85%) with a sensitivity of 71%, a specificity of91%, and positive and negative predictive powers of 74% and89%, respectively.

Table 6Sensitivity, Specificity, Positive Predictive Power (PPP),Negative Predictive Power (NPP), an d OverallHit Rate in Percentages

Table 7Sensitivity, Specificity, Positive Predictive Power (PPP),Negative Predictive Power (NPP), an d OverallHit Rate in Percentages

Scalecutoff3

MAC-R>24>28AA S>60>65APS>60>65

MAC-R>24>28AA S>60>65APS>60>65

Sensitivity

361485694621

613371544829

SpecificityWomen

9310087969196Men

8110091919197

PPP

8310086958285

9010096949396

NPP

605586776456

433553413833

Overallhit rate

655886836960

665176645947

Scalecutoff"

MAC-R>24>28AA S>60>65APS>60>65

MAC-R>24>28AA S>60>65APS>60>65

Sensitivity

361485694621

613371544829

SpecificityWomen

9310087969196Men

8110091919197

PPP

5010056794853

5510074686577

NPP

888697949086

858089848279

Overallhit rate

848687928384

768285817979

Note. For women, base rate = 49%, and for men, base rate = 74%." MacAndrew Alcoholism ScaleRevised (MAC-R) raw score >24 isbased on Gottesman and Prescott (1989); MAC-R raw score >28 is basedon Graham (1993). Addiction Acknowledgment Scale (AAS) and Addic-tion Potential Scale (APS) linear T score >60 is based on Graham (1993);AA S and APS linear T score a65 is based Butcher et al. (1989).

Note. For women, base rate = 16%,and for men,base rate = 27%."MacAndrew Alcoholism ScaleRevised (MAC-R) raw score >24 isbased on Gottesman and Prescott (1989); MAC-R raw score >28 is basedon Graham (1993). Addiction Acknowledgment Scale (AAS) and Addic-tion Potential Scale (APS) linear T score 60 is based on Graham (1993);AAS and APS linear T score >65 is based Butcher et al. (1989).


6/7

SUBSTANCE ABUSE AND THE MMPI-2 99Discussion

Scores on the three MMPI-2 substance abuse scales, MAC-R,APS, and AAS, were related to interviewer ratings of substanceabuse in.this outpatient mental health sample. The three scalesdifferentiated significantly between individuals identified by ex-perienced interviewers as evidencing varied levels of substanceabu sive behavior. Hierarchical regression analyses indicated thatAAS added significantly and su bstantially to MAC-R and APS inpredicting variance in substance abuse and that APS added onlymodestly to MAC-R on this task in this sample. Although classi-fication accuracy analyses indicated that MAC-R and APS yieldedacceptable overall hit rates, AAS produced the best classificationrates in this sample.

MAC-R explained a modest, but significant amount of variancein substance use and abuse (about 10% and 15% for women andmen, respectively). For women and men, AAS added substantiallyto the amount of variance explained by MAC-R alone (varianceincreased by 18% and 13% for women and m en, respectively). Theaddition of APS to MAC-R and AAS did not add as muchinformation to the amount of variance in substance abuse ex-plained. Similarly, whereas the addition of APS to MAC-R wasstatistically significant, variance increased by only 3% for womenand 2% for men. Finally, AAS predicted a statistically significantand substantial proportion of additional variance beyond MAC-Rand AP S (variance increased by 15% and 11% fo r women andmen, respectively). The amount of variance in substance use andabuse explained by all three scales in the regression equation wasa little less than 30%, which suggests that these scales havepractical utility in screening for substance abusive behavior inoutpatient mental health settings.

The overall hit rates obtained in the classification accuracyanalyses were higher when base rates were adjusted to reflect thebase rate of substance abuse in the total sample (N = 500 forwomen; N = 333 for men). As compared with base rate = 49% forwomen and base rate = 74% for men, adjusted base rates of 16%and 27% for women and men, respectively, may be more realisticestimates of the base rates of substance abuse found in outpatientsettings. For both genders, a cutoff of 28 provided the highestoverall accurate classification rate on MAC-R. The positive andnegative predictive powers were high; however, sensitivity waspoor for both genders. For women, optimal classification accuracywa s found on AAS at a T-score cutoff of 65, whereas for men , theoptimal r-score cutoff was 60 on AAS. For both genders, all fouraccuracy indicators were acceptably high. Overall classificationaccuracy w as equal at T-score cutoffs of 60 and 65 for both womenand men on APS. How ever, positive predictive power was poor forwomen at both cutoffs an d acceptable for men only at a cutoffof 65.

These classification accuracy analyses indicate that an elevatedscore on any of these scales in outpatient mental health settingsshould alert clinicians to obtain more data regarding the possibilityof substance abuse. The higher the score, the greater the possibilityof substance-abuse problems. The hierarchical regression resultsindicate that scores on these scales are not completely redundant,an d therefore, each of the scales warrants examination whenscreening for a possible substance-abuse problem in outpatientmental health clients.

The finding that AAS demonstrated the greatest effect size inthis sample warrants further consideration. This brief, face validcontent-based scale consists primarily of items related overtly tosubstance abusive behavior. As its label implies, individuals withelevated scores on this scale are acknowledging problems in thearea of substance abuse. The current results are consistent withprevious research, ind icating that content-based scales can provideincrementally valid information when compared with empiricallykeyed scales. Archer, Elkins, Aiduk, an d Griffin (1997);Ben-Porath, Butcher, and Graham (1991); and Ben-Porath, Mc-Cully, and Almagor (1993) have reported similar findings for theMMPI-2 Content Scales compared with the empirically keyedclinical scales. The finding that AAS adds incrementally to theempirically keyed MAC-R and APS scales provides further evi-dence of the importance of considering item content themes inMMPI-2 interpretation.

As is the case with the MMPI-2 Co ntent Scales, the transparentnature of the AA S items imposes limitations on its interpretability.Although this study did not address the susceptibility of the sub-stance abuse scales to distortion, for obvious reasons, contentscales may be more susceptible to intentional distortion thanempirically keyed ones. Therefore, it is of particular importance toexamine scores on the validity scales of the MMPI-2 beforeinterpreting AAS. A defensive test-taking approach is likely toresult in artificially low scores on AAS and other content scales.Moreover, even if a general defensive approach to the test is notindicated, it remains possible that some test-takers would be re-luctant to acknowledge problems specifically in the area of sub-stance abuse. Therefore, the absence of elevation on this scalecannot be taken as a negative indicator of substance abuse. Thus,higher scores on AAS can be used to suggest the possibility ofsubstance-abuse difficulties in outpatient mental health settings;however, lo w scores on this scale cannot be used to rule thisproblem out. Future studies ma y address the effects of defensivetest-taking style on the substance abuse scales.Findings regarding APS indicate a significant, but perhaps,more limited role for this scale in detecting substance-abuse prob-lems in outpatient settings. Results for this scale were generallycomparable to those of the MAC-R scale. Unlike the findingsreported by Weed et al. (1992), the present results do not point toa substantial contribution of APS beyond MAC-R in identifyingsubstance abuse in this setting. Population differences may accountfor the discrepant findings. Participants in the Weed et al. studycame from two different types of settings: an inpatient facilitydevoted exclu sively to substance-abuse treatmen t and a nu mbe r ofinpatient psychiatric u nits. It is possible that APS is better able todifferentiate between individuals more clearly identified as havingeither substance-abuse or significant psychiatric difficulties than itca n distinguish among outpatients, all receiving mental healthservices and some of whom have secondary substance-abuseproblems.A limitation of our study concerns the broad definition used forrating substance abu se. Our substance-abuse variable did not mea-sure a specific class of substances. S ignificant levels of comorbid-ity precluded breaking our sample down into mutually exclusivealcohol- or drug-abuse subsamples. Consequently, the ability ofthe MMPI-2 substance abuse scales to identify alcohol-only ordrug-only abuse could not be examined here and remains to bestudied in future investigations. Another limitation of this study is


7/7

100 STEIN, GRAHAM, BEN-PORATH, AND McNULTYthat both interrater reliability and temporal stability data wereunavailable regarding substance-abuse ratings by intake workers.It was simply unrealistic for intake workers at this large commu-nity mental health center to conduct two interviews per client on asubsample of participants for this field study. In addition, futurestudies may measure substance use and abuse using methods otherthan intensive interview data. For example, standardized instru-ments may be used as may reports from significant others.

In summary, the current findings indicate that scores on theMMPI-2 substance abuse scales are related to rated substanceabuse in an outpatient mental health setting. These results illustratethat the utility of the recently developed AAS and APS scalesgeneralizes beyond inpatient settings where they were initiallyconstructed and studied. Identifying potential substance-abuseproblems in individuals referred primarily for outpatient mentalhealth services is important to the development of a successfultreatment plan. The MMPI-2 substance abuse scales can play asignificant role in this process. There may be times when a clientis not honest in an interview but is honest on a test that does notinvolve face to face contact, and this argues for the usefulness oftesting. This study depended in part on the truthfulness of the clientand the skill of the interviewer in determining substance abuse,even when perhaps the client was reluctant to divulge information.However, there may be times when a more skillful and experi-enced interviewer is unavailable to conduct a thorough interview,and testing can alert professionals to areas needing more attention.The MMPI-2 can also assist in streamlining following interviews.This study indicates that the MMPI-2 is useful in screening forsubstance abuse in outpatient mental health settings where sub-stance abuse is not the primary reason for referral.

ReferencesArcher, R. P., Elkins, D. E., Aiduk, R., & Griffin, R. (1997). The incre-mental validity of MMPI-2 Supplementary scales. Assessment, I, 193-205.Ben-Porath, Y. S., Butcher, J. N., & Graham, J. R. (1991). Contribution ofthe MMPI-2 Content scales to the differential diagnosis of schizophreniaand major depression. Psychological Assessment, 3, 634-640.Ben-Porath, Y. S., McCully, E., & Almagor, M. (1993). Incrementalvalidity of the MMPI-2 Content scales in the assessment of personality

and psychopathology by self-report. Journal of Personality Assess-ment, 61 , 557-575.Butcher, J. N., Dahlstrom, W . G., Graham, J. R., Tellegen, A., & Kaem-mer, B. (1989). MMPI-2: Manual for administration and scoring. Min-neapolis: University of Minnesota Press.Butcher, J. N., Graham, J. R., & Ben-Porath, Y. S. (1995). Methodologicalproblems and issues in MMPI, MMPI-2, and MMPI-A research. Psy-chological Assessment, 7, 320329.Gottesman, I. I., & Prescott, C. A. (1989). Abuses of the MacAndrewMMPI Alcoholism scale: A critical review. Clinical Psychology Re-view, 9, 223-242.Graham, J. R. (1993). MMPI-2: Assessing personality and psychopathol-ogy (2nd ed.). New York: Oxford University Press.Graham, J. R., Ben-Porath, Y. S., & McNulty, J. L. (in press). Using theMMPI-2 in outpatient mental health settings. Minneapolis: University ofMinnesota Press.Graham, J. R., & Stronger, V. E. (1988). MMPI characteristics of alcohol-ics: A review. Journal of Consulting and Clinical Psychology, 56,197-205.Graham, J. R., Watts, D., & Timbrook, R. E. (1991). Detecting fake-goodand fake-bad MMPI-2 profiles. Journal of Personality Assessment, 57 ,264-277.Greene, R. L., Weed, N. C., Butcher, J. N., Arrendondo, R. , & Davis, H. G.(1992). A cross-validation of MMPI-2 Substance Abuse scales. Journalo f Personality Assessment, 58, 405-410.Hathaway, S. R. , & McKinley, J. C. (1940). A multiphasic personalityschedule (Minnesota): I. Construction of the schedule. Journal of Psy-chology, 10 , 249-254.MacAndrew, C. (1965). The differentiation of male alcoholic outpatientsfrom nonalcoholic psychiatric outpatients by means of the MMPI. Quar-terly Journal of Studies on Alcohol, 26 , 238-246.

MacAndrew, C. (1981). What the MAC scale tells us about alcoholics: Aninterpretive review. Journal of Studies on Alcohol, 42, 604-625.Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the effi-ciency of psychometric signs, patters, or cutting scores. PsychologicalBulletin, 52 , 194-216.Weed, N. C., Butcher, J. N ., McKenna, T., & Ben-Porath, Y. S. (1992).New measures for assessing alcohol and drug abuse with the MMPI-2:The APS and AAS. Journal of Personality Assessment, 58, 389-404.

Received April 7, 1998Revision received October 22, 1998

Accepted October 26, 1998

using the mmpi-2 to detect substance abuse in an outpatient mental health setting

Documents