factors associated to outcome in psychotherapy: an effectiveness study in puerto rico

Factors Associated to Outcome in Psychotherapy:An Effectiveness Study in Puerto Rico

Ä

Guillermo BernalUniversity of Puerto Rico, Río Piedras Campus, Puerto Rico

Ä

Janet BonillaUniversity of Puerto Rico, Mayagüez Campus

Ä

Lymaries Padilla–CottoUniversity of Puerto Rico, Río Piedras Campus, Puerto Rico

Ä

Esther Mariae Pérez–PradoNortheastern University

Psychotherapy research has not received much attention among Latinosin the U.S. or in Latin America including Puerto Rico. The present studyexplores factors associated with the effectiveness of psychotherapy with asample of Puerto Ricans. Seventy-nine persons who received psychother-apy participated in a study aimed at evaluating the process and outcomeof psychotherapy. Participants completed a questionnaire on the effects oftreatment on the presenting problem. Information from the questionnaireand other information from the clinical records included the BDI and theSymptom Check List-36 scores, the number of sessions received, etc.constituted the data set. The results show that symptomatic severity, age,the number of sessions, and the therapeutic alliance were associated with

An earlier version of the article was presented at theXXV Interamerican Congress of Psychology, San Juan, PuertoRico, in 1995. This research was supported by a grant fromInstitutional Funds for Researchat the University of PuertoRico (1992–1995) and, in part, by a grant from the National Institute on Mental Health (NIMH Grant R24-MH49368).This research was also supported byUniversity Center for Psychological Services and Research(CUSEP, for itsinitials in Spanish). We are grateful to the clients at CUSEP for their participation in this study. We also appreciate thecollaboration of Ivonne M. Ocasio, Luis Viera, Mineira Serrano, and Gerardo Sotomayor as research assistants and ofMaribel Matos (psychologist at CUSEP) who facilitated the process of access to client’s data while safeguardingconfidentiality.Requests for reprints should be addressed to the first author at CUSEP, University of Puerto Rico, PO Box 23174, SanJuan, PR 00931-3174.

JOURNAL OF CLINICAL PSYCHOLOGY, Vol. 54(3), 329–342 (1998)© 1998 John Wiley & Sons, Inc. CCC 0021-9762/98/030329-14

effectiveness. Also, the alliance explained 45% of the variance in effec-tiveness. The results suggest that the therapeutic alliance merits furtherstudy in psychotherapy research and in clinical training. © 1998 JohnWiley & Sons, Inc. J Clin Psychol 54: 329–342, 1998.

This study is part of a larger effort to develop instruments and measures for the evaluation ofpsychotherapy outcome for Latino populations and to initiate both effectiveness and efficacystudies with Latino and Spanish speaking samples. Here we present a study that aims to exam-ine factors associated to the effectiveness of psychotherapy using an exploratory and descrip-tive approach.

EFFICACY AND EFFECTIVENESS STUDIES

Effectiveness studies of psychotherapy often utilize an exploratory approach. Howard, Orlinsky,and Lueger (1994) suggest that for clinical research to advance, exploratory methods need to beadopted. Studies of effectiveness often use naturalistic methods and place a greater emphasis ongeneralization and replication of research findings. Also, exploratory methods tend to use quasi-experimental designs that allow the evaluation of possible threats to internal and external validity.

The growing literature on psychotherapy research supports the thesis that a positive rela-tionship between therapist and client (therapy alliance) is an important aspect of treatmentoutcome (Alexander & Luborsky, 1986; Alvarez, 1991; Bordin, 1985; Hartley, 1985; Orlinsky& Howard, 1986). The therapy alliance may be defined as the relationship between therapistand client as part of the therapeutic process (Hartley, 1985).

Our study was based on a metatheoretical conceptualization of the psychotherapy allianceas proposed by Bordin (1985), Pinsoff and Catherall (1986), and Howard, Orlinsky, and Lueger(1994). These authors conceptualize the psychotherapy alliance as an agreement between thetherapist and client to the extent that they share tasks, establish goals and objectives, anddevelop a positive bond. Independent of the specific theoretical framework of a particulartreatment, both client and therapist contribute to the effectiveness of the process of psychother-apy through a set of activities, such as goals, tasks, and bonds inherent to psychotherapy.

One of the basic questions of this study was to examine the degree to which the therapeuticalliance contributes to a positive therapeutic outcome from the client’s perspective. The designof our study was based, in part, on the framework for effectiveness research suggested byStrupp, Wallach, and Wogan (1964) and by Howard, Orlinsky, and Lueger (1994). The studyutilizes a questionnaire method with persons who have received psychotherapy at a psycholog-ical services center.

METHOD

Participants

Invitations to participate in the study were mailed to former clients of theUniversity Center forPsychological Services and Research(CUSEP for its name in Spanish) at the University ofPuerto Rico, Río Piedras Campus, between 1987 to 1994. The sample was selected using thefollowing criteria: clients who received at least four sessions of psychotherapy and who were18 years or older. From a total of 343 individuals who met these criteria, only 219 had the

330 Journal of Clinical Psychology, April 1998

minimal information needed to be contacted (i.e., complete current address or phone). Of the219 persons who were invited to participate, 79 answered the questionnaire and returned it bymail. Thus a 36% response rate of participation was achieved.

Of the 79 participants, approximately 81% were women, mainly from the San Juan met-ropolitan area (81%), single (59%), integrated in the labor force (69%), with an average age of31, and an average salary of $11,855 dollars annually.

Psychotherapy and the Therapist

Psychotherapy was offered as part of the training program at CUSEP. This Center was foundedin 1986 as part of the Doctoral Psychology Program at the University of Puerto Rico, RíoPiedras Campus. An objective of the Center is to train psychology graduate students in provid-ing psychological services of a clinical, community, and research nature (Bernal, 1993b).

The training of psychotherapy is offered using several theoretical orientations. All trainingis organized in teams comprised of a faculty member as supervisor with three or four graduatestudents who meet on a weekly basis. In addition to the clinical teams that provide psychother-apy, there is a screening clinical team comprised of a faculty supervisor and predoctoral interns.Between 1987 to 1990, there were six to eight clinical teams. Since 1991, there have been eightto twelve of these teams operating per year.

The clinical teams oriented to provide psychotherapy were centered on the following psy-chological models: Psychoanalytic Therapy with a critical-social perspective; System Therapywith structural, strategic, communicational and contextual variants; therapy based on the socialsystems; Object Relations Theory and Ego Psychology based psychotherapy; PsychoanalyticTherapy based on Lacan; Interpersonal Therapy; and Constructionist based psychotherapy.

All the therapists were graduate students. They received at least one hour of clinical super-vision every week. The clinical teams used different resources for supervision such as video oraudio recording of sessions, live supervision, co-therapy, and case discussion, and all wererequired to participate in a weekly case conference.

Instruments

The evaluation instrument included the following six questionnaires: Demographic Informa-tion, Satisfaction Questionnaire, the Integrative Psychotherapeutic Alliance Scale (13-item ver-sion), and the Follow-up Questionnaire. Another two instruments measured psychiatricsymptomatology: The Beck Depression Inventory (BDI) and the Symptom Check List-36(SCL-36).

Demographics Questionnaire.This instrument consists of closed and open ended questionsrelated to the participants’ gender, age, marital status, work place, salary, and schooling.

Satisfaction Questionnaire.This was an adaptation of the 8-item Satisfaction Questionnaire(SQ-8) developed by Roberts, Attkinson, and Mendías (1984) and used with Latinos. The SQ-8includes questions about the satisfaction of the clients with the services received. Each questionis answered on a 4-point Likert scale. Also, this questionnaire includes two open ended ques-tions on what the participants liked or did not like about the services received.

Integrative Psychotherapy Alliance Scale (IPAS).The IPAS is a psychotherapeutic alliancescale developed by Pinsoff and Catherall (1986). The original scale consisted of 29 items whichevaluated three dimensions of the alliance: tasks, goals, and bond. The items are answered by

Factors Associated to Outcome 331

a 7-point Likert scale. For this study we used a 13-item short version of this scale based on anearlier work (Bernal, Bonilla, Alvarez, & Greaux, 1993).

Follow-up Questionnaire (FQ).Many of the items for the FQ were based on the work ofHoward (1986) and translated using a back translation procedure. The details of this process isdescribed elsewhere (Bernal, Padilla, Perez–Prado, Bonilla, & Serrano, 1995). Other items inthe FQ were developed entirely by our research team. The FQ includes questions about thesituation that lead the person to seek help and on the degree of distress experienced at the timetherapy began (e.g., What problem(s) lead you to look for professional help the first time? Howmuch distress or discomfort did you experience at the time that you began with your sessions?).The questionnaire asked about the experience in therapy (e.g., How did you feel about yourtherapist? How did you feel about the psychotherapy services received?) and about the situa-tion with the presenting problem at the conclusion of therapy (e.g., Upon ending your sessions,what happened with the problem or situation that lead you to seek psychotherapy? How muchdistress or discomfort did you experience at the time that you ended with your sessions?). Also,a question was included on the overall benefit or harm as a result of receiving psychotherapy.The question read as follows: How much did you benefit or were harmed as a result of thetreatment received. An illustration of a thermometer was used as a rating metaphor with a rangefrom 1100 (benefited a great deal) to zero (neither benefit nor harm) and2100 (harmed agreat deal). The questionnaire also includes 24 items on the kinds of problems for which clientsseek therapy and six questions that evaluate the therapeutic bonds between the therapist and theclient. These items use a 5-point Likert Scale.

Beck Depression Inventory (BDI).The BDI (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961)is a 21 item self report instrument that measures depression. Each item corresponds to onecategory of symptoms and attitudes. Total scores for the BDI fluctuate from 0 (no depression)to 63 (severe depression). The scores are divided in different levels of depression: 0–9 nonde-pressive, 10–15 slight depression, 16–23 moderate depression and 24–63 severe depression(Beck et al., 1961). Good indices of internal reliability and construct validity have been reportedfor this instrument with Puerto Rican samples (Bernal, Bonilla, & Santiago, 1995).

Symptom Checklist-36 (SCL-36).The SCL-36 (McNeil, Greenfield, Attkinson, & Binder, 1989)is a symptom checklist with 36 items that evaluates psychological or psychiatric distress occur-ring during the last 7 days. Distress is represented in a 5-point scale. Recently, good internalreliability and construct validity have been reported for this instrument in a Puerto Ricanclinical sample (Bernal, Bonilla, & Santiago, 1995).

PROCEDURES

The first step in this study was the development, translation, and adaptation of the instruments.For example, some items for the Demographic Questionnaire, the SQ-8, and the IPAS-13 wereconstructed and piloted by the research team. The translation of the IPAS-13, SQ-8, and itemsfrom Howard’s (1986) questionnaire in the FQ consisted of a back translation procedure. Weevaluated the consistency between the Spanish and English versions of the instruments in termsof content, thematic, technical, structural, and conceptual criteria (Bravo, Canino, & Bird,1987; Bravo, Canino, Rubio–Stipec, & Woodbury–Fariña, 1991). This process allowed us toconclude that the Spanish version of the instruments or items were equivalent to the Englishversions in terms of the criteria previously noted. Next, we proceed to adapt the Spanish ver-sions of the IPAS-13 and the SQ-8. During this process we modified the instructions and someitems.


The second step was the identification of individuals who had received psychologicalservices and met the inclusion criteria. A list of potential participants was prepared. A letterinviting participation in the study and a brochure on the nature of the study was mailed toformer clients. Three weeks later, follow-up phone calls were made to confirm the participationof clients in the study. A complete questionnaire was mailed to individuals who agreed toparticipate. Potential participants who did not have a phone number were sent the letter ofinvitation, the brochure, and the questionnaire were mailed at the same time.

Finally, the questionnaires were returned, they were coded and entered into a data base.The data base included the following information: responses by participants to the question-naires; data from the participant at the time they entered treatment (BDI, SCL-36, and numberof psychotherapy sessions); and symptomatic information at the time that the questionnaire wascompleted (BDI and SCL-36).

RESULTS

Characteristics of the Sample and Gender Differences

The demographic characteristics of the sample by gender appear in Table 1 and 2. No signifi-cant differences for gender were found for the demographic variables (age and income), ther-apy alliance scales, satisfaction, number of sessions, symptom severity at the beginning oftreatment (BDI and SCL-36), and benefit from psychotherapy. As Tables 1 and 2 show, while81% of the sample was female, in general, the demographic characteristics between males andfemales were comparable. Also, with the exception of income, the demographic characteristicsfrom the sample were similar to those reported by Vessey & Howard (1993) on individuals whoseek psychotherapy.

The demographic characteristics of the sample were similar to that of the clinic as a whole.For example, more women than men sought psychological services, maintaining the same ratioas in the study sample. The average age for clients in the clinic is about 24 (SD5 13) sugges-tive of a relatively young clientele. The mean age of the study sample was higher (31.4) since

Table 1. Demographic Characteristics for Gender, Marital Status,and Occupation of Participants by Gender.

Females(n = 64)

%

Males(n = 15)

%

Total(n = 79)

%

Gender 81.0 19.0 100Marital Status

Single 59.4 64.7 60.8Married 21.9 35.3 24.1Widowed 1.6 0.0 1.3Divorced 15.6 0.0 12.7Other 1.6 0.0 1.3

OccupationProfessional 56.3 30.8 50.8Technical 8.3 15.4 9.8Blue collar 8.3 0.0 6.6Clerk 4.7 7.7 6.6Student 10.9 23.1 16.4Other 4.7 23.1 9.8


age (over 18) was an inclusion criteria for participation. The clinic data for the BDI and SCL-36total scores were similar to the study sample and there were no significant gender differencesfound as was the case with the study sample.

Psychometric Properties of the Instruments

The reliability index of the instruments in this study was assessed through the Cronbachamethod. Table 3 presents the alpha coefficients for the following instruments: BDI, SCL-36,SQ-8, IPAS-13, and for the six-item alliance scale (AS-6) from the Follow-up Questionnaire.As the table shows, the reliability indices for all the instruments are over .90. This suggests ahigh degree of intercorrelation between the items of each instrument and excellent internalconsistency in the measurement of the particular constructs. Other studies also support thereliability obtained for the BDI, SCL-36 and IPAS (Bernal, Bonilla, & Santiago, 1995; Bernal,Bonilla, Alvarez, & Greaux, 1993). Bernal, Bonilla, and Santiago (1995) reported ana coef-ficient of .89 for the BDI and of .94 for the SCL-36 in a Puerto Rican clinical sample. Similarresults have been obtained within North American and European contexts (Beck, Ward, Men-delson, Mock, & Erbaugh, 1961; McNeil, Greenfield, Atkinson, & Binder, 1989; Vasquez &Sanz, 1991). Bernal, Bonilla, Alvarez, & Greaux (1993) reported a reliability index for the fullIPAS of .83.

Symptomatology and the Presented Problem

At the Center, reliable psychiatric diagnostic data was not available. However, psychiatric andpsychological symptomatology was available on all clients at intake with BDI and SCL-36

Table 2. Means and Standard Deviations for Age, Income, and Numberof Sessions by Gender

Females Males Total

AgeM 31.86 29.60 31.43SD 10.15 8.04 9.78

Annual IncomeM 11,992 11,222 11,855SD 6,905 8,490 7,115

SessionsM 11.3 10.47 10.92SD 10.81 8.31 10.34

Table 3. Reliability Coefficients for theSymptomatology (BDI, SCL-36), Satisfaction (SQ-8),and Alliance (IPAS-13, AS-6) Instruments

Instrument a Index

BDI .94SCL-36 .96SQ-8 .96IPAS-13 .92AS-6 .93


scores. Table 4 presents the mean total scores for the BDI and SCL-36. The mean score for thesample was of 15 (SD5 8.04). This score represents a moderate level (10–18) of depression forthe total sample using the Beck’s cut off score of nine. Overall, the sample’s symptomatologyby level of depression indicated that 29% fell in the nondepressed range. About 40.6% showedmild depression, 24.6% had moderate depression, and the remaining 5.8% was in the severelevel of depression. A greater percentage of females were in the mild and moderate range ofdepression in comparison with men. Most males (53.3%) scored at the level of nondepression.While females reported more symptoms of depression than males, the differences were notstatistically significant. The total score for the SCL-36 was 83.4 (SD5 26.0), suggesting thatin general persons reported some discomfort related to psychiatric symptoms.

Table 5 presents the means and standard deviations of the degree of discomfort reported onthe presenting problems by clients. The problems with higher ratings for discomfort were:feelings of anxiety, depression, anger; other problems; low self esteem; difficulties in familyrelations; not understanding oneself; emotional overreactions to events; and lacking a sense ofdirection or goals in life. In general, mood (anxiety, depression, and anger) was the presentingproblem associated with highest degrees of discomfort for the sample. Other problems wererelated to self-esteem and interpersonal difficulties.

Effectiveness of Psychotherapy According to Clients

Effectiveness was defined according to three criteria. First, the client’s retrospective evaluationof the degree of discomfort on the primary presenting problem at thestart and at theendoftreatment. Second, a global evaluation of the benefit versus harm from treatment, and third, ascale on client’s treatment satisfaction.

In terms of the degree of discomfort with the presenting problem at the beginning oftreatment, 31.1% of the sample indicated that the problem caused a “lot of discomfort” and59.5% indicated “extreme discomfort.” Thus, 90.6% indicated that at the beginning of treat-ment they were experiencing a lot or extreme discomfort. At the beginning of treatment, themean level of discomfort (on a 7-point scale) was 6.3% (SD5 .13). At the end of treatment, themean level was of 4.6 (SD5 .23). Approximately 39.3% indicated having ended treatment withlittle, very little, or almost no discomfort. Another 13.5% reported ending treatment with “mod-erate” discomfort, 21.6% with a “lot,” and 20.3% with “extreme” discomfort.

Another way to analyze these data on the presenting problem is by an index of change.Jacobson and Truax (1991) have argued for the need for indices of clinical significant change.They proposed a “Reliable Change Index” (RCI) to determine if the magnitude of change for

Table 4. Means and Standard Deviations forthe Total Scores on BDI and SCL-36 by Gender

M SD

BDIFemales 15.89 8.02Males 11.15 7.17Total 15.01 8.04

SCL-36Females 84.71 28.20Males 74.78 16.25Both 82.73 26.45


each case is statistically reliable. Differences between the pre and post scores are divided by thestandard error of the differences yields a ratio that can estimate therapeutic change. The stan-dard error describes the expected distribution assuming that no change has occurred. The obser-vation of a RCI greater than 1.96 is unlikely (p , .05) without real change and thus would beindicative of a clinically significant change. For each case in the study, we calculate a RCI assuggested by Jacobson & Truax (1991) subtracting the degree of discomfort perceived beforetherapy from the degree of discomfort perceived at the end of therapy and dividing this differ-ence by the standard error of the mean at pre.

The results suggest that 66.7% of the sample report a change in terms of degree of dis-comfort on the presenting problem above 1.96. In other words, two thirds of the sample reporteda reliable and significant change in terms of the presenting problem.

With the ratings of benefit/harm from psychotherapy, 59.5% reported moderate or abovemoderate benefit. Again, the scale for this question ranged from1100 (extreme benefit) to2100 (extreme harm). A moderate benefit was equivalent to 50 points on the scale. Of theclients, 17.7% indicated no benefit or harm from therapy. Two participants (2.7%) indicatedsome degree of harm.

For satisfaction with psychotherapy, the item mean was 2.99 in a 4-point scale, in which 4indicated much satisfaction and 1 indicated little or no satisfaction. Half the clients (50.5%)reported being satisfied and 11.2% indicated being very satisfied with the psychotherapy ser-vices. About 26.1% reported indifference and 12.2% indicated not being satisfied with theservices. Nearly two thirds of the persons that received psychotherapy indicated feeling satis-fied or very satisfied.

Table 5. Means and Standard Deviations for Degree of Discomfort on the Presenting Problemin Psychotherapy

Presenting Problems M SD

1. Feeling distress, anxiety, depression, or anger 4.26 1.192. Other 3.84 1.803. Low self esteem 3.58 1.564. Difficulties getting along with family members (not spouse) 3.46 1.545. Not understanding myself 3.42 1.596. Reacting too emotionally to events 3.28 1.547. Not having a sense of direction or goals in life 3.10 1.648. Unsatisfactory social life 2.94 1.669. Not managing life well in general 2.83 1.54

10. Problem handling family responsibilities 2.76 1.5611. Problems developing or managing my career 2.70 1.6712. Problems with my spouse or romantic partner 2.60 1.7913. Feeling uncomfortable with people in general 2.48 1.5414. Not getting things done at work or school 2.34 1.5715. Problems getting along with friends 2.30 1.5616. Problems getting along with people at school/work 2.23 1.4417. Eating problems or weight control 2.21 1.5418. Illness, pain, or physical problems 2.18 1.5419. Difficulties caused by the substance abuse or emotional

problems of a member of my family2.14 2.13

20. Difficulties forming or maintaining intimate relations 2.06 1.4522. Sexual problems 2.01 1.5123. Missing work or school or not getting there on time 1.99 1.4424. Alcohol or drug use 1.32 .81


Factors Associated to Therapy Effectiveness

To evaluate the factors associated to the effectiveness of psychotherapy a data reduction strat-egy was employed. Specifically, principle components analyses were used to construct factorsbased on the study variables of interest. We began with a factor for effectiveness that explained72% of the variance shared between the reliable change index for the presenting problem,benefit, and satisfaction. The second factor was based on two measures for the psychotherapyalliance: The IPAS-13 (X 5 69.16,SD5 19.16) and six items from Howard’s (1986) question-naire (X 5 25.08,SD5 6.2) included in the FQ. The alliance factor accounted for 91% of thevariance between the two scale previously mentioned. Finally, a factor of symptom severitywas constructed that accounted for 84% of the variance between the BDI and the SCL-36scores prior to treatment.

This data reduction strategy enabled us to develop a conceptual framework that guided sub-sequent analyses: predictive factors on clients’characteristics such as symptom severity; media-tional factors of psychotherapy process such as the alliance and number of therapy sessions; andcriterion factors (outcome) on effectiveness based on change in the ratings of distress on the pre-senting problem prior and after therapy, ratings of benefit, and satisfaction from therapy.

Associations Between the Factors and Effectiveness

Asetofcorrelationsbetweenmonthssince therapy terminated,symptomseverity,age,salary,num-ber of sessions, alliance and the effectiveness factor were performed. Positive and significant as-sociations were found between symptom severity (r 5 .22), age (r 5 .24), number of sessions (r 5.18) and alliance (r 5 .73) with the effectiveness factor. These correlations suggest that older cli-ents, those with higher symptom severity at the beginning of treatment, those that received moresessions, and those that established a stronger therapeutic alliance report better outcomes.Also, apositive association between the alliance factor and the number of sessions (r 5 .22) was found,suggesting that the stronger the alliance the more number of sessions are received.

The magnitude and direction of these correlations by gender were also examined. In gen-eral, the same associations were maintained for females. For males, however, only the alliancewas associated to the factor of effectiveness (r 5 .75).

Most of the correlations found were of a relatively small magnitude except for the associ-ation between alliance and effectiveness. While we expected correlations of a somewhat largermagnitude, the degree of associations found enabled us to meet one of the basic assumptions ofmultiple regression (i.e., that the predictive variables are not highly intercorrelated).

Factors Predictive of Effectiveness

A multiple regression equation was constructed, using the enter method, to evaluate the pre-dictive value of the following factors in relation to the effectiveness (outcome) in psychother-apy: time since therapy ended (months), symptom severity at the beginning of therapy (BDI &SCL-36), client characteristics (age and salary), numbers of sessions in therapy, and the psy-chotherapy alliance. Since we were interested in evaluating the proportion of variance in theeffectiveness factor explained by the predictive factors, the predictive factors were entered intothe equation in the following steps: time since therapy ended (months), symptom severity at thebeginning of therapy (BDI & SCL-36), client characteristics (age and salary), number of ses-sions, and the psychotherapy alliance factor.

In summary, the results of the regression analysis appear in Table 6. Time out of therapy(months) did not explain a significant part of the variance for the effectiveness factor (R2 5.0001, F5 .001, df5 1,77). The proportion of the variance explained by symptom severity at


the beginning of treatment (second step), in addition to the variance explained by the time outof therapy (first step) was not significant (R2 5 .05, F5 1.86, df5 2,76). Client’s age andsalary (third step) contributed about 4% to the prediction of effectiveness above the exclusivevariance accounted by the first two steps (R2 5 .09, F5 1.78, df5 4,74). Number of sessions(fourth step) only increased the variance accounted by about 3% over the prior steps (R2 5 .12,F 5 1.96, df 5 5,73). Finally, the proportion of variance on effectiveness explained by thepsychotherapy alliance (fifth step) over the prior steps was highly significant (R2 5 .57, F515.81, df5 6,72). The regression coefficients of the first four steps were not significant. Theregression coefficient for the factor of alliance and number of sessions was highly significant.

The findings suggest that process variables such as the alliance and the number of therapysessions are predictive of the effectiveness of psychotherapy. In other words, the alliance is animportant element in predicting effectiveness, defined as changes in the presenting problem,benefit, and satisfaction with therapy.

DISCUSSION

Although psychotherapy research is one of the cornerstones of the discipline of clinical psy-chology (Bergin & Lambert, 1978; Paul, 1967; Smith & Glass, 1977) this area of research hasnot received much attention among U.S. Latinos or in Latin American countries, includingPuerto Rico (Bernal, 1993a; Beutler & Crago, 1991). While there is a substantial body ofpsychotherapy research in North America and parts of Europe on the efficacy and effectivenessof a variety of psychosocial treatments, the results from much of this research is not necessarilygeneralizable to ethnic minorities in the U.S. (Bernal, 1993a; Mays & Albee, 1992; Miranda,1996; Sue, Zane, & Young 1994), Latin Americans in general, and specifically Puerto Ricans(Cervantes & Castro, 1985; Greene, 1987; Rogler, Malgady, & Rodríquez, 1989). Therefore,researchers interested in working with ethnic minorities and Third World communities face thedual task of evaluating instruments and models of interventions already developed in othercontexts and evaluating and developing psychosocial interventions that take into account lan-guage, culture, and other factors that respond to a Latino context (Bernal, Bonilla, & Bellido,1994; Bravo, Canino, Rubio–Stipec, & Woodbury–Fariña, 1991).

One way to approach the study of effectiveness of psychosocial treatments is from theclient’s perspective on the evaluation of such services. From this study, which was based on theclient’s perspective, we may conclude that psychotherapy was perceived as relatively effective.Over two thirds of the sample reported a reliable change in the presenting problem that brought

Table 6. The Proportion of Variance (R2 ), Regression Coefficients (B), Standard error ofB(SEB), and t Values for Months (Since Treatment Ended), Symptom Severity (at Start ofTreatment), Age & Income, Number of Sessions, and Alliance on the Effectiveness Factor

Factors and Variables R 2 B SEB t

Months .000 .001 .003 .299Symptom severity .047 .110 .080 1.380Age & .091 .010 .000 1.195Income .000 .009 −1.462Sessions .119 .001 .008 .195Alliance .568 .712 .082 8.664***

***p < .001.R2 reflects increment.


them to therapy, 60% indicated having benefited (moderately or more) from treatment, andover 60% indicated satisfaction from treatment. These findings are congruent with the resultsreported in studies of effectiveness (Beutler & Crago, 1991; Howard, Kopta, Krause, & Orlin-sky, 1986; Luborsky, Auerback, Crits–Cristoph, Mints, 1988; Orlinsky & Howard, 1986; Selig-man, 1996; Strupp, Wallach, & Wogan, 1964).

Nearly 34 years ago, Strupp, Wallach, and Wogan (1964) conducted a study on the expe-rience of psychotherapy from the client’s perspective. This study was of an exploratory anddescriptive nature. The authors’ aim was to gain a better understanding of the factors betweenclients and therapist that explained therapeutic change. A total of 44 former patients and theirrespective therapists participated in the study by completing an extensive questionnaire. Strupp,Wallach, and Wogan (1964) found a consensus among therapists and clients on the essentialfactors associated to treatment outcome. The majority of participants indicated having ben-efited from psychotherapy. Specifically, 55% indicated having felt extremely satisfied with theresults of treatment, 23% indicated moderate satisfaction, and 20% indicated different degreesof ambivalence. This study was one of the first efforts to systematically evaluate the effective-ness of psychotherapy from the clients’ perspective.

More recently, theConsumer Reports(CR) (1995) study on the effectiveness of psychother-apy (Seligman, 1995) has been one of the most extensive efforts to document the client’s experi-ence of psychotherapy. The CR study found that 44% of the respondents who were emotionallyworse at the beginning of therapy reported feeling good at the end of treatment.Another 43% whobegan therapy as “fairly poor” markedly improved as a result of treatment. The study concludesthat the “vast majority said that therapy helped” (Consumer Reports, 1995, p. 735).

The CR report and Seligman’s (1995) article on the CR study have stimulated a great deal ofcontroversy on the epistemology and methodology of psychotherapy research. Indeed a completeissue of theAmerican Psychologistwas centered on this debate. However, little if any discussionhas focused on the nature of the CR sample. For example, 180,000 subscribers were invited to par-ticipate in the survey of mental health as well as of other services and products. Over 22,000 re-sponded and almost 7,000 of these answered questions related to mental health. Nearly 4,100individuals sought some combination of professional and nonprofessional help. Of these 4,100,2,900 consulted a mental health professional. While the CR sample is clearly one of the largest inthe history of psychotherapy research, Seligman (1996) indicates that the sample was predomi-nantly middle class, white, educated, female, and characterized by individuals who seek treat-ment. Given the nature of the sample, it is unclear how appropriate is the generalization of thesefindings to ethnic minorities, language minorities, and other Third World communities. The ex-ternal validity of the CR study for ethnic minorities and other groups is unknown.

Part of the problem in conducting studies on effectiveness in a Spanish speaking context isthe lack of adequate instruments to evaluate psychosocial interventions. One of the objectivesof this study was to develop, translate, and adapt instruments appropriate to the Puerto Ricancontext. In this area, we can conclude that the instruments and items translated and adaptedwere equivalent to the original instruments in terms of semantic, thematic, technical, content,conceptual, and structural criteria. In addition, all of the instruments used in the study hadadequate psychometric properties (reliability) for the population studied.

Another goal of the study was to explore the relationship between client characteristics, psy-chotherapy process, and effectiveness from the client’s perspective. In this sample, we found thatsymptom severity, age, number of sessions, and the therapy alliance are associated to effective-ness. However, the correlation coefficients were relatively low with the exception of the therapyalliance.

The therapy alliance factor accounted for 45% of the variance of effectiveness. Theseresults suggest that the psychotherapy alliance merits further investigation in effectivenessstudies. The relationship with the therapist and the elements of a positive relationship, from the


client’s point of view, may be a key element of positive outcomes. These findings have impor-tant implications for clinical practice. For example, the evaluation of strengths and weaknessesin the therapeutic alliance at different points in the course of therapy and the attention to basicelements of the therapy relationship (e.g., trust, support, shared tasks and goals, etc.) is likelyto produce beneficial outcomes. One way to strengthen these basic aspects of the therapyalliance is to incorporate culturally sensitive criteria in the treatment itself that consider thelanguage, metaphors, content, concepts, goals, method, and context for specific minority orcultural groups (Bernal, Bonilla, & Bellido, 1994).

The predictive value of the psychotherapy alliance on outcome reported in our study isconsistent with data on the relative value of different methods of matching treatment to patientsin psychotherapy research (Beutler, 1989). In most studies, about 50% of the variance in out-come is derived from error variance. In our study, we found that about 55% of the variance wasunexplained. However, 45% of the variance on outcome was explained by the alliance factor.While our study did not attempt an a priori match between patient and therapist, the alliance isin a sense a measure of the fit between patient and therapist. Our findings support the Luborskyet al. (1986) argument that psychotherapy outcome may be influenced more by the fit betweenpatient and therapist than by other factors such as types of treatment.

This study also has implications for the design of psychotherapy effectiveness research inclinical settings and with minority populations. The design of this study had several uniquefeatures. First, the treatment “as delivered” was evaluated by clients. Similar to the CR study,the clients perspective on the treatment was the basic criteria for determining outcome. Second,we integrated retrospective data with current information. While retrospective reports of thetherapy alliance may have been distorted by the passage of time, the meaning of these reportsoffers important information on the nature of the alliance itself. Recollections or memories ofthe alliance in psychotherapy are not necessarily arbitrary elaborations of experience since theyare based on the construction of empirical events.

The design used serves as an alternative methodology to carry out research with minoritypopulations. In the absence of, and perhaps even in addition to, clinical trials with ethnicminority groups, effectiveness studies contribute to clinical practice and to the field of psycho-therapy research. For example, since the effectiveness rates found in our sample are similar tothose reported by Strupp, Wallach, and Wogan (1964) and in the CR study (1995), the validityof our findings, are in part supported. Furthermore, utilizing retrospective reports of treatmentin combination with data form the clinical records and with ongoing reports of treatment progressmay be the only way for clinics that serve ethnic and language minorities to assess the effec-tiveness of treatment. Such evaluations can contribute to the scant literature of psychotherapyoutcome research with ethnic minorities.

There are a number of conditions that limit the conclusions and generalizations that may bedrawn from this study. First, as noted earlier, this study was based on an exploratory strategy.We used a quasiexperimental design of individuals who had received psychotherapy at a psy-chological services and research center. Second, the providers of these services were graduatestudents at a training clinic.1 Third, psychotherapy as the intervention was not controlled ormeasured in any way. Fourth, the study did not include a control or comparison group.

Nevertheless, our findings are remarkably consistent with the CR study (1995) on thebenefits of psychotherapy, as well as with other findings reported in the literature (Orlinsky &

1 Clearly the use of trainees in an effectiveness study may be questioned. However, given the current climate ofmanaged care in the delivery of mental health and psychotherapy services, the trainees in our study (who meet withminimal qualification to deliver services and were under close faculty supervision) were probably not too dissimilar tothe average managed care provider with a master’s degree. At a minimum, the findings reported have implications fortherapists-in-training.


Howard, 1986; Strupp, Wallach, & Wogan, 1964). Also, we have taken an important step in thearea of instrument development and in the development of preliminary information on theeffectiveness of psychotherapy in a Puerto Rican context, which may have implications forother Latin American contexts, as well as for Latinos and Latinas in the United States. At thispoint, prospective studies of effectiveness are feasible with more sophisticated effectivenessresearch questions, instruments, and designs. A next step might be a prospective study of pre topost changes in community clinics with the instruments and indices developed.

REFERENCES

Alexander, L.B., & Luborsky, L. (1986). The Penn Helping Alliance Scale. In L.S. Greenberg & W.M.Pinsoff (Eds.),The psychotherapeutic process: A research handbook(pp. 325–366). New York:Guilford.

Alvarez, M.A. (1991).The therapeutic alliance in family therapy: Intergenerational family therapy anddrug abuse.Unpublished master’s thesis, The Wright Institute, Berkeley.

Beck, A.T., Ward, C.H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measur-ing depression.Archives of General Psychiatry, 4, 561–571.

Bergin, A.E., & Lambert, M. (1978). The evaluations of therapeutic outcomes. In S.L. Garfield & A.E.Bergin (Eds.),Handbook of psychotherapy and behavior change. (pp. 139–181). New York: Wiley.

Bernal, G. (1993a). Psychotherapy research: Introduction to the special issue (in Spanish).Interamer-ican Journal of Psychology, 27, 127–128.

Bernal, G. (1993b). From the clinic to the community and from the community to the clinic: Develop-ment of a psychological services and research center (in Spanish).Revista Argentina de ClínicaPsicológica, 2, 30–43.

Bernal, G., Bonilla, J., Alvarez, M.A., & Graux, B. (1993). The psychotherapy alliance as apredictor of outcome: A preliminary study (in Spanish).Interamerican Journal of Psychology, 27,229–238.

Bernal, G., Bonilla, J., & Bellido, C. (1994). Ecological validity and cultural sensitivity for outcomeresearch: Issues for the cultural adaptation and development of psychosocial treatments with His-panics.Journal of Abnormal and Child Psychology, 23, 67–82.

Bernal, G., Bonilla, J., & Santiago, I.J. (1995). Psychometric properties of the BDI and the SCL-36in a Puerto Rican sample (in Spanish).Revista Latinoamericana de Psicología, 27, 207–230.

Bernal, G., Padilla, L., Pérez–Prado, E.M., Bonilla, J., & Serrano, M. (1995). La alianzapsicoterapéutica: Evaluación de dos instrumentos y desarrollo de una escala breve de la alianzapara la terapia individual.Río Piedras: Centro Universitario de Servicios y Estudios Psicológicos,Universidad de Puerto Rico.

Beutler, L.E. (1989). Differential treatment selection: The role of diagnosis in psychotherapy.Psycho-therapy, 26, 271–281.

Beutler, L.E., & Crago, M. (Eds.), (1991).Psychotherapy research: An international review of pro-grammatic studies.Washington, DC: APA.

Bordin, E.S. (1985).Research on therapeutic alliance.Paper presented at the Society for PsychotherapyResearch, Dallas, Texas.

Bravo, M., Canino, G.J., & Bird, H. (1987). El DIS en Español: Su traducción y adaptación en PuertoRico. Acta Psiquiátrica Psicológica de America Latina, 33, 27–42.

Bravo, M., Canino, G.J., Rubio–Stipec, M., & Woodbury–Fariña, M. (1991). A cross-culturaladaptation of a psychiatric epidemiology instrument: The Diagnostic Interview Schedule adaptationin Puerto Rico.Culture, Medicine, and Psychiatry, 15, 1–18.

Cervantes, R.C., & Castro, F.G. (1985). Stress, coping and Mexican American mental health: Asystematic review.Hispanic Journal of Behavioral Sciences, 7, 1–73.

Consumer Reports.(1995). Mental health: Does therapy help? 734–739.


Greene, R.N. (1987). Ethnicity and MMPI performance. A review.Journal of Consulting and ClinicalPsychology, 55, 497–512.

Hartley, D.E. (1985). Research on the therapeutic alliance.Psychiatric update, American PsychiatricAssociation Annual Review, 4, 532–549.

Howard, K.I. (1986).Psychotherapy process and outcome: Final report.Evanston, IL: NorthwesternUniversity.

Howard, K.I., Kopta, S.M., Krause, M.S., & Orlinsky, D.E. (1986). The dose-effect relationship inpsychotherapy.American Psychologist, 41, 159–164.

Howard, K.I., Orlinsky, D., & Luger, R. (1994). Clinically relevant outcome research in individualpsychotherapy.British Journal of Psychiatry, 165, 4–8.

Jacobson, N.S., & Truax, R. (1991). Clinical significance: A statistical approach to defining meaning-ful change in psychotherapy research.Journal of Consulting and Clinical Psychology, 59, 12–19.

Luborsky, L., Crits–Christoph, P., McLellan, A.T., Woody, G., Piper, W., Liberman, B., Imber,S., & Pilkonis, P. (1986). Do therapists vary much in their success? Findings from four outcomestudies.American Journal of Orthopsychiatry, 56, 501–502.

Luborsky, L., Auerback, A., Crits–Christoph, P., & Mints, J. (1988).Who will benefit from psy-chotherapy?: Predicting therapeutic outcomes.New York: Basic Books.

Mays, V.M., & Albee, G.W. (1992). Psychotherapy and ethnic minorities. In D.K. Freedheim (Ed.),History of psychotherapy: A century of change.(pp. 552–570). Washington, DC: American Psycho-logical Association.

Miranda, G. (1996). Introduction to the Special Section on recruiting and retaining minorities in psy-chotherapy research.Journal of Consulting and Clinical Psychology, 64, 848–850.

McNeil, D.E., Greenfield, T.K., Atkinson, C.C., & Binder, R.L. (1989). Factor structure of a briefsymptom checklist for acute psychiatric inpatients.Journal of Clinical Psychology, 45, 66–72.

Orlinsky, D., & Howard, K.I. (1986). Process and relation to outcome. In S.L. Garfield & A.E. Bergin(Eds.),Handbook of psychotherapy and behavior change.New York: Guilford.

Paul, G.L. (1967). Strategy of outcome research in psychotherapy.Journal of Consulting Psychology,31, 109–118.

Pinsoff, W.M. & Catheral, D.R. (1986). The integrative psychotherapy alliance: Family, couple andindividual therapy.Journal of Marital and Family Therapy, 12, 137–151.

Roberts, R.E., Atkinson, C.C., & Mendias, R.M. (1984). Assessing the client satisfaction question-naire in English and Spanish.Hispanic Journal of Behavioral Sciences, 6, 385–395.

Rogler, L.H., Malgady, R.G., & Rodríguez, O. (1989).Hispanics and mental health: A frameworkfor research.Malabar, FL: Robert & Kruger.

Seligman, M.P.E. (1995). The effectiveness of psychotherapy: The Consumer Reports Study.AmericanPsychologist, 50, 965–974.

Seligman, M.P.E. (1996). Science as an ally of practice.American Psychologist, 51, 1072–1079.

Smith, M.L., & Glass, G.V. (1977). Meta-analysis of psychotherapy outcome studies.American Psy-chologist, 32, 165–180.

Strupp, H.H., Wallach, M.S., & Wogan, M. (1964). Psychotherapy experience in retrospect: Ques-tionnaire survey of former patients and their therapists.Psychological Monographs: General andApplied, 78, 1–45.

Sue, S., Zane, N., & Young, K. (1994). Research on psychotherapy with culturally diverse populations.In A.E. Bergin & S.L. Garfield (Eds.),Handbook of psychotherapy and behavior change(4th ed.,pp. 783–820). New York: Wiley.

Vessey, J.T., & Howard, K.I. (1993). Who seeks psychotherapy?Psychotherapy, 30, 546–553.

Vasquez, C., & Sanz, J. (1991).Fiabilidad y validez factorial de la versión española del Inventario deDepresión de Beck.Paper presented at the III Congress on Psychological Evaluation, Barcelona,September.


factors associated to outcome in psychotherapy: an effectiveness study in puerto rico

Documents