Available online at www.sciencedirect.com

General Hospital Psychiatr

y 32 (2010) 334–336

Measuring improvement in depression in cancer patients: a 50% drop onthe self-rated SCL-20 compared with a diagnostic interview☆,☆☆,★,★★

Mark O'Connor, M.B., Ch.B.a,⁎, Isabella Butcher, Ph.D.b, Christian Holm Hansen, M.Sc.a,Annet Kleiboer, Ph.D.a, Gordon Murray, Ph.D.b, Neelom Sharma, M.B., Ch.B.a,

Parvez Thekkumpurath, M.B., B.S.a, Jane Walker, M.B., Ch.B.a, Michael Sharpe, M.D.aaPsychological Medicine Research, University of Edinburgh Cancer Research Centre, School of Molecular and Clinical Medicine,

University of Edinburgh, Edinburgh EH4 2XR, Scotland, UKbPublic Health Sciences Section, Division of Community Health Sciences, The University of Edinburgh Medical School, Edinburgh EH8 9AG, Scotland, UK

Received 23 August 2009; accepted 5 January 2010

Abstract

Objective: To determine the validity of a 50% drop in the 20-item Symptom Checklist Depression Scale (SCL-20) score against the “goldstandard” of no longer meeting criteria for major depression as assessed using a diagnostic interview in an outpatient cancer population andalso to examine the validity of other potential cut-offs (i.e., percentage drops).Materials and Methods: Secondary analysis of data from a randomized trial which compared collaborative care with usual care for cancerpatients with major depression. A total of 194 trial participants who had both SCL-20 scores and depression diagnoses on the StructuredClinical Interview for DSM-IV at both baseline and at 12-week outcome formed the analyzed sample.Results: A 50% reduction in the SCL-20 score from baseline to 12 weeks correctly identified the patients who no longer met criteria formajor depression in 153 (78.9%) of 194 (95% CI 73.1% to 84.6%) cases. Most of those misclassified had not achieved a 50% reduction inSCL-20 score despite no longer meeting criteria for major depression. Examination of the performance of percentage drops other than 50%on the SCL-20 using a receiver operating characteristics (ROC) curve and histogram of misclassification suggested that the 50% drop wasbest if both a low overall misclassification rate and the minimizing of false positives of improvement were required.Conclusions: A 50% reduction in the SCL-20 score performs well as a conservative measure of change in depression status in cancer patients.© 2010 Elsevier Inc. All rights reserved.

Keywords: Depression; MDD; SCL-20; Response; Cancer

1. Introduction

A commonly accepted operational definition of clinicallysignificant improvement in depression status in clinical trials

☆ Work performed at Psychological Medicine Research, The Univer-sity of Edinburgh Cancer Research Centre, School of Molecular and ClinicalMedicine, University of Edinburgh and the Edinburgh NHS Cancer Centre,Edinburgh, Scotland, UK.

☆☆ Funding: Cancer Research UK.★ This work is original and has not been published elsewhere.★★ Disclosures: The authors report no competing interests.⁎ Corresponding author. Psychological Medicine Research, Kennedy

Tower, Royal Edinburgh Hospital, EH10 5HF Edinburgh, UK. Tel.: +440131 537 6934; fax: +44 0131 537 6641.

E-mail address: m.oconnor@doctors.org.uk (M. O'Connor).

0163-8343/$ – see front matter © 2010 Elsevier Inc. All rights reserved.doi:10.1016/j.genhosppsych.2010.01.001

is a decrease of at least 50% from the baseline value of astandard scale [1]. A 50% reduction in the 20-item SymptomChecklist self-rated depression scale (SCL-20) [2–4] hasbeen used to this end in a number of primary care depressiontrials [5–10]. We are, however, not aware of any previouslypublished validation of this 50% change. Furthermore, it ispossible that other cut-offs are better for identifying aclinically significant improvement in depression status. Wetherefore aimed to assess the performance of a 50% drop inthe SCL-20 score as a valid measure of improvement indepression status by comparing it with no longer meetingdiagnostic criteria for major depression as measured by a“gold standard” interview [11,12]. We also aimed to examinealternative cut-offs of reduction in SCL-20 score todetermine which was best.

Fig. 1. Receiver operating characteristics (ROC) curve for a range ofpotential SCL-20 reductions as indicators of improvement in depressionstatus. (The numbers in parentheses present the percentage-wise SCL-20reduction from baseline relating to the given points on the curve.)

335M. O'Connor et al. / General Hospital Psychiatry 32 (2010) 334–336

2. Material and methods

We performed a secondary analysis of data from a clinicaltrial (SMaRT Oncology-1) that had compared collaborativecare with usual care for cancer patients who met criteria formajor depressive disorder (MDD) [13]. We used the baselineand 12-week outcome data from this trial to compare a 50%reduction in SCL-20 score with change in major depressionstatus to no longer meeting criteria as measured using the“gold standard” Structured Clinical Interview for DSM-IV(SCID) [12].

The trial sample was 200 patients with cancer and MDDwho had been recruited by screening outpatient clinics of aregional cancer center in the United Kingdom. At the12-week outcome point, data were available for 194 of thesepatients (four patients were lost to follow-up and two did nothave their MDD status assessed) [13].

The SCL-20 was self-rated by the patients at both baselineand outcome. The MDD section of the SCID was alsoadministered (by trained nurses or psychology graduatessupervised by a psychiatrist) over the telephone at bothbaseline and outcome. All SCID interviews were completedwithin a maximum of 2 weeks of the patient's completion ofthe SCL-20.

We first cross-tabulated the number of patients with andwithout a 50% reduction in SCL-20 score at 12 weeksagainst depression status as determined at interview. We thenanalyzed a range of other potential cut-offs on the SCL-20using a receiver operating characteristics (ROC) curve and ahistogram of the number and type of misclassification.

3. Results

Our sample had a mean age of 56.4 (S.D. 11.9; range 24.9to 87.1 years) and included 136 (70%) females. There was avariety of cancer types (83 breast; 30 gynecological; 20hematological; 19 prostate; 16 testicular; 13 colorectal; 4lung; 4 urinary tract; 3 skin; 2 sarcoma).

As can be seen from Table 1, in the majority of cases a50% reduction from baseline in SCL-20 scores coincidedwith an interview-determined improvement in depressionstatus. Overall, 153 (78.9%) of the 194 (95% C.I. 73.1% to84.6%) patients were classified correctly. The majority

Table 1Change in SCL-20 score compared with change in major depression status

Still meets criteria(% correctly classifiedby SCL-20)

No longer meetscriteria (% correctlyclassified by SCL-20)

Less than 50% reductionin SCL-20 score

77 (91%) 33 (30%)

50% or greater reductionin SCL-20 score

8 (9%) 76 (70%)

Numbers in italics show those correctly identified according to the 50%SCL-20 cut-off.

(33 of 41) of those in whom the SCL-20 and interview werediscrepant had improved to the extent that they no longer metcriteria for major depression but had reported less than a 50%reduction in their SCL-20 score. In other words, the self-report scale tended to underestimate improvement.

Analysis of other potential SCL-20 cut-offs (see Figs. 1and 2) showed that comparably low rates of overall

Fig. 2. Histogram of overall misclassifications and type of misclassificationat a range of potential SCL-20 cut-offs.

336 M. O'Connor et al. / General Hospital Psychiatry 32 (2010) 334–336

misclassification occurred across the range of 25% up to50% reductions in baseline score, but that moving from a25% reduction to a 50% reduction was associated with animprovement in the percentage of patients misclassified asno longer being depressed. In other words, the 50%reduction was more conservative.

4. Discussion

Clinical trials of interventions for depression need to beable to measure change in depression status in a feasiblemanner. Consequently, the scores of self-rated questionnairesare often used. In order to determine a clinically significantchange in this score a 50% reduction from baseline hasfrequently been chosen. We found that using this criterion theself-rated SCL-20 measured improvement was in goodagreement with that measured by a “gold standard”diagnostic interview. The misclassification rate was lowand was conservative in not overestimating improvement.

Examination of the ROC curve and misclassificationshistogram showed similar low rates of misclassification forcut-offs in the range of 25% up to a 50% reduction in SCL-20 scores. However, examination of the types of misclassi-fication from a 25% reduction up to a 50% reduction found acorresponding reduction in the number of patients wronglyidentified as no longer depressed. As the “optimum” cut-offon any scale depends on the purpose for which it is beingused, we would argue that the evaluation of a treatmentrequires a conservative measure with low overall misclassi-fications and a tendency to underestimate rather thanoverestimate significant change.

This study has several limitations: the first concerns thegeneralizability of the findings. Our analysis was of datafrom a trial of depression management in cancer patients inthe UK, and some patients either refused or were too unwellto participate in that trial. It is therefore possible that theresults may not generalize to all cancer patients. The secondlimitation is in measurement. The SCID interview adminis-tered was designed to be used by psychiatrists and in the trialnurses or psychology graduates delivered the interview overthe telephone. However, the interviewers were all trained andsupervised by an experienced psychiatrist and there is goodagreement between telephone and face-to-face SCID inter-views [14]. Thirdly, the SCL-20 and the SCID were not

administered at the same time but within a maximum of 2weeks of each other. There is therefore the possibility thatthere was a change in depression severity during theintervening time. Finally, the change in diagnostic statuswe used as gold standard is only one of many possiblemeasures of clinical significance.

We conclude that a 50% reduction in the SCL-20 scoreperforms well as a conservative measure of change indepression status in cancer patients.

References

[1] Keller MB. Past, present, and future directions for defining optimaltreatment outcome in depression: remission and beyond. JAMA 2003;289:3152–60.

[2] Derogatis LR, Lipman RS, Covi L. SCL-90: An outpatient psychiatricrating scale— preliminary report. PsychopharmacolBull 1973;9:13–28.

[3] Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L. TheHopkins Symptom Checklist (HSCL): a measure of primary symptomdimensions. Mod Probl Pharmacopsychiatry 1974;7:43–5.

[4] Derogatis LR. SCL-90-R®. Administration, Scoring, and ProceduresManual. 3rd ed. Minneapolis (MN): National Computer Systems,Inc.; 1994.

[5] Katon W, Von Korff M, Lin E, et al. Collaborative management toachieve treatment guidelines. Impact on depression in primary care.JAMA 1995;273:1026–31.

[6] Katon W, Robinson P, Von Korff M, et al. A multifaceted interventionto improve treatment of depression in primary care. Arch GenPsychiatry 1996;53:924–32.

[7] Hedrick SC, Chaney EF, Felker B, Liu CF, Hasenberg N, Heagerty P,et al. Effectiveness of collaborative care depression treatment inVeteran's Affairs primary care. J Gen Intern Med 2003;18:9–16.

[8] Corey-Lisle PK, Nash R, Stang P, Swindle R. Response, partialresponse, and nonresponse in primary care treatment of depression.Arch Intern Med 2004;164:1197–204.

[9] Hunkeler EM, Katon W, Tang L, Williams JW, Kroenke K, Lin EH,et al. Long term outcomes from the IMPACT randomised trial fordepressed elderly patients in primary care. Br Med J 2006;332:259–63.

[10] Fortney JC, Pyne JM, Edlund MJ, Williams DK, Robinson DE, MittalD, et al. A randomized trial of telemedicine-based collaborative carefor depression. J Gen Intern Med 2007;22:1086–93.

[11] American Psychiatric Association. Diagnostic and Statistical Manualof Mental Disorders, 4th Edition, Text Revision (DSM-IV-TR).Washington (DC): American Psychiatric Association; 2000.

[12] First MB, Spitzer RL, Williams JBW, et al. Structured ClinicalInterview for DSM-IV (SCID). Washington (DC): American Psychi-atric Association; 1995.

[13] Strong V, Waters R, Hibberd C, Murray GD, Wall L, McHugh GL,et al. Management of depression for people with cancer (SMaRTOncology 1): a randomised trial. The Lancet 2008;372:40–8.

[14] Simon GE, Revicki D, VonKorff M. Telephone assessment ofdepression severity. J Psychiatric Res 1993;27:247–52.