Prognostic Significance of the ‘‘Surprise’’ Questionin Cancer Patients

Alvin H. Moss, M.D., June R. Lunney, Ph.D., R.N., Stacey Culp, Ph.D., Miklos Auber, M.D.,Sobha Kurian, M.D., John Rogers, M.D., Joshua Dower, M.D., and Jame Abraham, M.D.

Abstract

Background: Physicians consistently overestimate survival for patients with cancer. The ‘‘surprise’’ question—‘‘Would I be surprised if this patient died in the next year?’’—improves end-of-life care by identifying patientswith a poor prognosis. It has not been previously studied in patients with cancer.Objective: To determine the efficacy of the surprise question in patients with cancer.Design: Prospective cohort study.Setting: Academic cancer center.Patients: 853 consecutive patients with breast, lung, or colon cancer.Measurements: Surprise question classification and patient status at 12 months, alive or dead, by surprisequestion response.Results: Oncologists classified 826 of 853 prospective patients with cancer (97%) with 131 (16%) classified intothe ‘‘No’’ group and 695 (84%) into the ‘‘Yes’’ group. In multivariate analysis, a ‘‘No’’ response identified patientswith cancer who had a seven times greater hazard of death in the next year compared to patients in the ‘‘Yes’’group (HR 7.787, p< 0.001).Limitations: Single center study.Conclusion: The surprise question is a simple, feasible, and effective tool to identify patients with cancer whohave a greatly increased risk of 1-year mortality.

Introduction

In patients with advanced cancer, failure to estimateand communicate prognosis can lead to overly aggressive

treatment at the end of life with less attention to importantpalliative care issues such as pain and symptom managementand patients’ values and goals for care.1,2 The ‘‘surprise’’question—‘‘Would I be surprised if this patient died inthe next year?’’—has been recognized as an innovation toimprove end-of-life care in the primary care3 and dialysispopulations4,5 by identifying patients with a poor prognosiswho are appropriate for palliative care. Though it has beenused as a tool to screen patients with advanced cancer,6 out-comes of oncologists’ use of the surprise question have notbeen previously assessed. The purposes of the study were todetermine the feasibility and outcomes of the use of the sur-prise question in a population of patients with cancer.

Methods

Between July and November 2007, four oncologists pro-spectively classified consecutive patients with breast, lung, or

colon cancer as they were being seen for scheduled follow-upvisits at the Mary Babb Randolph Cancer Center of WestVirginia University into ‘‘Yes, I would be surprised’’ and ‘‘No,I would not be surprised’’ groups based on the surprisequestion. The oncologists classified patients according to theirclinical judgment; they did not receive specific training inanswering the question. They completed their fellowships atfour separate programs and had been in oncology practice fora mean of 16 years (range 5–32 years). The oncologists did notshare their classifications with patients or families, allowingthem complete candor in their responses. Patients’ demo-graphics, type and stage of cancer, advance directive com-pletion, presence of a do-not-resuscitate order, and completionof a Physician Orders for Scope of Treatment (POST) formwere collected and their status at 1 year post-visit—alive ordead—was determined.

The research protocol was approved by the West VirginiaUniversity Institutional Review Board for the Protection ofHuman Subjects. Chi-square analysis was used to comparethe differences in the proportions of categorical variables.Univariate and multivariate Cox proportional hazards

West Virginia University, Morgantown, West Virginia.Accepted February 15, 2010.

JOURNAL OF PALLIATIVE MEDICINEVolume 13, Number 7, 2010ª Mary Ann Liebert, Inc.DOI: 10.1089/jpm.2010.0018

837

regression analyses were used to determine the prognosticfactors most highly associated with patients’ status at 12months, alive or dead. A p value of <0.05 was consideredstatistically significant.

Results

Oncologists classified 826 of 853 prospective patients withcancer (97%); 131 (16%) were classified into the ‘‘No’’ groupand 695 (84%) into the ‘‘Yes’’ group (Table 1). The ‘‘No’’ grouppatients were older, more likely to have a diagnosis of lungcancer, and more likely to have Stage IV disease (all p< 0.05).They were significantly more likely to have completed anadvance directive or to have obtained a do-not-resuscitatecard or a POST form than patients in the ‘‘Yes’’ group (71% vs.44%, p< 0.001). At the end of 12 months, 71 patients had died(8.3%); 41% of the ‘‘No’’ patients had died compared to 3% ofthe ‘‘Yes’’ patients ( p< 0.001). In univariate (Table 2) andmultivariate analysis (Table 3) the surprise question ‘‘No’’response was more predictive of patient death than type orstage of cancer or age. The ‘‘Yes’’ patients lived longer than the‘‘No’’ patients (Fig. 1). The sensitivity of the surprise ques-

tion ‘‘No’’ response was 75% and the specificity was 90%.The positive predictive value was 41% and the negative pre-dictive value was 97%.

Discussion

There is a growing body of evidence on cancer communi-cation near the end of life. Discussing prognosis is one of themore challenging aspects of this communication.7 The abilityto estimate prognosis is key to the timing of the communi-cation. A failure to estimate prognosis may lead to a delay incommunication and in identifying the patient’s values andwishes for subsequent treatment, which may include a pref-erence for palliative care and hospice referral rather thancontinued aggressive therapy.

Studies of physicians’ estimates of survival in patients withcancer have consistently shown that they overestimatesurvival.8,9 Researchers are seeking to improve prognosticaccuracy for patients with cancer by developing an integratedprognostic model that combines clinical predictions andprognostic factors such as functional status, nutritional status,and comorbidities.10,11 The surprise question is one suchclinical prediction tool that has been validated as part of anintegrated prognostic model for dialysis patients,5 but it hasnot previously been tested in patients with cancer.12 The GoldStandards Framework in their Prognostic Indicator Guidancerecommended the surprise question as a trigger to encourage

Table 1. Demographics of Cancer Study Patientsa

Patients All (N¼ 853) Yes (N¼ 695) No (N¼ 131) p value

Mean age (SD) 60 (13) 59 (12) 65 (13) <0.001Men, n (%) 126 (14.8%) 73 (10.5%) 49 (37.4%) <0.001Race, n (%)

White 807 (98.5%) 654 (98.3%) 126 (99.2%)African American 11 (1.3%) 10 (1.5%) 1 (0.8%) 0.91Other 1 (0.1%) 1 (0.2%) 0 (0.0%)

Cancer, n (%)Breast 609 (71.4%) 560 (80.6%) 31 (23.7%)Lung 97 (11.4%) 26 (3.7%) 66 (50.4%) <0.001Colon 147 (17.2%) 109 (15.7%) 34 (26.0%)

Stage, n (%)I 241 (28.3%) 225 (32.4%) 8 (6.1%)

<0.001II 265 (31.1%) 241 (34.7%) 16 (12.2%)III 173 (20.3%) 131 (18.8%) 36 (27.5%)IV 125 (14.7%) 53 (7.6%) 71 (54.2%)

Advance directives, n (%)Yes 407 (47.7%) 305 (43.9%) 93 (71.0%)

<0.001No 446 (52.3%) 390 (56.1%) 38 (29.0%)

a‘‘Yes’’ indicates patients in the ‘‘Yes, I would be surprised’’ group. ‘‘No’’ indicates the ‘‘No, I would not be surprised’’ group.

Table 2. Univariate Cox Regression

to Predict Status at 1 Year

Variable Hazard 95% CI p value

Age 1.022 (1.003, 1.041) 0.024Gender (reference¼male) 0.222 (0.138, 0.356) <0.001Type of cancer

(reference¼ lung)Breast 0.088 (0.050, 0.153) <0.001Colon 0.359 (0.204, 0.633) <0.001Stage of cancer 3.657 (2.746, 4.870) <0.001Advance directive 2.888 (1.722, 4.843) <0.001‘‘Surprise’’ question answer

(reference¼ yes)20.048 (11.735, 34.249) <0.001

Table 3. Multivariate Cox Regression

to Predict Status at 1 Year

Variable Hazard 95% CI p value

Age 1.006 (0.987, 1.025) 0.555‘‘Surprise’’ question answer 7.787 (4.158,14.583) <0.001Stage of cancer 2.152 (1.578, 2.936) <0.001Type of cancer 0.862 (0.640,1.161) 0.328

838 MOSS ET AL.

better prediction of patients who might benefit from pallia-tive care, including discussions about end-of-life care.13 Inmultivariate analysis in this study, a ‘‘No’’ response to thesurprise question identified patients with cancer who had aseven times greater hazard of death in the next year com-pared to patients in the ‘‘Yes’’ group. By countering thetendency of physicians to overestimate prognosis, the sur-prise question incorporated into an integrated prognosticmodel offers the potential of improving physicians’ prog-nostic accuracy.

The surprise question may improve prognostic accuracybecause it requires physicians to frame prognosis in a broaderperspective and allows physicians to think in a new way abouttheir patients. Physicians do not need to definitely concludefrom the question that a patient is dying. The question forcesthem to consider whether the patient might be dying. It is anintuitive tool. It is also a feasible tool. In this study busy cancercenter oncologists classified 97% of consecutive patients. Inthis study the positive predictive value of the surprise questionwas not high, but positive predictive value calculations areknown to be dependent on factor prevalence and the 1-yearmortality in this cancer population was only 8.3%. The sensi-tivity of 75% does not negate the fact that the surprise questionis a simple screening test to use to determine if treatment forparticular patients with cancer should include advance careplanning and other palliative care interventions. Furthermore,as noted above it is not proposed that the surprise question beused alone to estimate prognosis.

There are several limitations to this study. The sample waslimited to one population of patients with cancer being treatedby four academic oncologists in West Virginia, and the ma-jority of the patients had breast cancer. Second the study pa-tient population was not ethnically diverse and thus theresults may not be generalizable. Third, because this waspreliminary research to determine the feasibility and out-comes of the surprise question in cancer, other prognosticfactors such as functional and nutritional status were notcollected. Fourth, no specific intervention was undertaken as aresult of a ‘‘No’’ classification. Although patients classified as

‘‘No’’ were significantly more likely to have completed anadvance directive, outcomes such as hospice referrals werenot tracked. Because of these multiple limitations, althoughthe findings with regard to the greatly increased hazard ofdeath in the ‘‘No’’ patients are robust, they need to be con-sidered tentative.

Taken together, our findings show that the surprise ques-tion is a simple, feasible, and effective tool to identify patientswith cancer with a greatly increased risk of 1-year mortality.Further research is necessary to determine if use of the sur-prise question in an integrated prognostic model for patientswith cancer improves prognostic accuracy and if earlieridentification of patients with cancer with poor prognosesusing the surprise question leads to improved patient careoutcomes.

Acknowledgment

The authors thank Cynthia McMillen for assistance withdata collection and analysis and manuscript preparation.

Author Disclosure Statement

No competing financial interests exist.

References

1. Weeks JC, Cook EF, O’Day SJ, Peterson LM, Wenger N,Reding D, et al.: Relationship between cancer patients’ pre-dictions of prognosis and their treatment preferences. JAMA1998;279:1709–1714.

2. Matsuyama R, Reddy S, Smith TJ: Why do patients choosechemotherapy near the end of life? A review of the per-spective of those facing death from cancer. J Clin Oncol2006;24:3490–3496.

3. Pattison M, Romer AL: Improving care through the end oflife: launching a primary care clinic-based program. J PalliatMed 2001;4:249–254.

4. Moss AH, Ganjoo J, Sharma S, Gansor J, Senft S, Weaner B,et al.: Utility of the ‘‘surprise’’ question to identify dialysispatients with high mortality. Clin J Am Soc Nephrol 2008;3:1379–1384.

5. Cohen LM, Ruthazer R, Moss AH, Germain MJ: Predictingsix-month mortality for patients who are on maintenancehemodialysis. Clin J Am Soc Nephrol 2010;5:72–79.

6. Robinson TM, Alexander SC, Hays M, Jeffreys AS, OlsenMK, Rodriguez KL, et al.: Patient-oncologist communicationin advanced cancer: predictors of patient perception ofprognosis. Support Care Cancer 2008;16:1049–1057.

7. Back AL, Anderson WG, Bunch L, Marr LA, Wallace JA,Yang HB, Arnold RM: Communication about cancer nearthe end of life. Cancer 2008;113(7 suppl):1897–1910.

8. Lamont EB, Christakis NA: Extent and determinants of errorin doctors’ prognoses in terminally ill patients: prospectivecohort study. BMJ 2000;320:469–473.

9. Lamont EB, Christakis NA: Prognostic disclosure to patientswith cancer near the end of life. Ann Intern Med 2001;134:1096–1105.

10. Muers MF, Shevlin P, Brown J. Prognosis in lung cancer:physicians’ opinions compared with outcome and predictivemodel. Thorax 1996;51:894–902.

11. Lamont EB, Christakis NA: Epidemiology and prognostica-tion in advanced cancer. In: Berger AM, Shuster JL Jr, VonRoen JH, eds. Principles and Practice of Palliative Care and

FIG. 1. Product-limit survival plot of surprise questionoutcomes.

THE ‘‘SURPRISE’’ QUESTION 839

Supportive Oncology. 3rd ed. Philadelphia: LippincottWilliams & Wilkins; 2007:469–475.

12. Lorenz KA, Lynn J, Dy SM, Shugarman LR, Wilkinson A,Mularski RA, et al.: Evidence for improving palliative care atthe end of life: a systematic review. Ann Intern Med 2008;148:147–159.

13. http://www.goldstandardsframework.nhs.uk/ (Last accessedJune 4, 2009).

Address correspondence to:Alvin H. Moss, M.D.

West Virginia UniversityP.O. Box 9022

Morgantown, WV 26506

E-mail: amoss@hsc.wvu.edu

840 MOSS ET AL.