www.ScienceTranslationalMedicine.org 8 May 2013 Vol 5 Issue 184 184fs16 1

F O C U SC

RE

DIT

: Y. H

AM

MO

ND

/SC

IEN

CE

TR

AN

SLA

TIO

NA

L M

ED

ICIN

E

Both the abundance of new therapeutic strategies in the drug-development pipe-line and the high rate of attrition of medi-cal products during clinical trials place extraordinary pressure on stages of drug de-velopment in which clinical activity is f rst evaluated—typically phase 2. T e abundant pipeline demands that such trials quickly evaluate candidates, whereas the prospect of heavy human subject burdens and costs of late attrition demands that phase 2 tri-als accurately predict results of subsequent conf rmatory trials.

Concerns about phase 2 predictivity—both in terms of accurate and ef cient pipe-line screening (that is, negative predictivity) and reduction of late-phase attrition (that is, positive predictivity)—have prompted a series of innovations in phase 2 trial design. However, many of the contemplated trial reforms aimed at boosting phase 2 positive predictivity have important repercussions for human subjects and for the capacity of the research enterprise to discharge its so-cial mission. Here, we articulate four factors that should guide the level of positive pre-dictivity sought in middle stages of clinical development.

QUEST FOR PREDICTIVE TRIALSAs many as two-thirds of the interventions entering phase 3 fail to reproduce success ob-served in phase 2 trials (1). Ostensibly, this poor rate of translation betrays an inef cient use of research resources and needless bur-dens imposed on patient-subjects. Concerns about the number of negative conf rmatory trials have prompted a series of innovations in phase 2 design. T ese include the use of more predictive biomarkers, tiered approaches to outcome assessment, patient enrichment, seamless phase 2/3 designs, larger trials, use of clinical end points, real-time pharmacoki-netic analysis, randomization (for areas such

as oncology in which phase 2 studies use his-torical controls), variations in statistical error rates, and adaptive designs (2, 3).

Reducing false positives in phase 2 trials is ethically attractive for two reasons. First, by reducing occurrence of failure in phase 3, it limits the number of patient-volunteers exposed to unsafe and inef ective drugs. Given that the number of patients in phase 3 studies is typically 10-fold greater than in phase 2, these reductions in subject burden can be substantial. Second, more predictive phase 2 trials enable more ef cient alloca-tion of resources in clinical translation; such studies can free up material and human resources by focusing their deployment on conf rmatory trials that are more likely to meet their end points.

PREDICTIVITY AND PROTECTIONHowever, trial designs aimed at reducing false positives also have costs for human subjects. Some of the gains in subject wel-fare described above are of set by greater burden. Introducing randomization in phase 2 studies, for example, roughly dou-bles the number of patients in trials because they now require comparator arms. Using enrichment designs, pharmacodyamics, or real-time pharmacokinetics all entail more frequent (and of en invasive) tissue collec-tion from volunteers. In areas of vaccine development, the quest for predictive phase 2 designs has kindled interest in phase 2 “challenge studies,” which deliberately infect healthy volunteers with a manageable form of disease.

T ese extra burdens in phase 2 are not morally equivalent to those typically en-countered in phase 3. Risks of drug admin-istration in conf rmatory trials are ethically justif ed by clinical equipoise—a state of collective uncertainty about whether ex-perimental treatment is preferable to stan-dard care—and hence can plausibly claim therapeutic value for subjects. In contrast, the case for clinical equipoise is far weaker

P O L I C Y

Ethics, Error, and Initial Trials of Ef cacySpencer Phillips Hey and Jonathan Kimmelman*

*Corresponding author. E-mail: jonathan.kimmelman@mcgill.ca

Studies for Translation, Ethics, and Medicine Group (STREAM), Biomedical Ethics Unit, McGill University, Montreal, QC H3A 1X1, Canada.

Clinical trial reforms aimed at boosting phase 2 positive predictivity may involve ethical and social trade-of s.

Fig. 1. Balancing act. Shown are four factors that should inform the level of positive predictivity sought in a phase 2 trial.

by guest on June 17, 2020http://stm

.sciencemag.org/

Dow

nloaded from

www.ScienceTranslationalMedicine.org 8 May 2013 Vol 5 Issue 184 184fs16 2

F O C U S

in initial tests of ef cacy because evidence of clinical utility is lacking at the outset, and base rates for discovering a useful interven-tion are low at that stage (4). T e risk of more intensive tissue collection or disease challenge are morally justif ed by ends that are predominantly external to the volunteer.

Minimizing phase 2 false positives also threatens clinical equipoise for subsequent conf rmatory trials (5). If the pretest likeli-hood of successful outcomes are too high, patients randomized to comparators are systematically disadvantaged. Further, knowledge gained per patient enrolled is diminished because less is learned from the successful prosecution of conf rmatory trials.

THE RESEARCH ENTERPRISEPredictive phase 2 trial designs also entail costs for the integrity of the research enter-prise. T e social mission of clinical research is to furnish health care decision-makers with information and evidence for addressing un-met health needs. T e delivery of this social good is threatened by any process that desta-bilizes the types of stakeholder collaborations that are required by clinical research (6), or where the scarce resources of a research en-terprise are not directed in accordance with social priorities. Highly predictive phase 2 designs have potential unintended conse-quences for each.

First, there are social costs associated with contemplated design innovations. Gains in reducing false positives in phase 2 are poten-tially at the expense of greater false negatives (that is, eliminating truly useful agents in phase 2 trials). In areas with few therapeu-tic candidates and pressing clinical need, such loss is associated with large opportunity costs. Further, more predictive designs can strain the capacity of research systems to vet new drug candidates because larger sample sizes, lengthened observation periods, or ex-tensive real-time laboratory analyses demand greater resources. Although preempted con-f rmatory studies free up resources for vetting other candidates, some resources cannot be redirected in this way. For example, the f -nancial resources that pharmaceutical f rms save by avoiding conf rmatory testing will be distributed according to company priorities rather than toward upstream investigations of other candidates for the disease.

Second, highly predictive phase 2 stud-ies threaten a fragile social consensus that enables investigators to randomize patients in conf rmatory trials and regulators to con-dition distribution of new drugs on positive,

replicated trials that have suf cient power to detect at least commonly occurring tox-icities. In recent years, this social consensus has come under repeated attack (7, 8). T e prospect that the ef cacy of new drugs can be reliably inferred af er phase 2 substantially weakens the moral argument for withholding market access until larger studies are com-pleted. T is, in turn, threatens the capacity of the research enterprise to rigorously evaluate new drugs before clinical uptake.

T ird, negative, adequately powered phase 3 studies of en have af rmative sci-entif c and medical value. For agents that have not been licensed, decisive disconf ma-tion—or safety signals—can inform deci-sion-making for the testing of other drugs in the same class. For licensed drugs that are tested in new indications or combinations, adequately powered negative trials warn caregivers against of ering drugs in speci-f ed, of -label applications.

DEFINING A SOCIAL OPTIMUMAt a certain point, the gains from reducing false positives in phase 2 are exceeded by so-cial losses. T us, the task for study planners and funding bodies is to determine the opti-mal level of phase 2 false-positive results—at which the gains in avoiding negative conf r-matory trials outweigh the costs—and then to deploy policies that encourage trialists to approximate this social optimum. We of-fer four factors that should inform the level and type of error tolerance sought in middle stages of clinical development (Fig. 1).

First, clinical equipoise in conf rmatory trials establishes upper and lower boundar-ies for predictivity in phase 2 studies. Al-though the precise placement of these moral boundaries is hotly debated, we suggest that, as a general rule, when the base rate of false positives in phase 2 is high, greater ef ort should be invested in early- to mid-stage testing before advancing drugs into conf rmatory trials; when this rate is low, re-searchers can scale back their ef orts in early and middle stages.

Second, the rate of false positives toler-ated in phase 2 should be inversely propor-tional to the abundance of candidates in the pipeline. Where both pipeline abundance and prior odds of f nding an active agent are low, the opportunity cost of a false negative will be substantial; therefore, phase 2 studies should be relatively permissive with respect to false positives (9). Where pipeline abun-dance is high but prior odds of success are low, studies should aim for higher positive

predictivity so that resources from preempt-ed conf rmatory testing can be redirected toward earlier phases (10). However, if too high a level of positive predictivity is at-tained and resources from preempted phase 3 studies cannot be rerouted upstream, more resource-intensive phase 2 studies will diminish the pool of resources available for screening new candidates, leading to similar opportunity costs.

T ird, predictivity must strike a delicate moral balance. T e moral justif cation for exposing subjects to risk in phase 2 stud-ies dif ers from that for conf rmatory stud-ies. A shif toward more predictive studies entails an escalation of activities in which the medical interests of subjects conf ict with research activities. As a general rule, we suggest that where individuals are least able to protect their own interests or endure harms (for example, pediatric populations and economically deprived populations), researchers should tread cautiously with implementing study designs that increase positive predictivity through increases in volunteer burden.

Fourth, the level of positive predictiv-ity sought should be informed by the social utility of decisively negative conf rmatory trials. We can envision two circumstances with large social value. T e f rst is when conf rmatory-trial outcomes enable the re-search community to update key theories of pharmacology or pathophysiology that are guiding drug development. Insofar as theo-ries drive development of other drugs, dis-conf rmatory trials have substantial social utility because they allow drug developers to reassess priorities. In such circumstances, phase 2 designs should have greater toler-ance for false positives. A second circum-stance in which conf rmatory trials are criti-cal is for costly or risky interventions that have been taken up into clinical practice (for example, the of -label use of a drug). In such circumstances, only decisive disconf r-mations will be adequate for altering clini-cal practice, putting a premium on phase 2 studies that produce fewer false positives.

REFERENCES AND NOTES 1. M. I. Zia, L. L. Siu, G. R. Pond, E. X. Chen, Comparison of

outcomes of phase II studies and subsequent random-ized control studies using identical chemotherapeutic regimens. J. Clin. Oncol. 23, 6982–6991 (2005).

2. A. Stone, C. Wheeler, A. Barge, Improving the design of phase II trials of cytostatic anticancer agents. Contemp. Clin. Trials 28, 138–145 (2007).

3. M. R. Sharma, M. L. Maitland, M. J. Ratain, Other para-digms: Better treatments are identifi ed by better trials:

by guest on June 17, 2020http://stm

.sciencemag.org/

Dow

nloaded from

www.ScienceTranslationalMedicine.org 8 May 2013 Vol 5 Issue 184 184fs16 3

F O C U S

The value of randomized phase II studies. Cancer J. 15, 426–430 (2009).

4. J. Kimmelman, A. J. London, Predicting harms and ben-efi ts in translational trials: Ethics, evidence, and uncer-tainty. PLoS Med. 8, e1001010 (2011).

5. B. Djulbegovic, A. Kumar, A. Magazin, A. T. Schroen, H. Soares, I. Hozo, M. Clarke, D. Sargent, M. J. Schell, Op-timism bias leads to inconclusive results-an empirical study. J. Clin. Epidemiol. 64, 583–593 (2011).

6. A. J. London, J. Kimmelman, M. E. Emborg, Research eth-ics. Beyond access vs. protection in trials of innovative therapies. Science 328, 829–830 (2010).

7. N. Bristol, Should terminally ill patients have access to phase I drugs? Lancet 369, 815–816 (2007).

8. A. Von Eschenbach, “Medical innovation: How the U.S. can retain its lead.” Wall Street Journal, 14 February 2012, Opinion.

9. P. B. Gilbert, Some design issues in phase 2B vs phase 3 prevention trials for testing effi cacy of products or con-cepts. Stat. Med. 29, 1061–1071 (2010).

10. P. P. J. Phillips, S. H. Gillespie, M. Boeree, N. Heinrich, R. Aarnoutse, T. McHugh, M. Pletschette, C. Lienhardt, R. Hafner, C. Mgone, A. Zumla, A. J. Nunn, M. Hoelscher, Innovative trial designs are practical solutions for im-

proving the treatment of tuberculosis. J. Infect. Dis. 205, (suppl. 2), S250–S257 (2012).

Funding: This work was funded by CIHR (EOG 102823). Competing interests: The authors declare that they have no competing interests.

Citation: S. P. Hey, J. Kimmelman, Ethics, error, and initial trials of effi cacy. Sci. Transl. Med. 5, 184fs16 (2013).

10.1126/scitranslmed.3005684

by guest on June 17, 2020http://stm

.sciencemag.org/

Dow

nloaded from

Ethics, Error, and Initial Trials of EfficacySpencer Phillips Hey and Jonathan Kimmelman

DOI: 10.1126/scitranslmed.3005684, 184fs16184fs16.5Sci Transl Med

ARTICLE TOOLS http://stm.sciencemag.org/content/5/184/184fs16

CONTENTRELATED

http://stm.sciencemag.org/content/scitransmed/7/298/298ps16.fullhttp://stm.sciencemag.org/content/scitransmed/6/245/245ed17.fullhttp://stm.sciencemag.org/content/scitransmed/6/221/221fs5.fullhttp://stm.sciencemag.org/content/scitransmed/5/213/213ra165.full

REFERENCES

http://stm.sciencemag.org/content/5/184/184fs16#BIBLThis article cites 9 articles, 2 of which you can access for free

PERMISSIONS http://www.sciencemag.org/help/reprints-and-permissions

Terms of ServiceUse of this article is subject to the

registered trademark of AAAS. is aScience Translational MedicineScience, 1200 New York Avenue NW, Washington, DC 20005. The title

(ISSN 1946-6242) is published by the American Association for the Advancement ofScience Translational Medicine

Copyright © 2013, American Association for the Advancement of Science

by guest on June 17, 2020http://stm

.sciencemag.org/

Dow

nloaded from