1

Multiplicity Adjustment For Clinical Trials with Two Doses of an Active Treatment and

Multiple Primary and Secondary Endpoints

Hui Quan1, Tom Capizzi1, Ji Zhang2

1Merck Research Laboratories2Sanofi-Synthelabo Research

FDA/Industry Statistics Workshop September 21-23, 2004

2

Outline

Background and clinical trial settingProcedures for a two-dimensional problem Procedures for a three-dimensional problem

Numerical examples for comparing powersReal data exampleProcedures without overall strong controlDiscussion

3

Background

Multiple primary and secondary endpoints are often simultaneously considered in Phase III trials.

Positive findings on secondary endpoints could be for:

1: Supporting primary results or Hypotheses generating

2: Additional claims or description in drug label

For category 2 recent regulatory guidances and publications emphasized to have Type I error rate controlled for secondary endpoints.

4

Background (2) Different approaches for interpreting trial results:

Conventional standard for positive trial: positive effect on primary endpoints.

Others: a trial as positive even when effect is positive only on a secondary endpoint secondary endpoints must be considered in multiplicity adjustment.

For example: Moyé (2000) proposed a PAAS for controlling the FWE rate αF with αP for the primary endpoints and αF-αP=αS for the secondary endpoints.

5

Trial Setting

Trial examples:

Parathyroid hormone fracture trial (Neer et al. 2000): 20 and 40 μg vs. placebo on vertebral fracture and other fractures

Rofecoxib RA trial (Geunese, et al. 2002): 25 and 50 mg vs. placebo on four primary and other secondary endpoints

Estrogen Alzheimer trial (Mulnard, et al. 2000): 0.625 and 1.25 mg vs. placebo on CGIC and other scores

Metformin and rosiglitazone combination trial (Fonseca, et al. 2000): Rosiglitazone 4 and 8 mg vs. placebo on glucose and lipid endpoints

6

Trial Setting (2)

Three-dimensional multiplicity adjustment problem: • Two doses • Two priority categories of endpoints: primary & secondary

• Multiple endpoints in each priority categoryHigher dose-the primary dose, lower dose-the secondary

dose (no monotonic order on parameter space for dose response is assumed for any endpoint)

Trial positive only if positive on one or more primary endpoints – except when PAAS is used for multiplicity adjustment

No further priority order for endpoints within each category

7

Trial Setting (3)

Traditional procedures may be lack of power.

For example: for a trial with two primary and two secondary endpoints-a total of 8 active to control comparisons, if a Hochberg procedure is used and if the lower dose shows no effect on the two secondary endpoints, the higher dose on the primary endpoints will be assessed at most at level of /3.

Since no monotonic order in dose response is assumed, stepdown trend tests are not applicable.

More specific gatekeeping procedures may have higher power.

8

Procedures for a Two-dimensional Problem: Two doses and multiple endpoints

Hij is the null hypothesis for comparing the i th dose to the control on the j th endpoint, and pij is the corresponding p-value, i=1, 2 and j=1, 2, …, g. (Dose 1: the higher and primary dose)

Modified Bonferroni-Closed Procedure (2.1) Step1. Reject all Hij if all pij ≤, i=1,2, j=1,…, g.

Otherwise,

Step 2. Reject H1j if p1j ≤/g

Reject H2j if p1j ≤/g and p2j ≤/g

9

Modified Hochberg-Bonferroni Procedure (2.2)

Step 1. Reject all Hij if all pij ≤, i=1,2, j=1,…, g.Otherwise,Step 2. Use Hochberg procedure for the higher dose at ’=2/3.Step 3. Let l =# of endpoints of no effect for the higher dose. If l=0,

use a Hochberg procedure for the lower dose at level . Otherwise,Step 4: a) Use Hochberg procedure for the lower dose at with all endpoints; b) Assume no effect on the l (>0) endpoints for the lower dose; c) Use Hochberg procedure for the lower dose on the other (g-l) endpoints at level /3; then d) Claim the effect for the lower dose on an endpoint if the effect is significant at both a) and c).

10

Weighted Modified Bonferroni-Closed Procedure (2.3)

Step 1. Reject all Hij if all pij≤Otherwise,Step 2. Reject H1j if p1j ≤wj ;

reject H2j if p1j ≤wj and p2j ≤wj .

where wj ≥0 and Σwj=1

Applicable to a three-dimensional problem by assigning more weights on the primary and less weights on the secondary endpoints. However, different from PAAS, no need to split the significance level if all pij ≤.

11

Properties of Procedures (2.n)

Reject all the hypotheses if all p-values are ≤.Step down from the higher dose to the lower dose

to form closed procedures. Theoretically or based on simulation, provide

strong control on the FWE rate under appropriate conditions.

for (2.1) and (2.2): the condition for the Simes test

for (2.3): positive regression dependency condition for

the weighted Simes test.

12

Procedures for the Three-dimensional Problem

Hij is the null hypothesis for comparing the i th dose to the control on the j th primary endpoint, i=1, 2 and j=1,…, gp

Sik is the null hypothesis for comparing the i th dose to the control on the k th secondary endpoint, i=1, 2 and k=1,…, gs.

Following Westfall and Krishen (2001),

Gatekeeping Procedure (3.1) for gp=1 and gs=1

(a) {H11} → (b) {H21, S11} → (c) {S21}level α for each { }.

13

Generalized Gatekeeping Procedure (3.2) (a) {H1j} (b) {[H2j], [S1k]} One-dimensional

(c) {Sik} Two-dimensional

One-dimensional procedure at (a) and (b), and two-dimensional procedure at (c).

Reject H1j (H2j) if it should be rejected at (a) ((b)) Reject S1k if it should be rejected at both (b) and (c). Reject S2k if all H2j’s have been rejected, S1k has been

rejected at (b) and (c), and S2k should be rejected at (c).

14

Other Procedures for the Three-dimensional Problem

PAAS: {Hij} two-dimensional at level αp and

{Sik} two-dimensional at level αs (αp+αs= α)

Primary-Secondary Gatekeeping:

(a) {Hij} two-dimensional ↓

(b) {Sik} two-dimensional each at level α

Procedure (2.2) or others for the two-dimensional Problems

15

Numerical Examples

(R=Rejection, NR=No Rejection)

4 Procedures are compared to the regular Hochberg Procedure

P1=PAAS with P(2.2) for primary and secondary endpoints

P2=P(2.3) with weight wp for primary and ws for secondary

P3=P(3.2) with P(2.1) for Step (c)

P4=P(3.2) with P(2.2) for Step (c)

16

Numerical Example 1

gp=1, gs=3, αp=.0375, αs=.0125, wp=3/6 and ws=1/6

H11 H12 S11 S12 S13 S21 S22 S23

P-value .005 .018 .006 .014 .070 .012 .018 .100

P1 R R NR NR NR NR NR NR

P2 R R R NR NR NR NR NR

P3 R R R R NR R NR NR

P4 R R R R NR NR NR NR

HB R NR R NR NR NR NR NR

17

Numerical Example 2

gp=2, gs=2, αp=.0375, αs=.0125, wp=3/8 and ws=1/8

H11 H12 H21 H22 S11 S12 S21 S22

P-value .001 .014 .012 .045 .006 .033 .009 .130

P1 R R R NR NR NR NR NR

P2 R R R NR R NR NR NR

P3 R R R R R NR R NR

P4 R R R R R R R NR

HB R NR NR NR R NR NR NR

18

Real Data Example

A randomized trial to compare mitoxantrone 12 mg/m2 (n=64) or 5 mg/m2 (n=64) to placebo (n=60) on patients with worsening relapsing remitting or secondary progressive multiple sclerosis (Hartung et al. 2002).

Here, two primary and two secondary endpoints for which p-values for both doses are available: EDSS, ambulation index, with relapses and admitted to hospital.

19

Real Data Example (2)

gp=2, gs=2, αp=.040, αs=.010, wp=4/10 and ws=1/10

H11 H12 H21 H22 S11 S12 S21 S22

P-value .0194 .0306 .0100 >.050 .0206 .0024 .7150 .2031

P1 NR NR NR NR NR R NR NR

P2 R NR R NR NR R NR NR

P3 R R R NR R R NR NR

P4 R R R NR R R NR NR

HB NR NR NR NR NR R NR NR

20

Procedures without Overall Strong Control

Strong control is theoretically appealing. However, the procedures are generally conservative.

Strong control in certain key sub-families of hypotheses may be sufficient.

Procedure (*) (a) {H1j H2j} two-dimensional at level α ↓ ↓ (b) {S1k S2k } two-dimensional at level α Procedure (*) strongly controls the FWE rate in the sub-

families of individual doses, individual endpoints, within the families of primary endpoints and secondary endpoints. Also, it weakly controls the overall FWE rate.

21

Procedures without Overall Strong Control (2)

Procedure (*) does not strongly control the overall FWE rate.

When gp=1 and gs=1, Procedure (*) becomes

{H11} → {H21} ↓ ↓ each { } at level α {S11} → and → {S21}

(pij is the p-value for Hij and qik is the p-value for Sik)

The probability of rejecting H21S11 when H21S11 is true is

Sup Pr [(p11 ≤α)(p21 ≤α) or (p11 ≤α)(q11 ≤α) | H21S11] =Sup Pr[(p21≤α) or (q11≤α) | H21S11]≥2α-α2

22

Other Procedures without Strong ControlFor example: Step-down Closed Procedure:Step a. Apply a procedure for each dose vs. the

control separately at level for all primary and secondary endpoints.

Step b. Reject H1j (or S1k) if it should be rejected at Step a. Reject H2j (or S2k) if both H1j (or S1k) & H2j (or S2k) should be rejected at Step a.

It controls the FWE rates in the strong sense within sub-families of individual doses and endpoints, but not the family of the primary endpoints nor the family of the secondary endpoints. It controls the overall FWE rate in the weak sense.

23

Discussion

After Phase IIb dose finding study, Phase III confirmatory trials may still have two doses of the active treatment. The setting considered here should have very broad application.

As shown by numerical examples (here and in our manuscript), no procedure is uniformly more powerful than the others.

All procedures need only individual p-values -- no need to calculate the adjusted p-values based on different configurations of intersection hypotheses.

One should think hard whether the overall strong control is necessary for a particular trial.

24

Major ReferencesBenjamini Y and Hochberg T. Multiple hypotheses testing with weights. Scandinavian Journal of Statistics

1997; 24: 407-418.CPMP Points to Consider on Multiplicity Issues in Clinical Trials. 2002.D’Agostino RB. Controlling alpha in a clinical trial: the case for secondary endpoints. Statistics in Medicine

2000; 19, 763-766.Dmitrienko A, Offen WW and Westfall PH. Getekeeping strategies for clinical trials that do not require all

primary effects to be significant. Statistics in Medicine 2003; 22: 2387-2400.Hartung HP, Gonsette R, Konig N, Kwiecinski H, Guseo A, Morrissey SP, Krapf H, Zwingers T and the

Mitoxantrone in Multiple Sclerosis Study Group. Mitoxantrone in progressive multiple sclerosis: a placebo-controlled, double-blind, randomized, multicentre trial. Lancet 2002; 360, 2018-2025.

Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988; 75, 800-802.

Macus R, Peritz E and Gabriel KR. On closed testing procedures with special reference to ordered analysis of variance. Biometrika 1976; 64, 655-660.

Moye LA. Alpha calculus in clinical trials: considerations and commentary for the new millennium. Statistics in Medicine 2000; 19, 767-779.

Neer RM, Arnaud CD, Zanchetta JR, Prince R, Gaich GA, Reginster JY, Hodsman AB, Eriksen EF, Ish-Shalom S, Genant HK, Wang O and Mitlak BH. Effect of parathyroid hormone (1-34) on fractures and bone mineral density in postmenopausal women with osteoporosis. The New England Journal of Medicine 2001; 344, 19, 1434-1441.

Quan H, Luo E, Capizzi T. Xun Chen, Lynn Wei, and Bruce Binkowitz, Multiplicity adjustment for multiple endpoints in clinical trials with multiple doses of an active control. JSM Proceedings 2003.

Samuel-Cahn, E. Is the Simes improved Bonferroni procedure conservative? Biometrika 1996; 83, 928-933.

Simes RJ. An improved Bonferroni procedure for multiple tests of significance. Biometrika, 1986; 63: 655-660.

Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, Singh S, Kinnersley N, Ward P and Mills RG. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza. JAMA 2000; 283, 8, 1016-1024.

Westfall PH and Krishen A. Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference 2001; 99, 25-40.