demonstrating that an hiv vaccine lowers the risk and/or severity of hiv infection

Demonstrating that an HIV Vaccine Lowers the Risk and/or Severity of HIV

infectionD.Mehrotra1*, X.Li1, P.Gilbert2

1Merck Research Laboratories, Blue Bell, PA2Fred Hutchinson Cancer Research Center & Univ. of Washington, Seattle,

WA

Adapted from Mehrotra’s talk at ENAR, Austin, TXMarch 21, 2005

Biostat 578A Lecture 6

2

Outline• HIV vaccine POC efficacy trial• BOI vs. Simes’ method• Adjusting for selection bias• BOI vs. adjusted Simes’ method• Concluding remarks

3

HIV Vaccine POC Efficacy Trial• Design

- Randomized, double-blind, multinational trial- MRKAd5 gag/pol/nef versus placebo (1:1)- 1500 subjects at high risk of becoming HIV+- Continue until 50 events (HIV infections) accrue

• Co-Primary Endpoints- HIV infection status (infected/uninfected)- Viral load set-point (mean of log10 HIV RNA at 2 and 3 months after HIV+ diagnosis)Note: lowering of viral load set-point will presumably prevent or delay the onset of AIDS

4

POC Efficacy Trial (continued)

• Null Hypothesis: Vaccine is same as PlaceboVE = 0 and = 0

• Alternative Hypothesis: Vaccine is better than PlaceboVE > 0 and/or > 0

VE =

= true difference in mean viral load set- point among infected subjects [placebo – vaccine]

• Proof-of-concept (POC) is established if the composite null hypothesis is rejected at = 5%

PLACEBO for rate infectionHIV trueVACCINE for rate infectionHIV true1

5

POC Efficacy Trial: Data Set-Up

Vaccine Placebo

Number randomized vN pN Number HI V infected vn pn

Proportion infected v

v

Nn

p

p

Nn

Viral load set-points of infected subjects (log10 copies/ ml)

)(

)(1

vn

v

vy

y

)(

)(1

pn

p

py

y

6

Establishing POC: BOI vs. Simes’ Method

Burden- of- I llness (BOI ) (Chang, Guess, Heyse, 1994)

Diff erence in BOI per subject: p

n

i

pi

v

n

i

vi

N

y

N

yT

pv

1

)(

1

)(

Let ),|(

,|

0

0

HnnTVarHnnTET

Zpv

pvBOI

(see CGH) [unconditional test]

Reject null if 05.|Pr 0 HZZvaluep BOI

Simes’ method (Simes, 1986) 1p p-value f or infection endpoint [unconditional test] 2p p-value f or viral load endpoint [conditional test]

Reject null if 05.)),min(2 ),,min(max( 2121 ppppvaluep

(Similar results with Fisher’s combined p-value method)

7

Power (%) to Reject the Composite Null Hypothesis(assuming = 1 log10 copies/ml)

0

20

40

60

80

100

10 30 50 70 90

Pow

er (%

)VE = 0%

0

20

40

60

80

100

10 30 50 70 90

VE = 30%

0

20

40

60

80

100

10 30 50 70 90

Pow

er (%

)

VE = 60%

Number of HIV Infections

BOISimes'

*Viral load reduction is 1 log10 in all plots

8

Adjusting for Selection Bias• Simulations led us to choose Simes’ over BOI for the

POC efficacy trial. However …• Test for viral load component in Simes’ method:

- Is restricted to subjects that are selected based on a post-randomization outcome (HIV infection) can suffer from selection bias.- Assesses mixture of (i) causal effect of vaccine and (ii) effect of variables correlated with VL that are unevenly distributed among the infected subgroups.References Rubin (1978), Rosenbaum (1984), Robins and Greenland (1992), Frangakis and Rubin (2002)

9

Adjusting for Selection Bias (continued)

• Proposed approach- Adjust the viral load test for plausible levels of

selection bias such that rejection of the null hypothesis becomes harder.

- If the adjusted test is significant, then we have robust evidence of a causal vaccine effect. Hudgens, Hoering, Self (Statistics in Medicine, 2003)

Gilbert, Bosch, Hudgens (Biometrics, 2003) [GBH]Mehrotra, Li, Gilbert (Biometrics, 2006)

• Adjustment is derived via the principal stratification framework of causal inference (Frangakis and Rubin, Biometrics, 2002)

10

Adjusting for Selection Bias (continued)

• Subjects infected under placebo {Si(p)=1} partition

into the protected and always-infected principal strata

To assess a causal vaccine effect: we need to compare Yi

(v) (= Yi(v, alw.inf.)) with Yi

(p, alw.inf.), but the placebo VLs are a mixture of Yi

(p, prot.) and Yi(p, alw. inf.)

How to identify the distribution of Yi(p, alw.inf.)?

Principal Stratum Potential infection outcome under Z

Potential VL outcome under Z given Si

(z) = 1Protected Si

(v) = 0, Si(p) = 1 undefined Yi

(p,prot.)

Always infected Si(v) = 1, Si

(p) = 1 Yi(v) Yi

(p,alw.inf.)

Z = assigned treatment

11

Adjusting for Selection Bias (continued)

• fp(y) = (VE)fp(prot)(y) + (1-VE)fp

(alw.inf)(y)

fp(alw.inf)(y) = [w(y)/(1-VE)]fp(y)

where w(y) = Pr{Si(v)=1|Yi

(p)=y, Si(p)=1} is the

unknownprobability that a placebo infectee with VL set-

pointy would have been infected if given vaccine.

• VE and fp(y) can be estimated from the data, but not w(y).Solution: assume a “known” model for w(y).

12

Adjusting for Selection Bias (continued)

• GBH (2003) assume a logistic model for w(y):wi, = w(yi|,) = exp( + yi)/{1+exp( + yi)},

inp

where is a fixed (pre-set) parameter:(i) = 0 wi, = 1 – VE for all i(ii) < 0 for a 1-unit decrease in Yi

(p), the odds of being in the always infected stratum increase multiplicatively by exp(-)

(iii) is a constant satisfying Fp(|) = 1

• For a given , fp(alw.inf) can now be estimated.

13

VL Distributions for the Protected and Always Infected Principal Strata Implied by the Logistic

Model for wi(y) = 0

(e- = 1)= -

(e- = ) = -2

(e- = 7.4)

= 0: vaccine does not selectively protect subjects same distribution for Yi

(p, prot.) and Yi(p, prot.)

< 0: vaccine selectively protects subjects with higher VLs selection bias leads to biased estimation of the causal effect that makes the vaccine look poorer than it is.

log10 VL

prob

abili

ty d

ensi

ty

2 3 4 5 6 7

0.0

0.1

0.2

0.3

0.4

VE

1-VE

log10 VL

prob

abili

ty d

ensi

ty

2 3 4 5 6 7

0.0

0.1

0.2

0.3

0.4

VE1-VE

log10 VL

prob

abili

ty d

ensi

ty

2 3 4 5 6 7

0.0

0.1

0.2

0.3

0.4

VE1-VE

14

Adjusting for Selection Bias (continued)• Adjust the viral load test in Simes’ method:

1) Fix the selection bias parameter 0.2) Adjust (reduce) all the VLs of placebo infectees:

is non-parametric m.l.e. of

3) Let = Wilcoxon rank sum statistic comparing

with

T

p

pp

n

i i

n

ipii

p

n

ipip

ipi

yw

yywny

yy1

1)(

1)(

)(*)(,

,ˆ|

,ˆ|

vniviy }{ )(

pnipiy }{ *)(

15

Adjusting for Selection Bias (continued)

• When VE = 0, T is the Wilcoxon rank sum statistic used for the unadjusted VL test.

• The distribution of T is intractable, so the p-value for the adjusted VL test ( = p2, ) is obtained using a non-parametric bootstrap.

• Adjusted Simes’ method: for the specified , reject the composite null hypothesis if

• Robust evidence of a causal vaccine effect on either the infection or VL endpoint: reject the composite null hypothesis using the adjusted Simes’ method for all plausible values of .

2/),min( ),max( ,21,21 pporpp

16

BOI vs. Simes’ Method: Power (%)Assuming = 1 log10 copies/ml

0

20

40

60

80

100

10 30 50 70 90

Pow

er (%

)

VE = 0%

0

20

40

60

80

100

10 30 50 70 90

VE = 30%

0

20

40

60

80

100

10 30 50 70 90

Pow

er (%

)

VE = 60%

Number of HIV Infections

BOIUnadjusted Simes' (Beta=0)Adjusted Simes' (Beta=-1)Adjusted Simes' (Beta=-2)

*Viral load reduction is 1 log10 in all plots

0

20

40

60

80

100

10 30 50 70 90

Pow

er (%

)

VE = 0%

0

20

40

60

80

100

10 30 50 70 90

VE = 30%

0

20

40

60

80

100

10 30 50 70 90

Pow

er (%

)

VE = 60%

Number of HIV Infections

BOIUnadjusted Simes' (Beta=0)Adjusted Simes' (Beta=-1)Adjusted Simes' (Beta=-2)Adjusted Simes' (Beta=-Inf)

*Viral load reduction is 1 log10 in all plots

17

Concluding Remarks• The selection bias-adjusted Simes’ method is more

powerful than the BOI method, unless VE is “large” (unlikely for a CMI-based HIV vaccine).

• 50 events will provide at least 80% power to establish POC provided:VE 60% or 0.75 c/ml: unadjusted Simes’ methodVE 60% or 1.0 c/ml: adjusted Simes’ method.

• An -spending interim analysis after 30 events is proposed (details omitted here). Estimated time between 30 and 50 events is 9-15 months.

18

REFERENCES1. Chang MN, Guess HA, Heyse JF (1994). Reduction in the burden of illness: a new

efficacy measure for prevention trials. Statistics in Medicine, 13, 1807-1814.2. Fisher RA (1932). Statistical Methods for Research Workers. Oliver and Boyd,

Edinburgh and London.3. Frangakis CE, Rubin DB (2002). Principal Stratification in Causal Inference.

Biometrics, 58, 21-29.4. Gilbert PB, Bosch RJ, Hudgens MG (2003). Sensitivity analysis for the assessment

of causal vaccine effects on viral load in HIV vaccine clinical trials. Biometrics, 59, 531-541.

5. Hudgens MG, Hoering A, Self SG (2003). On the analysis of viral load endpoints in HIV vaccine trials. Statistics in Medicine, 22, 2281-2298.

6. Mehrotra DV, Li X, Gilbert PB. Dual-endpoint evaluation of vaccine efficacy: Application to a proof-of-concept clinical trial of a cell mediated immunity-based HIV vaccine. Biometrics, in press.

7. Robins JM, Greenland S (1992). Identifiability and exchangeability of direct and indirect effects. Epidemiology, 3, 143-155.

8. Rosenbaum PR (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. The Journal of the Royal Statistical Society, Series A, 147, 656-666.

9. Rubin DB (1978). Bayesian inference for causal effects: the role of randomization. The Annals of Statistics, 6, 34-58.

19

APPENDIX• Arguments against a selection-bias adjustment:

- POC (not phase III) trial: not essential to precisely characterize the vaccine effect.- VE (and hence selection bias) anticipated to be small.- If vaccine prevents infection only for less virulent strains, then selection bias is more likely to make placebo look better than vaccine when comparing VLs, so the unadjusted test is already conservative from a causal inference perspective!

• Arguments for a selection-bias adjustment:- Will we really proceed to phase III without robust evidence of a causal vaccine effect?- To satisfy statisticians who are wary of any non-randomized comparison.

20

Hypothetical Example Vaccine Group Placebo Group

I nf ected/ Enrolled 22/ 750 28/ 750 2.26 3.98 2.79 4.45 2.55 4.02 3.26 4.57 2.57 4.17 3.32 4.58 2.68 4.17 3.51 4.66 2.82 4.44 3.72 4.92 3.13 4.69 4.02 4.99 3.17 4.83 4.08 5.18 3.41 4.10 5.19 3.45 4.10 5.20 3.64 4.14 5.23 3.65 4.20 5.52 3.74 4.21 5.60 3.82 4.24 5.62 3.92 4.26

VL set-point (log10 copies/ ml)

3.95 4.40 %21750/28/750/221 obsVE , p1 = 0.240 (binomial test)

obs 4.43 – 3.59 = 0.84 log10 c/ ml, p2 = 0.0001 (rank-sum test)

Simes’ p-value = 0.0002*, BOI p-value = 0.062

21

Hypothetical Example (continued) Vaccine Group Placebo Group

I nfected/ Enrolled 22/ 750 28/ 750 X X X X 2.26 0 3.95 0 2.79 0 4.40 1 2.55 0 3.98 0 3.26 0 4.45 1 2.57 0 4.02 0 3.32 0 4.57 0 2.68 0 4.17 1 3.51 0 4.58 1 2.82 0 4.17 1 3.72 0 4.66 0 3.13 0 4.44 0 4.02 0 4.92 1 3.17 0 4.69 1 4.08 0 4.99 1 3.41 0 4.83 1 4.10 0 5.18 0 3.45 0 4.10 0 5.19 1 3.64 0 4.14 0 5.20 1 3.65 0 4.20 0 5.23 1 3.74 0 4.21 0 5.52 1 3.82 0 4.24 0 5.60 1

VL set-point (log10 c/ ml)

3.92 0 4.26 0 5.62 1

X = unobserved covariate (e.g., a genetic trait) A higher proportion of placebo infectees have X=1, and

subjects with X = 1 tend to have higher viral loads. Did vaccine cause lower VLs or is the observed vaccine

eff ect an artifact of the imbalance in the X distribution?

22

Hypothetical Example RevisitedAssigning weights to the VLs in the Placebo Group

weight (wi,) weight (wi,) log10 VL 0 2

log10 VL 0 2

2.79 0.79 0.99 1 4.40 0.79 0.86 1 3.26 0.79 0.98 1 4.45 0.79 0.85 1 3.32 0.79 0.98 1 4.57 0.79 0.82 1 3.51 0.79 0.97 1 4.58 0.79 0.82 1 3.72 0.79 0.96 1 4.66 0.79 0.79 1 4.02 0.79 0.93 1 4.92 0.79 0.69 1 4.08 0.79 0.92 1 4.99 0.79 0.66 1 4.10 0.79 0.92 1 5.18 0.79 0.57 1 4.10 0.79 0.92 1 5.19 0.79 0.57 0 4.14 0.79 0.92 1 5.20 0.79 0.56 0 4.20 0.79 0.91 1 5.23 0.79 0.55 0 4.21 0.79 0.90 1 5.52 0.79 0.41 0 4.24 0.79 0.90 1 5.60 0.79 0.37 0 4.26 0.79 0.89 1 5.62 0.79 0.36 0

23

Hypothetical Example Revisited

OR=exp

1-ta

iled

p-va

lue

1 2 3 4 5 6 7 8 9 10

0

0.025

0.05

0.075

0.1

= Inf

Robust evidence of a causal vaccine effect on VL