measuring the potential impact of frailty on the apparent declining efficacy in randomized trials of...

“Measuring the Potential Impact of Frailty on the Apparent Declining Efficacy in Randomized Trials of HIV Interventions: A

Simulation Study”

Felicia P. HardnettMathematical Statistician

Quantitative Sciences and Data Management Branch

Motivation Recent advancements in HIV prevention have

given researchers hope that effective HIV interventions might soon become widely available

Additional advancements in clinical trials methodology have also occurred to measure the efficacy of these interventions more accurately

Problem The results of recent HIV intervention trials have

been somewhat disappointing and difficult to explain

The efficacy of the interventions appears to decline over time

Recently Published Trials Two recently published trials (1 vaccine trial and

1 microbicide trial) concluded that intervention effectiveness decreased over time1,2.

The investigators attributed this to: Waning vaccine efficacy (vaccine trial) Decreasing adherence (microbicide trial)

 1Abdool Karim Q, Abdool Karim SS, Frohlich JA, Grobler AC, Baxter C, Mansoor LE, et al. Effectiveness and safety of tenofovir gel, an antiretroviral microbicide, for the prevention of HIV infection in women. Science 2010; 329:1168–1174. 

2Michael N. RV 144 update: vaccination with ALVAC and AIDSVAX to prevent hiv-1 infection in thai adults. 17th conference on retroviruses & opportunistic infections, (2010). http://app2.capitalreach.com/esp1204/servlet/tc?c¼10164&cn¼retro&e¼12354&m¼1&s¼20431&&espmt¼2&mp3file ¼12354&m4bfile¼12354.

Alternative Explanation

In addition to these phenomena, the authors of a recently published opinion piece assert that frailty (due to heterogeneity in infection risk) is another possible explanation1.

This explanation is rarely cited in the literature as a possible explanation for declining efficacy.

1O’Hagan JJ, Hernan MA, Walensky RP, Lipstitch M. Apparent declining efficacy in randomized trials: examples of the Thai RV144 HIV vaccine and South African CAPRISA 004 microbicide trials. AIDS 2012, 26:123-126.

The Potential Impact of Frailty Even if the efficacy of an intervention remained

constant, frailty could give the appearance that its declining

This could cause researchers to reject an effective intervention

Purpose

To explore the potential impact of frailty on the results of randomized trials of HIV interventions

What is Frailty?

What is Frailty? Heterogeneity in infection risk within a study

population Causes change in the composition of the study

population over time Causes the measure of effect (risk ratio) to

approach 1 over time

Illustration of a hypothetical disease process within a population

• As people become infected, the population at risk decreases over time and eventually plateaus• The rate of decline depends on disease incidence• The curve plateaus at the number of persons who will never develop the disease (low/no risk people)

Pop

ulat

ion

at r

isk

Time# persons who never develop disease

Illustration of Frailty as presented in the paperP

opul

atio

n at

ris

k

Time

High risk

The opinion piece asserts:• High risk individuals will be infected early on and will be removed from the population at risk first.• This will leave lower risk individuals in the risk population resulting in lower disease incidence at later time points.

Low risk

Graphical representation of disease incidenceP

opul

atio

n at

ris

k (N

)

Time# persons who never develop disease t0 t1

Incidence=Number who become infected (n) Number intially at risk (N0)

n1

From t0 to t1

N0

t2 t3

n2

Low risk

High risk

Graphical representation of disease incidenceP

opul

atio

n at

ris

k (N

)

Time# persons who never develop disease t0 t1

• Fewer cases diagnosed at a later time point because the high risk people are gone. • Incidence, therefore decreases.

N0

t2 t3

n1

n2

Intervention ScenarioP

opul

atio

n at

ris

k

Time# persons who never develop disease

Treatment arm

Placebo

• If the intervention is effective, it will prolong the time before high-risk individuals in the treatment arm will become infected.• Incidence decline in the placebo group will be larger because those at high risk will be quickly removed from the population at risk.

RR=1

Intervention ScenarioP

opul

atio

n at

ris

k

Time# persons who never develop disease

Treatment arm

Placebo

• As a result, the time-specific rate ratio will increase from a value of less than one to a value of one or greater.• This process is termed “frailty”,“survivor bias”, “survivor cohort effect”, “crossing of hazards” or “depletion of susceptibles”.

Rate ratio= incidence (treatment arm) incidence(placebo)

RR=1

Intervention ScenarioP

opul

atio

n at

ris

k

Time# persons who never develop disease

Treatment arm

Placebo

• As frailty increases, the curve becomes more steep early on and less steep towardsthe end.• RR approaches 1 sooner.

Rate ratio= incidence (treatment arm) incidence(placebo)

RR=1

Intervention ScenarioP

opul

atio

n at

ris

k

Time# persons who never develop disease

Treatment arm

Placebo

Rate ratio= incidence (treatment arm) incidence(placebo)

RR=1

• As frailty increases, the curve becomes more steep early on and less steep towardsthe end.• RR approaches 1 sooner.

Possible Impact on Rate Measures This risk ratio comparing the incidence in placebo

and treatment group becomes increasingly attenuated as follow-up time increases.

This occurs even if risk factors were balanced between study arms at baseline and if effect of intervention is constant over time.

Is Frailty Really Important?

Based on the information presented in the paper:

With competing factors such as waning vaccine efficacy and decreasing adherence, it’s not clear how important frailty is in explaining declining efficacy in the two trials.

Current Study Objective

To quantify the potential impact of frailty under various study scenarios.

Current Study Approach We designed several study scenarios using

study-related, intervention-related and population-related parameters.

We held study-related factors constant (e.g., sample size, follow-up time, intervention effectiveness).

Current Study Approach We varied population and intervention-related

parameters (e.g. waning and frailty) We estimated the risk ratio at each time point for

each scenario and quantified the change that is attributable to frailty.

Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions

Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions

Model Parameters

25

Study Intervention Population

Fixed • Sample Size• Follow-up

Period• Number of

Risk Groups

Intervention Efficacy

Distribution of Population across Risk

Groups

Varied--

Waning Probability of Disease

The study population is divided into mutually-exclusive groups ranging from very high risk to very low risk.

Risk Groups

The proportion of the at risk population that falls into each risk group ()

Risk Distribution

Probability of Disease

The probability of an individual within a risk group becoming infected within a given study period.

Frailty

The degree of heterogeneity in the probability of disease within the study population.

Intervention Effectiveness (E)

The percent reduction in the probability of disease that is conferred by the intervention.

Waning (W)

The proportional reduction in intervention effectiveness that occurs over time.

Et= E0 * (1-W)t

Measures of Effect

32

Disease IncidenceThe proportion of the population that becomes infected within a given time frame.

𝐼𝑝𝑙𝑎𝑐𝑒𝑏𝑜=∑𝑖=1

𝑟

(π 𝑖)(¿𝑝𝑖)¿

𝐼 𝑖𝑛𝑡𝑒𝑟𝑣𝑒𝑛𝑡𝑖𝑜𝑛=(1−𝐸𝑡)∗∑𝑖=1

𝑟

(π ′ 𝑖)(¿𝑝 ′ 𝑖)¿

Risk Ratio (Outcome Measure)

The ratio of the HIV incidence in the treatment group divided by the incidence in the placebo group.

=

Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions

Model Assumptions Sufficient sample size The treatment arms have equal sample sizes. Disease risk  is balanced between both treatment

arms at the beginning of the study. Non-differential loss to follow up.

Model Assumptions The intervention is effective at reducing the

probability of disease and presents no adverse effects (i.e., increasing in the probability of infection) at any point in time.

Intervention efficacy is constant across all risk groups.

Intervention waning/non-adherence is constant across all risk groups.

Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions

Features of Study Scenarios that remain fixed Equal sample size in each treatment arm Ten-year follow-up time Five HIV risk groups ranging from very high risk

to very low risk Intervention effectiveness - 50%

Features of Study Scenarios that remain fixed

Very High High Moderate Low Very Low0

0.1

0.2

0.3

0.4

0.5

0.6

Distribution of study population across risk groups

Features of Study Scenarios that are varied Waning- the rate at which the intervention loses

its effectiveness Frailty- heterogeneity in disease risk across the

5 risk groups

Waning ParameterIn

terv

entio

n E

ffica

cy

Time

43

Frailty Parameter

0% 20% 50% 80%30%Pro

babi

lity

of I

nfec

tion

Degree of Heterogeneity

44

Frailty Parameter

0% 20% 50% 80%30%Pro

babi

lity

of I

nfec

tion

Degree of Heterogeneity

45

Frailty Parameter

0% 20% 50% 80%30%Pro

babi

lity

of I

nfec

tion

Degree of Heterogeneity

46

Placebo GroupBaseline Time 1 Time 2 Time 3

Risk GroupPopulation proportion

Probability of disease

proportion that

become infected

(time 1)

proportion that

remains uninfected (time 1)

Population Proportion

(time 1)

proportion that

become infected

(time 2)

proportion that

remains uninfected (time 2)

Population Proportion

(time 2)

proportion that

become infected

(time 3)

proportion that

remains uninfected (time 3)

Population Proportion

(time 3)

1  .05 .05   0.025 0.025 0.05 0.025 0.025 0.05 0.025 0.025 0.05

2  .2 .05   0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0.2

3  .05 .05   0.25 0.25 0.5 0.25 0.25 0.5 0.25 0.25 0.5

4  .2 .05   0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0.2

5  .05 .05   0.025 0.025 0.05 0.025 0.025 0.05 0.025 0.025 0.05

  0.5 0.5 1 0.5 0.5 1 0.5 0.5 1

Treatment Group

Risk GroupUnderlying starting pop

Probability of disease

Intervention

Effectiveness

proportion that

become infected (time 1)

proportion that

remains uninfected

(time 1)

Population Proportion

(time 1)

proportion that

become infected

(time 2)

proportion that

remains uninfected (time 2)

Population Proportion

(time 2)

proportion that

become infected

(time 3)

proportion that

remains uninfected (time 3)

Population Proportion

(time 3)

1  .05 .05 .5  0.0125 0.0375 0.05 0.0125 0.0375 0.05 0.0125 0.0375 0.05

2  .2 .05 .5  0.05 0.15 0.2 0.05 0.15 0.2 0.05 0.15 0.2

3  .05 .05 .5  0.125 0.375 0.5 0.125 0.375 0.5 0.125 0.375 0.5

4  .2 .05 .5  0.05 0.15 0.2 0.05 0.15 0.2 0.05 0.15 0.2

5  .05 .05 .5  0.0125 0.0375 0.05 0.0125 0.0375 0.05 0.0125 0.0375 0.05

      .25 .75 1 0.25 0.75 1 0.25 0.75 1

IR=.5   IR=.5   IR=.5  

No Frailty/ No Waning

E=50% E=50% E=50%

47

Placebo GroupBaseline Time 1 Time 2 Time 3

Risk GroupPopulation proportion

Probability of disease

proportion that

become infected

(time 1)

proportion that

remains uninfected (time 1)

Population Proportion

(time 1)

proportion that

become infected

(time 2)

proportion that

remains uninfected (time 2)

Population Proportion

(time 2)

proportion that

become infected

(time 3)

proportion that

remains uninfected (time 3)

Population Proportion

(time 3)

1  .05 0.3   0.015 0.035 0.04140 0.01242 0.02898 0.0342 0.0102 0.0239 0.0281

2  .2 0.21   0.042 0.158 0.18691 0.03925 0.14766 0.1741 0.0366 0.1375 0.1615

3  .05 0.147   0.0735 0.4265 0.50454 0.07417 0.43038 0.5073 0.0746 0.4327 0.5084

4  .2 0.1029   0.0205 0.1794 0.21225 0.02184 0.19041 0.2244 0.0231 0.2013 0.2365

5  .05 0.07203   0.0036 0.0464 0.05488 0.00395 0.05094 0.0600 0.0043 0.0557 0.0655

  0.1546 0.8453 1 0.15164 0.84836 1 0.1488 0.8512 1

Treatment Group

Risk GroupUnderlying starting pop

Probability of disease

Intervention

Effectiveness

proportion that

become infected (time 1)

proportion that

remains uninfected

(time 1)

Population Proportion

(time 1)

proportion that

become infected

(time 2)

proportion that

remains uninfected (time 2)

Population Proportion

(time 2)

proportion that

become infected

(time 3)

proportion that

remains uninfected (time 3)

Population Proportion

(time 3)

1  .05 0.3 .5  0.0075 0.0425 0.0461 0.0076 0.0385 0.0420 0.0075 0.0345 0.0379

2  .2 0.21 .5  0.0210 0.1790 0.1940 0.0224 0.1716 0.1874 0.0234 0.1640 0.1803

3  .05 0.147 .5  0.0368 0.4633 0.5021 0.0406 0.4615 0.5040 0.0441 0.4599 0.5056

4  .2 0.1029 .5  0.0103 0.1897 0.2056 0.0116 0.1940 0.2118 0.0130 0.1989 0.2186

5  .05 0.07203 .5  0.0018 0.0482 0.0522 0.0021 0.0502 0.0548 0.0023 0.0524 0.0576

      0.0773 0.9227 1 0.0843 0.9157 1 0.0903 0.9097 1

IR=.5   IR=.56   IR=.61  

Moderate Frailty/ 10% Waning

E=45% E=40.5%E=50%

Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions

49

50

51

52

53

54

55

56

Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions

Conclusions With the exception of the most extreme cases,

frailty (heterogeneity in disease risk) doesn’t appear to have much of an impact on outcome measures in randomized trials of HIV interventions

The impact of frailty appears substantial in scenarios when HIV infection is a virtual certainty in the highest risk group and negligible in the lowest risk group

Conclusions This study condition is unlikely to occur in most

trials where higher risk individuals are commonly recruited.

Therefore, frailty is less likely to explain a substantial portion of the declining efficacy in many HIV intervention trials

QUESTIONS?