measuring the potential impact of frailty on the apparent declining efficacy in randomized trials of...
TRANSCRIPT
“Measuring the Potential Impact of Frailty on the Apparent Declining Efficacy in Randomized Trials of HIV Interventions: A
Simulation Study”
Felicia P. HardnettMathematical Statistician
Quantitative Sciences and Data Management Branch
Motivation Recent advancements in HIV prevention have
given researchers hope that effective HIV interventions might soon become widely available
Additional advancements in clinical trials methodology have also occurred to measure the efficacy of these interventions more accurately
Problem The results of recent HIV intervention trials have
been somewhat disappointing and difficult to explain
The efficacy of the interventions appears to decline over time
Recently Published Trials Two recently published trials (1 vaccine trial and
1 microbicide trial) concluded that intervention effectiveness decreased over time1,2.
The investigators attributed this to: Waning vaccine efficacy (vaccine trial) Decreasing adherence (microbicide trial)
1Abdool Karim Q, Abdool Karim SS, Frohlich JA, Grobler AC, Baxter C, Mansoor LE, et al. Effectiveness and safety of tenofovir gel, an antiretroviral microbicide, for the prevention of HIV infection in women. Science 2010; 329:1168–1174.
2Michael N. RV 144 update: vaccination with ALVAC and AIDSVAX to prevent hiv-1 infection in thai adults. 17th conference on retroviruses & opportunistic infections, (2010). http://app2.capitalreach.com/esp1204/servlet/tc?c¼10164&cn¼retro&e¼12354&m¼1&s¼20431&&espmt¼2&mp3file ¼12354&m4bfile¼12354.
Alternative Explanation
In addition to these phenomena, the authors of a recently published opinion piece assert that frailty (due to heterogeneity in infection risk) is another possible explanation1.
This explanation is rarely cited in the literature as a possible explanation for declining efficacy.
1O’Hagan JJ, Hernan MA, Walensky RP, Lipstitch M. Apparent declining efficacy in randomized trials: examples of the Thai RV144 HIV vaccine and South African CAPRISA 004 microbicide trials. AIDS 2012, 26:123-126.
The Potential Impact of Frailty Even if the efficacy of an intervention remained
constant, frailty could give the appearance that its declining
This could cause researchers to reject an effective intervention
Purpose
To explore the potential impact of frailty on the results of randomized trials of HIV interventions
What is Frailty?
What is Frailty? Heterogeneity in infection risk within a study
population Causes change in the composition of the study
population over time Causes the measure of effect (risk ratio) to
approach 1 over time
Illustration of a hypothetical disease process within a population
• As people become infected, the population at risk decreases over time and eventually plateaus• The rate of decline depends on disease incidence• The curve plateaus at the number of persons who will never develop the disease (low/no risk people)
Pop
ulat
ion
at r
isk
Time# persons who never develop disease
Illustration of Frailty as presented in the paperP
opul
atio
n at
ris
k
Time
High risk
The opinion piece asserts:• High risk individuals will be infected early on and will be removed from the population at risk first.• This will leave lower risk individuals in the risk population resulting in lower disease incidence at later time points.
Low risk
Graphical representation of disease incidenceP
opul
atio
n at
ris
k (N
)
Time# persons who never develop disease t0 t1
Incidence=Number who become infected (n) Number intially at risk (N0)
n1
From t0 to t1
N0
t2 t3
n2
Low risk
High risk
Graphical representation of disease incidenceP
opul
atio
n at
ris
k (N
)
Time# persons who never develop disease t0 t1
• Fewer cases diagnosed at a later time point because the high risk people are gone. • Incidence, therefore decreases.
N0
t2 t3
n1
n2
Intervention ScenarioP
opul
atio
n at
ris
k
Time# persons who never develop disease
Treatment arm
Placebo
• If the intervention is effective, it will prolong the time before high-risk individuals in the treatment arm will become infected.• Incidence decline in the placebo group will be larger because those at high risk will be quickly removed from the population at risk.
RR=1
Intervention ScenarioP
opul
atio
n at
ris
k
Time# persons who never develop disease
Treatment arm
Placebo
• As a result, the time-specific rate ratio will increase from a value of less than one to a value of one or greater.• This process is termed “frailty”,“survivor bias”, “survivor cohort effect”, “crossing of hazards” or “depletion of susceptibles”.
Rate ratio= incidence (treatment arm) incidence(placebo)
RR=1
Intervention ScenarioP
opul
atio
n at
ris
k
Time# persons who never develop disease
Treatment arm
Placebo
• As frailty increases, the curve becomes more steep early on and less steep towardsthe end.• RR approaches 1 sooner.
Rate ratio= incidence (treatment arm) incidence(placebo)
RR=1
Intervention ScenarioP
opul
atio
n at
ris
k
Time# persons who never develop disease
Treatment arm
Placebo
Rate ratio= incidence (treatment arm) incidence(placebo)
RR=1
• As frailty increases, the curve becomes more steep early on and less steep towardsthe end.• RR approaches 1 sooner.
Possible Impact on Rate Measures This risk ratio comparing the incidence in placebo
and treatment group becomes increasingly attenuated as follow-up time increases.
This occurs even if risk factors were balanced between study arms at baseline and if effect of intervention is constant over time.
Is Frailty Really Important?
Based on the information presented in the paper:
With competing factors such as waning vaccine efficacy and decreasing adherence, it’s not clear how important frailty is in explaining declining efficacy in the two trials.
Current Study Objective
To quantify the potential impact of frailty under various study scenarios.
Current Study Approach We designed several study scenarios using
study-related, intervention-related and population-related parameters.
We held study-related factors constant (e.g., sample size, follow-up time, intervention effectiveness).
Current Study Approach We varied population and intervention-related
parameters (e.g. waning and frailty) We estimated the risk ratio at each time point for
each scenario and quantified the change that is attributable to frailty.
Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions
Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions
Model Parameters
25
Study Intervention Population
Fixed • Sample Size• Follow-up
Period• Number of
Risk Groups
Intervention Efficacy
Distribution of Population across Risk
Groups
Varied--
Waning Probability of Disease
The study population is divided into mutually-exclusive groups ranging from very high risk to very low risk.
Risk Groups
The proportion of the at risk population that falls into each risk group ()
Risk Distribution
Probability of Disease
The probability of an individual within a risk group becoming infected within a given study period.
Frailty
The degree of heterogeneity in the probability of disease within the study population.
Intervention Effectiveness (E)
The percent reduction in the probability of disease that is conferred by the intervention.
Waning (W)
The proportional reduction in intervention effectiveness that occurs over time.
Et= E0 * (1-W)t
Measures of Effect
32
Disease IncidenceThe proportion of the population that becomes infected within a given time frame.
𝐼𝑝𝑙𝑎𝑐𝑒𝑏𝑜=∑𝑖=1
𝑟
(π 𝑖)(¿𝑝𝑖)¿
𝐼 𝑖𝑛𝑡𝑒𝑟𝑣𝑒𝑛𝑡𝑖𝑜𝑛=(1−𝐸𝑡)∗∑𝑖=1
𝑟
(π ′ 𝑖)(¿𝑝 ′ 𝑖)¿
Risk Ratio (Outcome Measure)
The ratio of the HIV incidence in the treatment group divided by the incidence in the placebo group.
=
Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions
Model Assumptions Sufficient sample size The treatment arms have equal sample sizes. Disease risk is balanced between both treatment
arms at the beginning of the study. Non-differential loss to follow up.
Model Assumptions The intervention is effective at reducing the
probability of disease and presents no adverse effects (i.e., increasing in the probability of infection) at any point in time.
Intervention efficacy is constant across all risk groups.
Intervention waning/non-adherence is constant across all risk groups.
Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions
Features of Study Scenarios that remain fixed Equal sample size in each treatment arm Ten-year follow-up time Five HIV risk groups ranging from very high risk
to very low risk Intervention effectiveness - 50%
Features of Study Scenarios that remain fixed
Very High High Moderate Low Very Low0
0.1
0.2
0.3
0.4
0.5
0.6
Distribution of study population across risk groups
Features of Study Scenarios that are varied Waning- the rate at which the intervention loses
its effectiveness Frailty- heterogeneity in disease risk across the
5 risk groups
Waning ParameterIn
terv
entio
n E
ffica
cy
Time
43
Frailty Parameter
0% 20% 50% 80%30%Pro
babi
lity
of I
nfec
tion
Degree of Heterogeneity
44
Frailty Parameter
0% 20% 50% 80%30%Pro
babi
lity
of I
nfec
tion
Degree of Heterogeneity
45
Frailty Parameter
0% 20% 50% 80%30%Pro
babi
lity
of I
nfec
tion
Degree of Heterogeneity
46
Placebo GroupBaseline Time 1 Time 2 Time 3
Risk GroupPopulation proportion
Probability of disease
proportion that
become infected
(time 1)
proportion that
remains uninfected (time 1)
Population Proportion
(time 1)
proportion that
become infected
(time 2)
proportion that
remains uninfected (time 2)
Population Proportion
(time 2)
proportion that
become infected
(time 3)
proportion that
remains uninfected (time 3)
Population Proportion
(time 3)
1 .05 .05 0.025 0.025 0.05 0.025 0.025 0.05 0.025 0.025 0.05
2 .2 .05 0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0.2
3 .05 .05 0.25 0.25 0.5 0.25 0.25 0.5 0.25 0.25 0.5
4 .2 .05 0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0.2
5 .05 .05 0.025 0.025 0.05 0.025 0.025 0.05 0.025 0.025 0.05
0.5 0.5 1 0.5 0.5 1 0.5 0.5 1
Treatment Group
Risk GroupUnderlying starting pop
Probability of disease
Intervention
Effectiveness
proportion that
become infected (time 1)
proportion that
remains uninfected
(time 1)
Population Proportion
(time 1)
proportion that
become infected
(time 2)
proportion that
remains uninfected (time 2)
Population Proportion
(time 2)
proportion that
become infected
(time 3)
proportion that
remains uninfected (time 3)
Population Proportion
(time 3)
1 .05 .05 .5 0.0125 0.0375 0.05 0.0125 0.0375 0.05 0.0125 0.0375 0.05
2 .2 .05 .5 0.05 0.15 0.2 0.05 0.15 0.2 0.05 0.15 0.2
3 .05 .05 .5 0.125 0.375 0.5 0.125 0.375 0.5 0.125 0.375 0.5
4 .2 .05 .5 0.05 0.15 0.2 0.05 0.15 0.2 0.05 0.15 0.2
5 .05 .05 .5 0.0125 0.0375 0.05 0.0125 0.0375 0.05 0.0125 0.0375 0.05
.25 .75 1 0.25 0.75 1 0.25 0.75 1
IR=.5 IR=.5 IR=.5
No Frailty/ No Waning
E=50% E=50% E=50%
47
Placebo GroupBaseline Time 1 Time 2 Time 3
Risk GroupPopulation proportion
Probability of disease
proportion that
become infected
(time 1)
proportion that
remains uninfected (time 1)
Population Proportion
(time 1)
proportion that
become infected
(time 2)
proportion that
remains uninfected (time 2)
Population Proportion
(time 2)
proportion that
become infected
(time 3)
proportion that
remains uninfected (time 3)
Population Proportion
(time 3)
1 .05 0.3 0.015 0.035 0.04140 0.01242 0.02898 0.0342 0.0102 0.0239 0.0281
2 .2 0.21 0.042 0.158 0.18691 0.03925 0.14766 0.1741 0.0366 0.1375 0.1615
3 .05 0.147 0.0735 0.4265 0.50454 0.07417 0.43038 0.5073 0.0746 0.4327 0.5084
4 .2 0.1029 0.0205 0.1794 0.21225 0.02184 0.19041 0.2244 0.0231 0.2013 0.2365
5 .05 0.07203 0.0036 0.0464 0.05488 0.00395 0.05094 0.0600 0.0043 0.0557 0.0655
0.1546 0.8453 1 0.15164 0.84836 1 0.1488 0.8512 1
Treatment Group
Risk GroupUnderlying starting pop
Probability of disease
Intervention
Effectiveness
proportion that
become infected (time 1)
proportion that
remains uninfected
(time 1)
Population Proportion
(time 1)
proportion that
become infected
(time 2)
proportion that
remains uninfected (time 2)
Population Proportion
(time 2)
proportion that
become infected
(time 3)
proportion that
remains uninfected (time 3)
Population Proportion
(time 3)
1 .05 0.3 .5 0.0075 0.0425 0.0461 0.0076 0.0385 0.0420 0.0075 0.0345 0.0379
2 .2 0.21 .5 0.0210 0.1790 0.1940 0.0224 0.1716 0.1874 0.0234 0.1640 0.1803
3 .05 0.147 .5 0.0368 0.4633 0.5021 0.0406 0.4615 0.5040 0.0441 0.4599 0.5056
4 .2 0.1029 .5 0.0103 0.1897 0.2056 0.0116 0.1940 0.2118 0.0130 0.1989 0.2186
5 .05 0.07203 .5 0.0018 0.0482 0.0522 0.0021 0.0502 0.0548 0.0023 0.0524 0.0576
0.0773 0.9227 1 0.0843 0.9157 1 0.0903 0.9097 1
IR=.5 IR=.56 IR=.61
Moderate Frailty/ 10% Waning
E=45% E=40.5%E=50%
Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions
49
50
51
52
53
54
55
56
Modeling Description Definition of Parameters Model Assumptions Scenario Design Results Conclusions
Conclusions With the exception of the most extreme cases,
frailty (heterogeneity in disease risk) doesn’t appear to have much of an impact on outcome measures in randomized trials of HIV interventions
The impact of frailty appears substantial in scenarios when HIV infection is a virtual certainty in the highest risk group and negligible in the lowest risk group
Conclusions This study condition is unlikely to occur in most
trials where higher risk individuals are commonly recruited.
Therefore, frailty is less likely to explain a substantial portion of the declining efficacy in many HIV intervention trials
QUESTIONS?