bios 6648: design & conduct of clinical research

Date: 30 Sep 2013

2. Scientific/statisticaldesign2.3-2.4: Hypotheses andinformation

Bios 6648- pg 1

Bios 6648: Design & conduct of clinical researchSection 2 - Formulating the scientific and statistical design2.3-2.4 Structuring study hypotheses and selecting studyinformation

I Sections 2.3-2.4 will be covered through examples:

Trial Type of Nature ofName Outcome Hypotheses

(a) CHEST trial continuous superiority(b) ICU stay continuous superiority(c) Sepsis trial binary superiority(d) Daptomycin binary non-inferiority(e) Rocket-AF time-to-event non-inferiority(f) CCF trial continuous 2-sided equivalence(g) PLCO time-to-event superiority(h) Iloprost phase III time-to-event superiority

I Section 2.5 will cover other important designs:I Time-to-event outcomes (revisited)I Binary outcomes (revisited)I Skewed outcomes (log transformation)I Change-from-baseline outcomes

Date: 30 Sep 2013


Bios 6648- pg 2

2.3-2.4 Structuring study hypothesesand selecting study information

(c) Sepsis trial

I Summary of designI Structure of design hypothesesI How much information will we have in this trial?I What is the potential inference before the trial starts?I How much information is needed to discriminate between the

design hypotheses?I How much information is needed to give 90% power?

Date: 30 Sep 2013


Bios 6648- pg 3

(c) Sepsis trial

Summary of design

I Reference:http://rctdesign.org/Software/SeqTrialUsersGuide.pdfChapter 12.

I Disease: Gram negative sepsis (systemic bacterial infection).I Study population: Patients admitted to ICU with culture-proven gram

negative sepsis.I Treatments: Antibody (injection) to endotoxin produced by the bacteria;

versus placebo injection.

I Outcome: 14-day mortality (death within 14-days after randomization).I Probability model: Binomial (proportions)I Functional: θ1 and θ0 denote the true mean risk of death

before 14 days with antibody and placebo, respectivelyI Contrast: θ = θ1 − θ0 (risk difference)

Date: 30 Sep 2013


Bios 6648- pg 4

(c) Sepsis trial

Design hypotheses

I Hypotheses: Superiority test of a less-than hypothesis;

H∅ : θ ≥ θ∅ Null hypo: inferiorityH+ : θ ≤ θ+ Alt hypo: clinically important benefits

In this trial investigators chose θ∅ = 0 and θ+ = −0.07(a 7% reduction in mortality).

I In a binomial distribution, then variance is determined bythe mean.

I From prior experience 14-day mortality is about 30%(i.e., θ0 ≈ 0.30).

I Investigators hope mortality is reduced by 7%(i.e., θ1 ≈ 0.23).

Date: 30 Sep 2013


Bios 6648- pg 5

(c) Sepsis trial

Evaluate the information

I How much information will we have in this trial?I What is the potential inference before the trial starts?I How much information is needed to discriminate between

the design hypotheses?I How much information is needed to give 90% power?

Date: 30 Sep 2013


Bios 6648- pg 6

(c) Sepsis trial

How much information will we have in this trial?

I Likely variance: Mean-variance relationship in the binomialprobability model:

var(θ̂) = var(θ̂1 − θ̂0) =θ1(1 − θ1)

rN+θ0(1 − θ0)

N

I Sample size: After several iterations the sample size wasselected to be 1700 (850 per group; r = 1 and N = 850).

I Information: Trial information can be evaluated undervarious hypotheses:

I Information (and CI width) under the design alternative:

I(θ̂) =NV

=850

0.23× 0.77 + 0.30× 0.70= 2195.8

CI width: 3.92

√VN

=3.92√2195.8

= 0.08365

So, the 95% CI would discriminate between:H∅ : θ ≥ 0 and H+ : θ ≤ −0.08365.

Date: 30 Sep 2013


Bios 6648- pg 7

(c) Sepsis trial

How much information will we have in this trial?

Information: Trial information under various hypotheses:

Hypo Variance = V Info = N/V Std Err CI widthAlt 0.23× 0.77 + 0.3× 0.7 2195.8 0.0213 0.0837Null 0.3× 0.7 + 0.3× 0.7 2023.8 0.02222 0.0871Intr 0.265× 0.735 + 0.3× 0.7 2099.9 0.02182 0.0855

Date: 30 Sep 2013


Bios 6648- pg 8

(c) Sepsis trial

How much information is needed to discriminate between thedesign hypotheses?

I Consider the variance under the alternative(V = 0.23 × 0.77 + 0.3 × 0.7 = 0.387):

I We seek N such that 3.92√

VN = −0.07:

N =

[3.92−0.07

]2

× 0.387 = 1213.9

A total of 2428 patients (1214 per group).I Using other variances:

I Variance under null requires 2636 patientsI Variance under intermediate hypothesis requires 2540

patients.

Date: 30 Sep 2013


Bios 6648- pg 9

(c) Sepsis trial

How much information is needed to for 90% power?

I Use the sample size formula with Zβ = Z0.9 = 1.28:

N =

[1.96 + 1.28

−0.07

]2

V

I I commonly discuss a table:Sample size required for selected levels of power

Method for variance calculationPower Null Alternative Intermediate0.800 1241 1346 12970.850 1419 1540 14840.900 1661 1802 17370.950 2054 2228 21470.975 2428 2635 2539

For example a sample size of 1802 (901 per group) gives 90%power for an absolute reduction in 14-day mortality of 7% (from30% to 23%).

Date: 30 Sep 2013


Bios 6648- pg 10

(c) Sepsis trial

How much information is needed to for adequate power?

I I find that the following table leads to better discussion(and better understanding) among investigators:

I Sample size required for selected levels of power

Placebo Risk Total sample sizeRisk Reduction 1600 2000 24000.25 -0.06 0.828 0.901 0.9450.25 -0.07 0.912 0.959 0.9820.25 -0.08 0.961 0.986 0.9950.30 -0.06 0.797 0.877 0.9280.30 -0.07 0.889 0.945 0.9740.30 -0.08 0.948 0.980 0.9920.35 -0.06 0.772 0.857 0.9120.35 -0.07 0.870 0.932 0.9660.35 -0.08 0.935 0.973 0.989

For example a trial with 2400 patients (1200 per group) gives 91.2% power foran absolute reduction of 6% (from 35% to 29%).

Date: 30 Sep 2013


Bios 6648- pg 11

(c) Sepsis trial

How much information is needed to for adequate power?

I Notice that the above table required a different version ofthe sample size equation:

zβ =(θ+ − θ∅)√

VN

− zα

then power (β) is determined from zβ .

Date: 30 Sep 2013


Bios 6648- pg 12


(d) Daptomycin Trial

I Reference: N Engl J Med 2006;355:653-65I Disease: Bacteremia and endocarditis due to Staph

aureusI Study population: ≥ 18yo; culture proven SA plus other

criteria (see paper).I Treatments: Randomized to “open-label" treatment with

Daptomycin or standard care.I Outcome: 42-day “success" probability

I Probability model: Binomial proportionsI Functional: θ0 and θ1 denote the true underlying 42-day

success probability with standard care and daptomycin,respectively.

I Contrast: θ = θ1 − θ0 (risk difference)

Date: 30 Sep 2013


Bios 6648- pg 13


Design hypotheses

I Hypotheses: Non-inferiority;I Reported criteria:

The noninferiority test was based on the lowerbound of the confidence intervals being within theprespecified noninferiority margin of 20 percentand the upper bounds containing 0 percent.

I Possible hypotheses:

H∅ : θ ≥ θ∅ Null hypo: equality and benefitH+ : θ ≤ θ+ Important harm – non-inferiority margin

As reported θ∅ = 0,and the non-inferiority margin is θ+ = −0.2.

Date: 30 Sep 2013


Bios 6648- pg 14


How much information is in this trial?

I Observed results:I Point estimate: θ̂ = θ̂1 − θ̂0 = 53/120− 48/115 = 0.024.I Interval estimate:

se(θ̂) =

√θ̂0(1− θ̂0)

N0+θ̂1(1− θ̂1)

N1

=

√0.42× 0.58

115+

0.44× 0.56120

= 0.06457

so the 95% CI is 0.024± 1.96se = (−0.1023, 0.151).I Information:

I I(θ̂) = 1/se2 = 239.83I Note: CI width ≈ 0.253, so the design could discriminate:

H+ : θ ≤ −0.253 from H∅ : θ ≥ 0.

Date: 30 Sep 2013


Bios 6648- pg 15



I Using the design criteria (see earlier slide):I Hypotheses: H∅ : θ ≥ 0 and H+ : θ ≤ −0.2I Variance: V = θ0(1− θ0)× 2 = 0.65× 0.35× 2 = 0.455

I Sample size for discrimination:

N =

[3.920.2

]2

V = 174.97

that is, 175 per group (350 total).

Date: 30 Sep 2013


Bios 6648- pg 16


How did the investigators calculate 80% power?

I Using V = 0.455 in the sample size formula for β = 0.8:

N =

[1.96 + 0.84

0.2

]2

V = 89.18

or 90 per group.I Notes:

I With 90 per group there is a risk of an inconclusive result(i.e., CI includes both θ < −0.2 and θ > 0.0).

I A study with 80% power is about half the size of a studywith 97.5% power (a study that discriminates between thedesign hypotheses).

Date: 30 Sep 2013


Bios 6648- pg 17


(e) Rocket-AF

I Reference: N Engl J Med 2011;365:883-91.I Disease: Atrial fibrillation not due to heart valve

abnormalityI Study population: Patients with nonvalvular AF who were

at high risk of stroke (see paper).I Treatments: Randomized to treatment with 20mg

rivaroxaban with warfarin placebo or warfarin withrivaroxaban placebo (“double blind, double-dummy").

I Outcome: Time to occurrence of an event (stoke orsystemic embolism)

I Probability model: Proportional hazardsI Functional: θ0 and θ1 denote the true underlying hazard with

warfarin and rivaroxaban, respectively.I Contrast: θ = θ1

θ0(hazard ratio)

Date: 30 Sep 2013


Bios 6648- pg 18

(e) Rocket-AF

Design hypotheses

I Hypotheses: Non-inferiority;I Reported criteria:

For the primary analysis, we determined that aminimum of 363 events would provide a power of95% to calculate a noninferiority margin of 1.46with a one-sided alpha level of 0.025.

I Hypotheses:

H∅ : θ ≤ θ∅ Null hypo: equality and benefitH+ : θ ≥ θ+ Important harm – non-inferiority margin

As reported θ∅ = 1.0,and the non-inferiority margin is θ+ = 1.46.

Date: 30 Sep 2013


Bios 6648- pg 19

(e) Rocket-AF

How much information is in this trial?

I Observed results:I Point estimate: θ̂ = θ̂1

θ0= 0.79.

I Interval estimate: 0.66 to 0.96I How to calculate the interval estimate (I will discuss again in

section 2.5)I Variance of log(HR): var( ̂(log(θ))) = V

D , where V = 4 and Dis the total number of events (in both groups)

I Standard error: se =√

VD =

√4

429 = 0.0966

95% CI = log(0.79)± 1.96× 0.0966

= (−0.425,−0.0465)

The above are the CI for the log(HR); the CI for the HR is(e−0.425, e−0.0465) = (0.65, 0.95) – about the same asreported.

I Information:I I(θ̂) = 1/se2 = 1/0.09662 = 107.25I Note: CI width ≈ 0.3785, so the design could discriminate:

H∅ : θ ≤ 1.0 andH+ : θ ≥ e0.3785 = 1.46 .

Date: 30 Sep 2013


Bios 6648- pg 20

(e) Rocket-AF


I Using the design criteria (see earlier slide):I Hypotheses: H∅ : θ ≤ 1 and H+ : θ ≥ 1.46I Variance for log hazard ratio: V = 4


D =

[3.92

log(1.46)

]2

× 4 = 429.19

that is, 430 events total (across both groups).

Date: 30 Sep 2013


Bios 6648- pg 21

(e) Rocket-AF

How did the investigators calculate 95% power?

I Using zβ = 1.645 in the sample size formula for β = 0.95:

N =

[1.96 + 1.645

log(1.46)

]2

× 4 = 362.98

or 363 total events total.

Date: 30 Sep 2013


Bios 6648- pg 22


(f) China Complementary Feeding Trial

I Question: Can alternative feeding (meat instead of rice)improve growth in rural Chinese infants

I Study population: Infants (age 6 months) in villages inrural China

I Treatments: Infants in villages randomized tocomplementary feeding with traditional rice cereal or withground pork.Some notes:

I This is an example of a ‘cluster-randomized’ trial (we willdiscuss that aspect later).

I The trial could not be blinded (parents, doctors, andinvestigators all knew the group).

I Outcome: Linear growth in 12-months (from age 6 mos to18 mod).

I Probability model: Continuous (normal)I Functional: θM and θC denote the true underlying mean

change in length with meat and rice cereal, respectively.I Contrast: θ = θM − θC (difference of means).

Date: 30 Sep 2013


Bios 6648- pg 23


Design hypotheses

I Hypotheses: 2-sided equivalence; three hypotheses

Meat preferred: H+ : θ ≥ θ+

Equality: H∅ : θ = 0Cereal preferred: H− : θ ≤ θ−

I Selected hypotheses:* θ+ = 0.055 cm/mo (about a 4% increase in linear growth

rate with meat when compared to cereal)* θ− = −0.055 cm/mo (about a 4% decrease in linear growth

rate with meat when compared to cereal)

Date: 30 Sep 2013


Bios 6648- pg 24


How much information will there be in this trial?

I Pre-trial design:I Likely variance: Based on earlier studies the variance of the

change in length from 6 months to 18 months is about 0.30cm/mo.

I Planned sample size: 1200 infants (600 receiving ricecereal, 600 receiving ground pork).

I Information:NV

=600

2× 0.302 = 3333.3

I Anticipated SE: Anticipated standard error of θ̂:

SE(θ̂) =1√info

= 0.0173

Date: 30 Sep 2013


Bios 6648- pg 25


What is the potential inference upon trial completion?

I Recall that there are 3 decisions in a 2-sided test:I Recommend meat over cereal:

I critical value: cv = θ̂ ≥ 1.96SE = 0.034.

I 95% CI at cv: (0.00, 3.92SE) = (0.00, 0.068)(i.e., reject θ ≤ 0)

I Recommend cereal over meat:I critical value: cv = θ̂ ≤ −1.96SE = −0.034.

I 95% CI at cv: (−3.92SE , 0.00) = (−0.068, 0.00)(i.e., reject θ ≥ 0)

I Use meat or cereal:I critical value: decide equivalence when:−0.034 < θ̂ < 0.034.

I That is, reject both θ ≥ 0.068 and θ ≤ −0.068.

Date: 30 Sep 2013


Bios 6648- pg 26



I Design hypotheses (see earlier slide):

Meat preferred: H+ : θ ≥ 0.055Equality: H∅ : θ = 0

Cereal preferred: H− : θ ≤ −0.055


N =

[3.92

0.0552

]2

× 2 × 0.303 = 914.36

that is, 915 infants per group (1830 total).

Date: 30 Sep 2013


Bios 6648- pg 27


How much information is needed for 90% power?

I Using V = 2 × 0.302 in the sample size formula forβ = 0.9:

N =

[1.96 + 1.28

0.0552

]2

× 2 × 0.303 = 624.65

or 2 × 625 = 1250 infants total.

Date: 30 Sep 2013


Bios 6648- pg 28


(g) Prostate cancer screening (PLCO)

I Background: PSA screening for prostate cancer iscommon, but the tradeoff between risks and benefits is notknown.

I Public Health Question: Should screening for prostatecancer be part of routine practice?

I Study Population: Men age 55-74I Outcomes:

I Death from prostate cancer (primary outcome)I Prostate cancer diagnosis (secondary)

Date: 30 Sep 2013


Bios 6648- pg 29



I Outcome Parameterization:I Outcome: Death from prostate cancerI Probability model: Incidence rate (number of deaths) follows

Poisson distribution:

Yk ∼ P(λk Nk )

where:Yk is the total number of deaths in group kλk is the incidence (risk) of death per person-year in group kNk is the number of person-years in group k

I Functional: Mean rate: θ0 = λ0, θ1 = λ1I Contrast: Ratio: θ = θ1/θ0.

Date: 30 Sep 2013


Bios 6648- pg 30


How much information will there be in this trial?(See paper table 2)

I Sample Size:Number of men ≈ 38, 350 in each groupNumber of person-years: Nk ≈ 250, 000 each group.

I Variance:I Death rate λk ≈ 2 per 10,000 person-years; thus:λk Nk = 2

10000 250000 ≈ 50 deaths in each group.I So from the formula sheet:

SE

[̂

log(θ1

θ0

)]=

√1

θ0N0+

1θ1N1

=

√1

50+

150

= 0.2

Date: 30 Sep 2013


Bios 6648- pg 31


Potential inference

I Critical value: cv = e0−1.96×0.2 = 0.676,i.e., critical value ≈ 1

3 reduction in risk.

I Interval estimate at cv:

(e−0.784,e0) = (0.457,1.0)

I Possible conclusions:I If the trial is not statistically significant (θ̂ ≥ 0.676),

then rule out θ ≤ 0.457.

I If is statistically significant (θ̂ < 0.676),then reject θ ≥ 1.0.

Date: 30 Sep 2013


Bios 6648- pg 32


Selecting the information (given design hypotheses)

I Suppose we had the following design hypotheses:I θk = cause-specific death rate with treatment kI θ = θ1/θ0 (θ < 1.0 indicates screening is better).

H0 : θ ≥ 1.0 harm

H+ : θ ≤ 0.5 important benefit

I How much information do we need to discriminatebetween these hypotheses?

Date: 30 Sep 2013


Bios 6648- pg 33


Selecting the information to discriminate hypotheses

I Information needed to discriminate the above hypotheses.I Variance:

var(log(θ̂1)− log(θ̂0)) =

√1

θ0N0+

1θ1N1

=

√1D

+1D

where D is the number of deaths in each arm.I Sample size (number of deaths):

D =

(2× 1.96θ+ − θ∅

)2

× 2

=

(2× 1.96log(0.5)

)2

× 2

= 64

i.e., 128 deaths (total) are required to discriminate theabove hypotheses.

Date: 30 Sep 2013


Bios 6648- pg 34



I Information needed to discriminate the above hypotheses.I How many people (person-years) will it take to get 64

deaths?I Assuming 2 deaths per 10,000 person-years:

642

10,000

= 320, 000 person-years per group

I Notice that this calculation is consistent with results shownin paper table 2.

Date: 30 Sep 2013


Bios 6648- pg 35



I Potential conclusions with 128 deaths:I Standard error:

var(log(θ̂1)− log(θ̂0)) =

√1

64+

164

= 0.1768

I Critical value: cv = e−1.96×0.1768 = 0.7072

I 95% CI at critical value: (0.500, 1.0).I Potential conclusions:

I Statistical significance:

θ̂ < 0.7072⇒ reject θ ≥ 1.0.

I No significant benefit:

θ̂ ≥ 0.7072⇒ reject θ ≤ 0.50.

Date: 30 Sep 2013


Bios 6648- pg 36


Selecting the information for 90% power

I As shown above: SE [log(θ̂)] ≈√

1D + 1

D .

I Using the usual sample size equation:

D =

(1.96 + 1.28θ+ − θ∅

)2

× 2

=

(1.96 + 1.28

log(0.5)

)2

× 2

= 43.699

That is, a total of 2 × 44 = 88 deaths.I It takes 440,000 person-years to produce 88 deaths

(at 2 deaths per 10,000 person-years).

Date: 30 Sep 2013


Bios 6648- pg 37


Notes on PLCO example

I The above calculations are approximate because:I Sample size was calculate under the null hypothesis (equal

number of deaths per group).

I Sample size can also be calculated under the alternativehypothesis (or an intermediate hypothesis).

I The total number of events is controlled by both:

I Number of subjects.I Length of follow-up time.

Date: 30 Sep 2013


Bios 6648- pg 38

(h) Iloprost phase III

Trial setting and design overview

I Background: The phase II iloprost trial showed significantimprovement in lung histology. Ongoing discussions onthe design of a phase III trial.

I Clinical question: Does iloprost prevent lung cancer?I Design elements: Double-blind randomized

placebo-controlled trial:I Study population: Resected stage I lung cancer patients.I Primary outcome: time to second primary tumor.

I Outcome parameterization:I Probability model: Ratio of hazards.I Functional: θ0 and θ1 denote the hazard of a second

primary tumor with placebo and iloprost, respectively.I Contrast: Hazard ratio: θ = θ1/θ0 .

Date: 30 Sep 2013


Bios 6648- pg 39


How much information will the trial have?

I Information in a hazard ratio is proportional to the numberof events:Information for log(θ̂) is ( D

V ) whereI Variance: V = 4I Sample size (D) is the total number of events.I Total number of patients is determined by risk of events

(see PLCO example).I Standard error:

SE[log(θ̂)

]=

√4D

Date: 30 Sep 2013


Bios 6648- pg 40


Selecting the information (design hypotheses)

I Design hypothesesI One-sided superiority test of a less-than alternative:

Inferiority: H∅ : θ ≥ 1.0

Important superiority: H+ : θ ≤ θ+

I Investigators considered the number of events required asa function of θ+ and power using the standard sample sizeequation:

D =

(zα + zβ

log(θ+)− log(1)

)2

× 4

Date: 30 Sep 2013


Bios 6648- pg 41


Selecting the information (number of events)

I Number of events (total) and power:

DesignHazard PowerRatio 0.80 0.90 0.95 0.9750.65 170 227 281 3320.60 121 162 200 2360.55 88 118 146 1720.50 66 88 109 128

I Decided to consider 160 events.

Date: 30 Sep 2013


Bios 6648- pg 42


Selecting the information (number of patients)

Number of patients required:

∗ Suppose: L years for full enrollment; 2 years min follow-up.

(Total duration = 2 + L; avg follow-up = 2 + L/2.)

∗ Expected number of events per 1000 patients enrolled:

Total Avg Annual incidence of recurrenceDuration f/u 0.005 0.01 0.02 0.03 0.04 0.05

4 3.0 14.9 29.7 58.8 87.3 115.3 142.65 3.5 17.4 34.6 68.3 101.1 133.1 164.36 4.0 19.9 39.4 77.6 114.7 150.7 185.57 4.5 22.3 44.2 86.9 128.1 167.8 206.1

∗ Example:Expect ≈ 160 events in a 6-year study of 1400 patients withaverage incidence of 3% (1.4× 114.7 = 160.6).

Date: 30 Sep 2013


Bios 6648- pg 43


Selecting the information (inference at the boundary)

I What is the inference at the boundary if there are 160events?

I Critical value: cv = e−1.96√

4160 = 0.734

I 95% CI at critical value: (e−3.92√

4160 , e0) = (0.538, 1.00)

I Potential inference:

* If θ̂ ≤ 0.734, then:Conclude significant benefit for iloprost.Rule out θ > 1.00.

* If θ̂ ≥ 0.734, then:Conclude no significant difference.Rule out θ < 0.538.

Date: 30 Sep 2013


Bios 6648- pg 44

Summary of key concepts (sections 2.3-2.4)

1. Define the outcome and form for statistical information:

(a) Specify outcome, functional (θ0, θ1), contrast (θ)(b) Know (or look up):

I Information: Info(θ̂) = NV

I Standard error: SE(θ̂) = 1√info(θ̂)

2. General structure of hypotheses( 1-sided: greater, less,non-inferiority; 2-sided).

3. Given the information (N and V ) calculate:

(a) Inference at the boundary (critical value, 95% CI at criticalvalue)

(b) Hypotheses discriminated(c) Value of θ+ with power β(d) Amount of power (β) for a given value of θ+

4. Given design hypotheses (θ∅ and θ+) calculate:

(a) Sample size (information) needed to discriminate thesehypotheses.

(b) Sample size (information) that gives power β.

bios 6648: design & conduct of clinical research

Documents