eduardo bergel’s controlled clinical trial simulator (cts) beta version 0.21
Post on 14-Jan-2016
36 Views
Preview:
DESCRIPTION
TRANSCRIPT
Eduardo Bergel’sControlled Clinical Trial
Simulator (CTS) Beta Version 0.21
Used by Dave Sackett (Oct Used by Dave Sackett (Oct ‘04)‘04)
Trout Research & Education Center Trout Research & Education Center at Irish Lake, Canadaat Irish Lake, Canada
Table of Contents
How to How to runrun the CTS the CTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Slide # 4Slide # 4
Effect of sample size on confidence intervalsEffect of sample size on confidence intervals . . . # . . . # 3838
The effects of admitting patients with different The effects of admitting patients with different risks and responsiveness to your RCT . . . . . . . . . risks and responsiveness to your RCT . . . . . . . . . . . . # 53. . . # 53
1. How to run the CTS
You tell the Controlled Trial Simulator
1.1. How manyHow many patients you hope to patients you hope to recruit.recruit.
2.2. Your guess at their risk of Your guess at their risk of eventsevents if if they receive they receive no Rxno Rx or the or the control Rx.control Rx.
3.3. Your hopes for their Your hopes for their responsivenessresponsiveness to to experimental Rx.experimental Rx.
4.4. With what rates of With what rates of cross-overcross-over, , loss-to-loss-to-follow-upfollow-up, and , and compliancecompliance
The CTS will then:
““Run” your trial hundreds or Run” your trial hundreds or thousands of times.thousands of times.
- and tell you, with tables and - and tell you, with tables and graphs, its results, their confidence graphs, its results, their confidence intervals, and the presence and intervals, and the presence and magnitude of bias (e.g., from magnitude of bias (e.g., from cross-overs, low compliance, etc.) cross-overs, low compliance, etc.)
For example, does aspirin reduce strokes after TIA ? We estimated that 21% of patients We estimated that 21% of patients
with TIAs would go on to stroke or with TIAs would go on to stroke or death within 2 years.death within 2 years.
We thought we could reduce this by We thought we could reduce this by one-third with aspirin (to 14%).one-third with aspirin (to 14%).
We thought we could recruit about 600 We thought we could recruit about 600 patients.patients.
For example, does aspirin reduce strokes after TIA ?
We enter these estimates and We enter these estimates and guesses into the RCT Simulator.guesses into the RCT Simulator.
And then tell the RCT Simulator how And then tell the RCT Simulator how many times to run our trial.many times to run our trial.
We thought that 21% would
stroke or die without Rx CER =
21%
We thought that 21% would
stroke or die without Rx CER =
21%We thought that aspirin could reduce these outcomes by 1/3, to 14%
EER = 14%
We thought we could recruit 600
patientsWe thought that
21% would stroke or die
without Rx CER = 21%
We thought that aspirin could reduce these outcomes by 1/3, to 14%
EER = 14%
We thought we could recruit 600
patientsWe thought that
21% would stroke or die
without Rx CER = 21%
We thought that aspirin could reduce these outcomes by 1/3, to 14%
EER = 14%
We wanted our RCT replicated
3000 times
3000 simulations take 5 seconds !
And their results are displayed in a And their results are displayed in a tabletable
CER
CER
EER
CER
EER
300 300
CER
EER
3000 simulatio
ns
300 300
CER
EER
Relative Risk
Reduction
3000 simulatio
ns
300 300
CER
EER
Relative Risk
Reduction
3000 simulatio
ns
Calculates Power !
300 300
CER
EER
Relative Risk
Reduction
3000 simulatio
ns
Calculates Power !
300 300
Can graph results
Graph Options
1.1. Can simply graph the overall RRR Can simply graph the overall RRR and its 95% confidence interval.and its 95% confidence interval.
RRR for aspirin in TIA
X = true RRR that is free of bias
Graphic Options
1.1. Can simply graph the overall RRR Can simply graph the overall RRR and its 95% confidence interval.and its 95% confidence interval.
2.2. Can create a histogram of Can create a histogram of individual RRRs from each individual RRRs from each simulation.simulation.
Histogram of RRR
Graphic Options
1.1. Can simply graph the overall RRR Can simply graph the overall RRR and its 95% confidence interval.and its 95% confidence interval.
2.2. Can create a histogram of Can create a histogram of individual RRRs from each individual RRRs from each simulation.simulation.
3.3. Can plot the individual RRRs from Can plot the individual RRRs from each simulation.each simulation.
Can plot every trial
Can plot every trial
Some RRRs < 0
Can plot every trial
Extreme results must
occur !
Some RRRs < 0
Graphic Options1.1. Can simply graph the overall RRR Can simply graph the overall RRR
and its 95% confidence interval.and its 95% confidence interval.
2.2. Can create a histogram of Can create a histogram of individual RRRs from each individual RRRs from each simulation.simulation.
3.3. Can plot the individual RRRs from Can plot the individual RRRs from each simulation.each simulation.
4.4. Can plot RRR vs. P-values.Can plot RRR vs. P-values.
Can chart the P-values for every trial
Can chart the P-values for every trial
RRR & 95% CI
Can chart the P-values for every trial
P> .05
P<.05
Can chart the P-values for every trial
P> .05
P<.05
Power = 60%
Can chart the P-values for every trial
P<.05
Power = 60%
P almost 0.05
against!
So, that is the basic system
Outputs can also be expressed in all Outputs can also be expressed in all these formats as:these formats as:Relative RisksRelative Risks
Currently under construction:Currently under construction:Absolute Measures (ARR, NNT, etc)Absolute Measures (ARR, NNT, etc)Cluster TrialsCluster Trials
2. The effect of sample size on 95% confidence intervals
Can use it to understand the effect of different sample sizes on the same RRR:
How to do thisHow to do this: : While keeping the treatment effect While keeping the treatment effect constant (say, RRR = 25%), constant (say, RRR = 25%), double, triple and quadruple your double, triple and quadruple your sample size.sample size.
Increasing Sample Size
Can have 4 different (sub) groups
Increasing Sample Size
Increasing Sample Size
Then simulate it 3000 times
Increasing Sample Size
All2500
Increasing Sample Size
250
All2500
Increasing Sample Size
250
1000
All2500
Now you can answer a frequently asked question:
If you want to cut the 95% If you want to cut the 95% confidence interval (CI) on your confidence interval (CI) on your RRR in half, by how much do you RRR in half, by how much do you need to increase your sample size need to increase your sample size (N)?(N)?
You need to remember “the only formula you’ll ever need”
CONFIDENCECONFIDENCE
(recalling that (recalling that narrowernarrower confidence confidence intervals provide intervals provide higherhigher confidence) confidence)
ConfidenceConfidence = =
You need to remember “the only formula you’ll ever need”
CONFIDENCECONFIDENCE
(recalling that (recalling that narrowernarrower confidence confidence intervals provide intervals provide higherhigher confidence) confidence)
ConfidenceConfidence = =
SignalNoise
You need to remember “the only formula you’ll ever need”
CONFIDENCECONFIDENCE
(recalling that (recalling that narrowernarrower confidence confidence intervals provide intervals provide higherhigher confidence) confidence)
ConfidenceConfidence = x = x
√√sample sizesample size
SignalNoise
Increasing Sample Size
250
1000
All2500To cut this
in half
Increasing Sample Size
250
1000
All2500To cut this
in half
You must quadruple
N
3. The effects of admitting patients with different risks and responsiveness to your RCT
Risk and Responsiveness
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHigh
LowLow
Risk and Responsiveness
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHigh IdealIdeal
LowLow
Risk and Responsiveness
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHigh IdealIdeal
LowLow Ever?Ever?
Risk and Responsiveness
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHigh IdealIdeal ??????
LowLow ?????? Ever?Ever?
Risk and Responsiveness
Suppose that . .Suppose that . .ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk(in terms (in terms of of CControl ontrol EEvent vent RRates)ates)
HighHigh
CER= .30CER= .30
LowLow
CER= .10CER= .10
Risk and Responsiveness
And And suppose . . . suppose . . .
ResponsivenessResponsiveness(in terms of (in terms of RRelative elative
RRisk isk RReduction)eduction)
HighHigh
RRR= RRR= 30%30%
LowLow
RRR= RRR= 10%10%
RiskRisk(in terms (in terms of of CControl ontrol EEvent vent RRates)ates)
HighHigh
CER= .30CER= .30
LowLow
CER= .10CER= .10
Risk and Responsiveness
These would These would result in the result in the following following reductions in reductions in events:events:
ResponsivenessResponsiveness(in terms of (in terms of RRelative elative
RRisk isk RReduction)eduction)
HighHigh
RRR= RRR= 30%30%
LowLow
RRR= RRR= 10%10%
RiskRisk(in terms (in terms of of CControl ontrol EEvent vent RRates)ates)
HighHigh
CER= .30CER= .30.30.30
.2.211
LowLow
CER= .10CER= .10
Risk and Responsiveness
These would These would result in the result in the following following reductions in reductions in events:events:
ResponsivenessResponsiveness(in terms of (in terms of RRelative elative
RRisk isk RReduction)eduction)
HighHigh
RRR= RRR= 30%30%
LowLow
RRR= RRR= 10%10%
RiskRisk(in terms (in terms of of CControl ontrol EEvent vent RRates)ates)
HighHigh
CER= .30CER= .30.30.30
.2.211
LowLow
CER= .10CER= .10.10.10
.0.077
Risk and Responsiveness
These would These would result in the result in the following following reductions in reductions in events:events:
ResponsivenessResponsiveness(in terms of (in terms of RRelative elative
RRisk isk RReduction)eduction)
HighHigh
RRR= RRR= 30%30%
LowLow
RRR= RRR= 10%10%
RiskRisk(in terms (in terms of of CControl ontrol EEvent vent RRates)ates)
HighHigh
CER= .30CER= .30.30.30
.2.211
.30.30
.2.277
LowLow
CER= .10CER= .10.10.10
.0.077
Risk and Responsiveness
These would These would result in the result in the following following reductions in reductions in events:events:
ResponsivenessResponsiveness(in terms of (in terms of RRelative elative
RRisk isk RReduction)eduction)
HighHigh
RRR= RRR= 30%30%
LowLow
RRR= RRR= 10%10%
RiskRisk(in terms (in terms of of CControl ontrol EEvent vent RRates)ates)
HighHigh
CER= .30CER= .30.30.30
.2.211
.30.30
.2.277
LowLow
CER= .10CER= .10.10.10
.0.077
.10.10
.0.099
Let’s start with the “ideal” Let’s start with the “ideal”
HighRisk - High Response group HighRisk - High Response group
High risk – high response
High risk – high response
Almost Stat. Sig.
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
LowLow
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
LowLowWould these pts help ?
Hi Risk-Hi Response + Lo Risk-Hi Response
Lo Risk – Hi Resp
Hi Risk-Hi Response + Lo Risk-Hi Response
Now Stat. Sig !
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
LowLowUsually helpful
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
Would these pts help ?
LowLowUsually helpful
Hi Risk-Hi Response + Hi Risk-Lo Response
Hi Risk-Hi Response + Hi Risk-Lo Response
Not helpful
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
Often not helpful
LowLowUsually helpful
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
Often not helpful
LowLowUsually helpful
Would these pts help?
Hi Risk-Hi Response + Lo Risk-Lo Response
Hi Risk-Hi Response + Lo Risk-Lo Response
Makes things much
worse !
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
Often not helpful
LowLowUsually helpful
Make things worse!
Taking all comers
If we put all four If we put all four groups of groups of patients into our patients into our trial:trial:
ResponsivenessResponsiveness(in terms of (in terms of RRelative elative
RRisk isk RReduction)eduction)
HighHigh
RRR= RRR= 30%30%
LowLow
RRR= RRR= 10%10%
RiskRisk(in terms (in terms of of CControl ontrol EEvent vent RRates)ates)
HighHigh
CER= .30CER= .30.30.30
.2.211
.30.30
.2.277
LowLow
CER= .10CER= .10.10.10
.0.077
.10.10
.0.099
All patients
All patients
Worse than if we stuck to just half as many high-
response pts !
Risk and Responsiveness
SUMMARY:SUMMARY:
Who do you Who do you want in your want in your RCT ?RCT ?
ResponsivenessResponsiveness
HighHigh LowLow
RiskRisk
HighHighIdeal, but often not enough
Often not helpful
LowLowUsually helpful
Make things worse!
Summer at Irish LakeSummer at Irish Lake
top related