objectives introduction to rctsintroduction to rcts level of evidenceslevel of evidences three step...

EBMRC Dr.Soltani EMRC

Three Step Guide in Three Step Guide in Using an Article to Using an Article to

Assess TherapyAssess Therapy


Three Step Guide in Using an Article to Three Step Guide in Using an Article to Assess TherapyAssess Therapy

1.1. Are the results of the study valid? Are the results of the study valid?

2.2. What are the results? What are the results? What measures of What measures of precision of effects were reported (CIs, p-precision of effects were reported (CIs, p-values)?values)?

3.3. How can I apply these results to patient care? How can I apply these results to patient care?

Assess Validity and Applicability to my practice setting

1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop)2.Were the patients properly selected for the trial and 2.Were the patients properly selected for the trial and

randomized with concealed assignment?randomized with concealed assignment? Yes (go on) No (stop)Yes (go on) No (stop)3.Were patients and study personnel “blind” to treatment?3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause)Yes (go on) No (pause)4.Were the intervention and control groups similar at the 4.Were the intervention and control groups similar at the

start? start? (Check “Table 1” of most studies)(Check “Table 1” of most studies) Yes (go on) No (stop)Yes (go on) No (stop)5.Was follow-up complete?5.Was follow-up complete?ii. Were patients analyzed in the groups to which they were ii. Were patients analyzed in the groups to which they were

randomized (“intention-to-treat” analysis)?randomized (“intention-to-treat” analysis)?


Simple randomization Simple randomization • An almost infinite number of methods can be used to generate a

simple randomisation sequence based on a random-number. For example, for equal allocation to two groups,predetermine the direction to read the table: up, down, left, right, or diagonal. Then select an arbitrary starting point—ie, first line, 7th number:

56 99 20 20 52 49 05 78 58 50 62 86 52 11 8831 60 26 13 69 74 80 71 48 73 72 18 60 58 20

55 59 06 67 02 . . .• For equal allocation, an investigator could equate odd and even

numbers to interventions A and B, respectively.• Therefore, a series of random numbers 05, 78, 58, 50, 62, 86, 52, 11,

88, 31, &c, represent allocation to intervention A,B, B, B, B, B, B, A, B, A, &c.

• Alternatively, 00–49 could equate to A and 50–99 to B, or numbers 00–09 to A and 10–19 to B,ignoring all numbers greater than 19.

• Any of a myriad of options suffice, provided the assignment probabilities and the investigator adhere to the predetermined scheme.


Benefits of Random Allocation Benefits of Random Allocation (Randomization)(Randomization)

1.Reduces bias in those selected for 1.Reduces bias in those selected for treatmenttreatment guarantees treatment assignment will not be guarantees treatment assignment will not be

based on patients’ prognosisbased on patients’ prognosis

2.Prevents confounding2.Prevents confounding known and unknown potential confounders are known and unknown potential confounders are

evenly distributedevenly distributed

Assess Validity and Applicability to Assess Validity and Applicability to my practice settingmy practice setting

1.Is the study a randomized control trial (RCT)?1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop)Yes (go on) No (stop)2.Were the patients properly selected for the trial and

randomized with concealed assignment? Yes (go on) No (stop)3.Were patients and study personnel “blind” to treatment?3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause)Yes (go on) No (pause)4.Were the intervention and control groups similar at the 4.Were the intervention and control groups similar at the




Is allocation concealed?Is allocation concealed?


Ensuring Allocation Ensuring Allocation ConcealmentConcealment

BEST – most valid technique Central computer randomization

DOUBTFUL Envelopes, etc

NOT RANDOMIZED Date of birth, alternate days, etc


Do Not Confuse Do Not Confuse Allocation Allocation ConcealmentConcealment

with with BlindingBlinding

• Allocation concealment seeks to Allocation concealment seeks to prevent prevent selection bias,selection bias, protects protects assignment sequence before and assignment sequence before and until allocation, and can until allocation, and can alwaysalways be be successfully implementedsuccessfully implemented


Importance of allocation concealment

• Trials that used inadequate or unclear allocation concealment, yielded up to 40% larger estimates of effect.

• Moreover, the worst concealed trials yielded greater heterogeneity in results—ie, the results fluctuated extensively above and below the estimates from better studies.

• Indeed, having a randomized (unpredictable) sequence should make little difference without adequate allocation concealment.


Do Not Confuse Allocation Do Not Confuse Allocation ConcealmentConcealment with with BlindingBlinding

(Cont’d)(Cont’d)

• Blinding seeks to prevent Blinding seeks to prevent information biasinformation bias, protects , protects sequence after allocation, and sequence after allocation, and cannotcannot always be successfully always be successfully implementedimplemented


1.Is the study a randomized control trial (RCT)?1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop)Yes (go on) No (stop)2.Were the patients properly selected for the trial and 2.Were the patients properly selected for the trial and

randomized with concealed assignment?randomized with concealed assignment? Yes (go on) No (stop)Yes (go on) No (stop)3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause)4.Were the intervention and control groups similar at the 4.Were the intervention and control groups similar at the




Placebo effectPlacebo effectTrial in patients with chronic severe itchingTrial in patients with chronic severe itching

0

10

20

30

40

50

60

I tching scoreCyproheptadine HCL

Trimeprazine tartrate

No treatment

Treatment vs no treatment for itching


Placebo effectPlacebo effectTrial in patients with chronic severe itchingTrial in patients with chronic severe itching

0

10

20

30

40

50

60

I tching scoreCyproheptadine HCL

Trimeprazine tartrate

Placebo

No treatment

Treatment vs no treatment vs placebo for itching

Placebo effect - attributable to the expectation thatthe treatment will have an effect


Double-Blinded Single-BlindedDouble-Blinded Single-Blinded


Blinding and ReportingBlinding and Reporting• Usually reduces differential assessment of Usually reduces differential assessment of

outcomes (outcomes (information biasinformation bias))

• Authors should explicitly state who was blinded – Authors should explicitly state who was blinded – and how. and how.

• Many investigators and readers consider a Many investigators and readers consider a randomized trial as randomized trial as high qualityhigh quality simply because simply because it is “double-blind,” as if double-blinding is the it is “double-blind,” as if double-blinding is the sine sine qua nonqua non of an RCT. of an RCT.

Trials not “double-blinded” should Trials not “double-blinded” should notnot automatically be deemed inferior trials.automatically be deemed inferior trials.


Blinding in randomised trials: hiding who got what

• Double blinding prevents bias but is less important, than adequate allocation concealment.

• open studies are more likely to favour experimental interventions over the controls and that studies that are not double-blinded can exaggerate effect estimates by 17%

• Furthermore, in some trials, blinding cannot be successfully implemented, whereas allocation concealment can always be successfully implemented.



randomized with concealed assignment?randomized with concealed assignment? Yes (go on) No (stop)Yes (go on) No (stop)3.Were patients and study personnel “blind” to treatment?3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause)Yes (go on) No (pause)4.Were the intervention and control groups similar at the

start? (Check “Table 1” of most studies) Yes (go on) No (stop)5.Was follow-up complete?5.Was follow-up complete?ii. Were patients analyzed in the groups to which they were ii. Were patients analyzed in the groups to which they were



Comparison of baseline data

Chan et al. Lancet 1997

Does P>0.05 indicate

comparability of treatment groups?


Significance tests for baseline differences

Chan et al. Lancet 1997

INAPPROPRIA

TE


Baseline dataBaseline data

AzathioprineAzathioprine PlaceboPlacebo

Mean ageMean age 54.754.7 54.954.9

Serum bilirubin (Serum bilirubin (mol/L)mol/L) 37.237.2 30.930.9

Stage I disease %Stage I disease % 1414 1212

Stage II disease %Stage II disease % 4444 4343

Stage III disease %Stage III disease % 1515 1515

Stage IV disease %Stage IV disease % 2727 3030

Christensen et al. Gastro 1985

Effect of azathioprine on the survival of patients with primary biliary cirrhosis


How complete was the How complete was the follow upfollow up? ? How many dropouts were there?How many dropouts were there?

• Conventionally, a Conventionally, a 20%20% drop out rate is drop out rate is regarded as acceptable, but this depends on regarded as acceptable, but this depends on the study question. the study question.

• Some regard should be paid to Some regard should be paid to whywhy participants dropped out, as well as how participants dropped out, as well as how many. many.

• It should be noted that the drop out rate may It should be noted that the drop out rate may be expected to be higher in studies conducted be expected to be higher in studies conducted over a over a long period of time.long period of time.

• A higher drop out rate will normally lead to A higher drop out rate will normally lead to downgradingdowngrading, rather than rejection of a study., rather than rejection of a study.


Over-estimation oftreatment effect

Bias: a one-sided inclinationBias: a one-sided inclinationOf the mindOf the mind

• Not random Not random 40%40%

• Not double-blindNot double-blind 17%17%

• Duplicate informationDuplicate information 20%20%

• Small trialsSmall trials 30%30%

• Poor reporting qualityPoor reporting quality 25%25%

EBMRC Dr.Soltani EMRCMontorri V, Guyatt G. CMAJ 2001 165(10) p1340

Intention to treat


Montorri V, Guyatt G. CMAJ 2001 165(10) p1340

Intention to treat


Montorri V, Guyatt G. CMAJ 2001 165(10) p1340

Intention to treat

High risk?


Summary Summary

1.Is the study a 1.Is the study a randomizedrandomized control trial (RCT)? control trial (RCT)? Yes (go on) No (stop)Yes (go on) No (stop)2.Were the patients properly selected for the trial and 2.Were the patients properly selected for the trial and

randomized with randomized with concealedconcealed assignment? assignment? Yes (go on) No (stop)Yes (go on) No (stop)3.Were patients and study personnel “3.Were patients and study personnel “blindblind” to ” to

treatment?treatment? Yes (go on) No (pause)Yes (go on) No (pause)4.Were the intervention and control groups similar at the 4.Were the intervention and control groups similar at the

start? start? (Check “Table 1” of most studies)(Check “Table 1” of most studies) Yes (go on) No (stop)Yes (go on) No (stop)5.Was follow-up complete?5.Was follow-up complete?


Three Step Guide in Three Step Guide in Using an Article to Using an Article to

Assess TherapyAssess Therapy(Part-2)(Part-2)




2.2. What are the results?What are the results? What measures of What measures of precision of effects were reported (CIs, p-precision of effects were reported (CIs, p-values)?values)?



Measuring Risk:Measuring Risk:Relative RiskRelative Risk

Treated Placebo

Patients with HT 2365 2371

Strokes 96 149

Rate 0.04 0.06

Relative Risk (RR) = rate in exposed = 0.04 = 0.67

rate in nonexposed 0.06

SHEP. JAMA. 1991;265:3255-3264


Communicating risk:Communicating risk:

10%

5%

10%

20% 20%

40%

RRR=?ARR=?

placebo


Measuring Risk:Measuring Risk:Relative RiskRelative Risk

Treated Placebo

Patients with HT 2365 2371

Strokes 96 149

Rate 0.04 0.06

Relative Risk (RR) = rate in exposed = 0.04 = 0.67

rate in nonexposed 0.06

SHEP. JAMA. 1991;265:3255-3264

4% 6%


Measuring Risk: ARR?Measuring Risk: ARR?

6%4%

2%


Measuring Risk:Measuring Risk:Absolute Risk ReductionAbsolute Risk Reduction

Absolute Risk Reduction (ARR)Absolute Risk Reduction (ARR) is the absolute difference is the absolute difference in in event ratesevent rates between the experimental and control patients. between the experimental and control patients.

Calculated by:Calculated by:

ARR = CER - EER = 0.06 - 0.04 = 0.02ARR = CER - EER = 0.06 - 0.04 = 0.02

In its decimal form the ARR is not easy to use!In its decimal form the ARR is not easy to use!

Converted to a percentage - there is an absolute risk reduction Converted to a percentage - there is an absolute risk reduction of of 2%2%


Measuring Risk: RRR?Measuring Risk: RRR?

6%4%

2%


Measuring Risk:Measuring Risk:Relative Risk ReductionRelative Risk Reduction

Relative Risk Reduction (RRR)Relative Risk Reduction (RRR) is the proportional reduction in is the proportional reduction in event ratesevent rates between the experimental and control patients.between the experimental and control patients.

Two ways to calculate:Two ways to calculate:RRR = (1 - RR) = (1 - 0.67) = 0.33 RRR = (1 - RR) = (1 - 0.67) = 0.33

ORORRRR = RRR = CER - EERCER - EER = = 0.06 - 0.040.06 - 0.04 = 0.33 = 0.33

CERCER 0.06 0.06

Therefore, treatment reduced the stroke rate by 33%Therefore, treatment reduced the stroke rate by 33% OR a OR a RRR of 33% means that the new treatment reduced the risk of death by RRR of 33% means that the new treatment reduced the risk of death by

33% relative to that occurring among control patients33% relative to that occurring among control patients

CER = control event rate EER = experimental event rate


Relative Risk Reduction:Relative Risk Reduction:

• Usually reported in studies. Usually reported in studies.

• Ratio of the improvement of outcome over Ratio of the improvement of outcome over outcome without intervention (Rx):outcome without intervention (Rx):

• {Control Event Rate (CER) — Experimental {Control Event Rate (CER) — Experimental Event Rate (EER)} / CEREvent Rate (EER)} / CER

• i.e. {CER-EER}/CERi.e. {CER-EER}/CER

• often independent of prevalence!often independent of prevalence!

• often similar at different ages!often similar at different ages!


Absolute Risk—> The risk our Absolute Risk—> The risk our patient is facing!patient is facing!

• How likely is our patient to die (or have the How likely is our patient to die (or have the outcome of interest) without intervention? = outcome of interest) without intervention? = Control Event Rate (CER)Control Event Rate (CER)

• consider this individual patient’s risk factors consider this individual patient’s risk factors to estimate Patient Expected Event Rate = to estimate Patient Expected Event Rate = PEER.PEER.

• Absolute Risk usually increases with age. Absolute Risk usually increases with age.

• Improvement measured as Absolute Risk Improvement measured as Absolute Risk Reduction (ARR)Reduction (ARR)


Usefulness of the ARR:Usefulness of the ARR:Number Needed to TreatNumber Needed to Treat

Number Needed to Treat (NNT)Number Needed to Treat (NNT) is the number of is the number of patients a clinician needs to treat in order to prevent patients a clinician needs to treat in order to prevent one additional adverse outcome. one additional adverse outcome.

NNT is for dichotomous outcomes.NNT is for dichotomous outcomes.

Calculated by:Calculated by:NNT = 1/ARR = 1/0.02 = 50NNT = 1/ARR = 1/0.02 = 50

Therefore, you would have to treat 50 hypertensive patients to prevent Therefore, you would have to treat 50 hypertensive patients to prevent one stroke.one stroke.


Number Needed to TreatNumber Needed to Treat

CER=0.06=100----------6CER=0.06=100----------6

EER=0.04=100----------4EER=0.04=100----------4

SO: 100-----------2SO: 100-----------2

X------------1X------------1

50-----------150-----------1Calculated by:Calculated by:

NNT = 1/ARR = 1/0.02 = 50NNT = 1/ARR = 1/0.02 = 50

Therefore, you would have to treat 50 hypertensive patients to Therefore, you would have to treat 50 hypertensive patients to prevent one stroke.prevent one stroke.


Why not just use RRR? PTH trial

CER = 6 % Age =70 + 7

EER = 3 %

ARR = CER – EER = 3%

NNT = 1/ARR = 1/ 0.03 = 33

CER = 1/1000 Age = 55 + 5

EER = 1/2000

ARR=1/1000-1/2000=1/2000

NNT = 1/ARR= 2000


Why not just use RRR?Why not just use RRR?

CER EER RRR ARR NNT

0.6 0.4 33% 20% 5

0.06 0.04 33% 2% 50

0.006 0.004 33% .2% 500

RRR remains the same despite differences in absolute rate of events.



CER EER RRR ARR NNT

0.6 0.4 33% 20% 5

0.06 0.04 33% 2% 50

0.006 0.004 33% .2% 500

ARRs reflect underlying susceptibility of patients and provides more complete information.



CER EER RRR ARR NNT

0.6 0.4 33% 20% 5

0.06 0.04 33% 2% 50

0.006 0.004 33% .2% 500

NNTs provide a useful measure of the clinical effort that must be expended to avoid bad events.


Randomised trial of cholesterol lowering in 4,444 patients with CHD: Randomised trial of cholesterol lowering in 4,444 patients with CHD: the Scandinavian Simvastatin Survival Study (4S)the Scandinavian Simvastatin Survival Study (4S)

Lancet 1994: 344; 1383-1389Lancet 1994: 344; 1383-1389

• 4,444 patients recruited as a sample4,444 patients recruited as a sample• inclusion criteriainclusion criteria

CHD and cholesterol 5.5 - 8 mmol/lCHD and cholesterol 5.5 - 8 mmol/l• exclusion criteriaexclusion criteria

planned cardiac surgery, HF, child bearing potentialplanned cardiac surgery, HF, child bearing potential• simvastatin Vs. placebosimvastatin Vs. placebo• double blinddouble blind• OutcomesOutcomes

mortality, major coronary events, admissions for acute mortality, major coronary events, admissions for acute CHD, incidence of revascularisation proceduresCHD, incidence of revascularisation procedures


4S Study: self evaluation4S Study: self evaluation• Median follow up 5.4 yMedian follow up 5.4 y

• analyse by intention to treatanalyse by intention to treat

• Significant reduction in all cause mortalitySignificant reduction in all cause mortality 11.5% placebo Vs. 8.2% simvastatin11.5% placebo Vs. 8.2% simvastatin ARR = ARR = RRR = RRR = NNT = patients with CHD and cholesterol 5.5 - 8 NNT = patients with CHD and cholesterol 5.5 - 8

need to treated with simvastatin (20 mg) for 5.4 need to treated with simvastatin (20 mg) for 5.4 years to save one lifeyears to save one life


4S Study Cont’d4S Study Cont’d• Median follow up 5.4 yMedian follow up 5.4 y• analyse by intention to treatanalyse by intention to treat• Significant reduction in all cause mortalitySignificant reduction in all cause mortality

11.5% placebo Vs. 8.2% simvastatin11.5% placebo Vs. 8.2% simvastatin ARR = ARR = 11.5 - 8.2 = 3.3%11.5 - 8.2 = 3.3% RRR = RRR = (11.5 - 8.2)/11.5 = 29%(11.5 - 8.2)/11.5 = 29% NNT = NNT = 1/ARR = 301/ARR = 30

• 30 patients with CHD and cholesterol 5.5 - 8 need to 30 patients with CHD and cholesterol 5.5 - 8 need to treated with simvastatin (20 mg) for 5.4 years to save treated with simvastatin (20 mg) for 5.4 years to save one lifeone life


Insulin treatment in MIInsulin treatment in MI

• acute MIacute MI• history of diabetes mellitushistory of diabetes mellitus

• At what level of insulin would you start intensive (i.v.) insulin At what level of insulin would you start intensive (i.v.) insulin treatment?treatment?

• 4,5 mmol/l (80 mg/dl)4,5 mmol/l (80 mg/dl)• 5,6 (100)5,6 (100)• 7,0 (126)7,0 (126)• 11,1 (200)11,1 (200)• 15 (270)15 (270)• 20 (360)20 (360)


InsulinInsulin treatment in MI treatment in MI (DIGAMI) (DIGAMI)BMJ 1997; 314: 1512 BMJ 1997; 314: 1512

• PopulaPopulationtion Acute MI, DMAcute MI, DM

• InterInterventionvention ((RCT, RCT, ITTAITTA)) intensive insulin Tx (i.v. infusion x 24h, than min. 4 intensive insulin Tx (i.v. infusion x 24h, than min. 4 x d. x d. for for

3 months, vs3 months, vs standard treatmentstandard treatment

• Outcome measureOutcome measure death at death at 3½ 3½ yearyear


InsulinInsulin treatment in MI treatment in MI (DIGAMI) (DIGAMI)BMJ 1997; 314: 1512 BMJ 1997; 314: 1512

• RRaa = =

• RRcc = =

ARR = RARR = Rcc – R – Raa

0,07 (0,01–0,14)0,07 (0,01–0,14)

NNT = 1/ARRNNT = 1/ARR 14 (8–174)14 (8–174)

RR = RRR = Raa/R/Rcc

0,73 (0,54–0,97)0,73 (0,54–0,97)

RRR = 1–RRRRR = 1–RR 27% (3–46)27% (3–46)

death

Yes No Total

intensive 58 248 306

standard 82 232 314

58/306 = 0,19

82/314 = 0,26


Relative and absoluteRelative and absolute

baseline risk

Ra Rc RR RRR ARR NNT

small

large

4% 3% 0,75 0,25 0,01 100

20% 15% 0,75 0,25 0,05 20


Relative and absolute (what’s Relative and absolute (what’s the differencethe difference?)?)

• spironolactone 2spironolactone 255 mg/ mg/dd

• death within 2 yearsdeath within 2 years

Ra Rc RR RRR ARR NNT # of pills Cost

35% 46% 0,76 0,24 0,11 9 6570 1000$

Drug lowering risk of death by 1/4 1000$ to delay one death What after 2 years


relative and absolute (what’s relative and absolute (what’s the differencethe difference?) ?)

• clopclopidogrel (75 mg/d)idogrel (75 mg/d)• cardio-vascular death, MI, CVA within 9 months after acute cardio-vascular death, MI, CVA within 9 months after acute

coronary syndrome.coronary syndrome.

• drug lowering the risk of MI by drug lowering the risk of MI by 1/31/3• 40 000 $ to prevent one MI40 000 $ to prevent one MI• what after 9 monthswhat after 9 months??• who who iis to decide (physician, patient, insurance, society)s to decide (physician, patient, insurance, society)??

Ra Rc RR RRR ARR NNT # of pills cost

13500 40000$9,3% 11,4% 0,82 0,18 0,02 50


Why NNT is not enough Why NNT is not enough

• NNT for continuous outcome is difficult to NNT for continuous outcome is difficult to calculatecalculate

• Time can be added to calculations cautiously Time can be added to calculations cautiously • We can not compare NNTs from different studies We can not compare NNTs from different studies

easilyeasily• Example: 2 RCT (Risedronate) 2 NNT! Can we Example: 2 RCT (Risedronate) 2 NNT! Can we

say Risedronate is better than Risedronate! say Risedronate is better than Risedronate!


Choice of Effect Size MeasuresChoice of Effect Size Measures

• RR and RRR most relevant for assessing RR and RRR most relevant for assessing causalitycausality• Relative risk and RRR for biological effectRelative risk and RRR for biological effect• Basal risk is very important for unbiased estimate Basal risk is very important for unbiased estimate

of risk in your patientsof risk in your patients• ARR and NNT are most relevant for ARR and NNT are most relevant for clinicalclinical and and

public healthpublic health decisionsdecisions• RR, RRR often favored because they look more RR, RRR often favored because they look more

impressiveimpressive• Papers should report absolute measures, relative Papers should report absolute measures, relative

measures and NNT (Sackett)measures and NNT (Sackett)


HHypothermia after cardiac arrestypothermia after cardiac arrestNEJM 2002; 346: 549NEJM 2002; 346: 549

• PopulaPopulationtion Patients resuscitated after cardiac arrestPatients resuscitated after cardiac arrest

• InterIntervention vention ((randomization, blinding, ITTArandomization, blinding, ITTA)) Cooling to Cooling to 32–34°C 32–34°C for for 24 h24 h Maintenance of normal temperatureMaintenance of normal temperature

• Outcome measureOutcome measure Death at 6 monthsDeath at 6 months


Hypothermia after cardiac arrestHypothermia after cardiac arrestNEJM NEJM 2002; 346: 5492002; 346: 549

• RRaa = =

• RRcc = =

ARR = RARR = Rcc – R – Raa

0,140,14

NNT = 1/ARRNNT = 1/ARR 88

RR = RRR = Raa/R/Rcc

0,740,74

RRR = 1–RRRRR = 1–RR 26%26%

death

Yes no total

Cooling 56 81 137

Normal 76 62 138

56/137 = 0,41

75/138 = 0,55


HHypothermia after cardiac arrestypothermia after cardiac arrestNEJM NEJM 2002; 346: 5492002; 346: 549

• Survival analysisSurvival analysis

• Relative Hazard Relative Hazard (Hazard Ratio)(Hazard Ratio)

• HR ~ 0,65HR ~ 0,65

100

100

hypothermia

normotermia

500 150 200

time [days

75

50

25

surv

ival [%

]

RR = 0,74


•• Survival analysisSurvival analysis

100

15

treated

Non-treated

100

Time [years]

75

50

25

Su

rviv

al w

ith

ou

t re

curr

en

ce [

%]

54 12

After 4 yearsRR ~ 0,16

After 12 yearsRR ~ 0,98

HR ~ 0,68


What measures of precision of effects were What measures of precision of effects were reported (CIs, p-values)? reported (CIs, p-values)?

• Either confidence intervals or p values Either confidence intervals or p values for the estimates of effect should be for the estimates of effect should be reported.reported.


P Value or CI?P Value or CI?


If no statistically significant effects If no statistically significant effects detected, was there sufficient power?detected, was there sufficient power?

• If an effect estimate is If an effect estimate is not significantlynot significantly different from no effect and the confidence different from no effect and the confidence interval is interval is widewide, the study is probably , the study is probably not not large enoughlarge enough to detect a real difference to detect a real difference between treatment and comparison groups (i.e. between treatment and comparison groups (i.e. a low power study). a low power study).

• A A non significantnon significant effect associated with a effect associated with a tighttight CICI suggests there is suggests there is no effectno effect and that the and that the study has adequate power. study has adequate power.


Confidence Intervals for Small NumeratorsConfidence Intervals for Small Numerators

• Example:Example: A new drug is given to A new drug is given to 60 60 people. It seems to work, and has no people. It seems to work, and has no serious adverse effects. serious adverse effects.

• The authors conclude it is "safe and The authors conclude it is "safe and effective." The upper limit for the 95% CI effective." The upper limit for the 95% CI for any serious adverse effect is, or 5%.for any serious adverse effect is, or 5%.


Confidence Intervals for Small Numerators

More than 10% of FDA approved have serious side effects.

Rule of three

Probability of event not occurring = 1-risk

Probability of event not occurring in n patient = (1-risk )^ n

(1- maxrisk ) ^ n = 0.05 1- maxrisk =(0.05)^(1/n)

For n>30 1-maxrisk = 1-3/n maxrisk = 3/n ; n > 3

95% confident of adverse event = 0/n – 3/n


Another example from Another example from textbooks!textbooks!

• Based on one observational study in 94 Based on one observational study in 94 women, no one report adverse reactions.women, no one report adverse reactions.

• Rule of 3: 3/94=upper limit of adverse Rule of 3: 3/94=upper limit of adverse effects that may not have been seen.effects that may not have been seen.


Approximate maximum event rate Approximate maximum event rate for small numeratorsfor small numeratorsObserved

NumeratorApproximate numerator for upper limit of

95% CI0 31 52 73 94 10


How can I apply the results to patient care?

• Were the study patients similar to my patient?

• Were all clinically important outcomes considered?

• Are the likely treatment benefits worth the potential harm and costs?


ExamplesExamples

•Brief Brief Critical Critical AppraisalAppraisal


Study 1:Study 1: MRC/BHF Heart Protection Study of MRC/BHF Heart Protection Study of Antioxidant vitamin supplementation, …Antioxidant vitamin supplementation, …

• Citation:Citation: Lancet 2002; 360: 23-33. Lancet 2002; 360: 23-33.• ?:?: Does Antiox sup (C, E, Beta Carotene) reduce death, Does Antiox sup (C, E, Beta Carotene) reduce death,

vascular events or Ca in people with high 5 year risk.vascular events or Ca in people with high 5 year risk.• Design:Design: RCT RCT• Subjects:Subjects: 20,536 (40-80 y.o.), 75% male, w high risk 20,536 (40-80 y.o.), 75% male, w high risk• Setting:Setting: 69 UK Hospitals, 5 yr f/u 69 UK Hospitals, 5 yr f/u• Control:Control: Placebo Placebo• Allocation concealed:Allocation concealed: Y Y• Blinded:Blinded: Pt, Dr, Collectors, Analyzers Pt, Dr, Collectors, Analyzers• Analysis:Analysis: ITT ITT


Study 1:Study 1: MRC/BHF Heart Protection Study of MRC/BHF Heart Protection Study of Antioxidant vitamin supplementation, …Antioxidant vitamin supplementation, …

• Findings:Findings: No benefit (in any outcome). All NNH No benefit (in any outcome). All NNH and NNT non-significant.and NNT non-significant.

• Limits:Limits: One dose, high risk, duration One dose, high risk, duration • Strengths:Strengths: RCT, # with 99.6% f/u RCT, # with 99.6% f/u (83 % (83 %

compliance)compliance)

• Funding:Funding: UK MRC, Brit Heart Found, Merck, UK MRC, Brit Heart Found, Merck, Roche Roche (*Reassured us analysis was independent of (*Reassured us analysis was independent of funding source)funding source)

• Comments:Comments: agree other RCT (put to rest a few agree other RCT (put to rest a few observational and Cambridge heart antioxidant observational and Cambridge heart antioxidant study)study)


Thank you Thank you

objectives introduction to rctsintroduction to rcts level of evidenceslevel of evidences three step...

Documents