non-inferiority - miceapps · primary (latent) tuberculosis standard tmt = 9 months isoniazid...
TRANSCRIPT
1
Non-inferiority RCTs
..& other ‘new breeds’ of RCTs
Non-inferiority RCTs
..& other ‘new breeds’ of RCTs
Craig Mellis2019
Learning Objectives
1. Recognise the ‘new’ RCT designs
2. Appreciate the roles of ‘new’ RCTs
3. How to appraise a Non-inferiority RCT
Controversy…-> Parachutes & Evidence Based Medicine
“..effectiveness parachutes not proved with RCTs..”
“..did not find any RCTs.”
Superiority, parallel group RCT
Superiority, parallel group RCT
Audience participation -> ‘Needs’ assessment
Q..How familiar are you with the following …
… “________” (type of) RCT..?
3 possible answers..
1. I’ve never heard of it
How familiar are you with the following …
… “________” (type of) RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
How familiar are you with the following …
… “________” (type of) RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
3. Clear understanding - & could explain to others
Q1. How familiar are you with…
… “Cluster” RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
3. Clear understanding & could explain to others
METHODS:
Design: Cluster, randomized, controlled, open trialPopulation:
n = 911 children aged 0 to 3 yearsN = 24 Day Care Centres
Intervention: Hand washing & education
Comparator:Usual care (Control)
Outcome: Rate of respiratory tract infections; F/Up= 8 months
Cluster RCT
ie, Randomisation of ‘GROUPS’eg, Schools, DCCs, Hospitals, ICUs.
Aim: Practical, & reduce ‘contamination’
But.. Loss of stats powerRequires ‘cluster analysis’
Q2. How familiar are you with…
… “N-of-1” RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
3. Clear understanding & could explain to others
“N-of-1” RCT
Repeated Randomised
Crossover trials
in
INDIVIDUAL patients
Intervention
Placebo (control)
Not new!.. NEJM, 1986, Guyatt & Sackett et al
Which patients are SUITABLE for N-of-1 RCT..?
Indications:
*Treatment -> rapid response & minimal carryover effect
*Chronic / frequently recurring symptoms..
eg, Back pain; Cough; Wheeze; Insomnia etc
* Patient / Clinician -> Highly motivated..!
Which patients are UNSUITABLE for N-of-1 RCT..?
Contraindications:
*Treatment -> slow onset / substantial carryover/ ‘cure’
*Rapidly progressive condition
* Patient / Clinician -> Not motivated..!
88 pageGuide..!
“N-of-1” RCT
Heirarchy of Evidence: Therapy
Q3. How familiar are you with…
… “Proof-of-Concept” RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
3. Clear understanding & could explain to others
Q4. How familiar are you with …
… “Pragmatic” RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
3. Clear understanding & could explain to others
continuum
‘Pragmatic’ RCTrials ‘Proof of Concept’ RCTs
Spectrum of RCT designs
continu
Pragmatic RCTrials ‘Proof of Concept’ RCTs
‘Real world’
High External Validity
Simple, practical design
Diverse settings
‘Explanatory’ RCTs
High Internal Validity
Sophisticated design
Controlled environment
Narrow range patients
Range of RCT designs
continuum
‘Pragmatic’ RCTrials ‘Proof of Concept’ RCTs
‘Real world’ RCT
High External Validity
Simple, practical design
Diverse settings (GP/eHR)
Broader range patients
‘Explanatory’ RCT
High Internal Validity
Sophisticated design
Controlled environment
Narrow range patients
Range of RCT designs
Example: ‘Real world effectiveness’
Parallel, multicenter, pragmatic RCT...
Setting: 53 primary care practices in the United Kingdom.
Patients:N=300, age 12 to 80 years with doctor diagnosis of asthma
(requiring initiation of asthma-controller therapy).
Randomised:Open-label -> LTRA (Montelukast) or Inhaled glucocorticoid
(Beclo; Bud; or FP).Duration: 24 months
Outcome:Primary outcome measure: Asthma Q-of-L Score.
Conclusion…for 1st line asthma Tmt..
“ ..LTRAs are as effective as ICSteroids” (re Asthma Related Quality of Life)
Months
Q5. How familiar are you with …
… “Equivalence” RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
3. Clear understanding & could explain to others
Q6. How familiar are you with…
… “Non-inferiority” RCT..?
Have….
1. Never heard of it
2. Heard term - ? some idea what it means
3. Clear understanding & could explain to others
Aim -> Two Active Treatments =
“ Not TOO Different..! ”
re clinically important outcomes
EQUIVALENCE
EQUIVALENCE
Tmt A vs Tmt B ..“Not TOO Different..”
OLD NEW
Aim -> ‘New’ treatment is WORSE…!
but… Not “TOO MUCH” worse..!( ie, ‘acceptably worse’ )
..than current, standard treatment.
NON-INFERIORITY
Aim -> ‘New’ treatment is WORSE…!
but… Not “TOO MUCH” worse..!( ie, ‘acceptably worse’ )
..than current, standard treatment.
AND.. ‘New’ Tmt has other advantages
NON-INFERIORITY
Tmt Bnew “Not MUCH worse..” than Tmt Aold
NON-INFERIORITY
OLD
NEW
Non-inferiority RCTs
• New Drug vs Old (‘Standard Tmt’)
• Assumes New is ‘similar’.. But LESS Effective
“Trade-off” re advantages ‘New’ ..
eg …cheaper / safer..
…more acceptable / more convenient
…less hassles
Non-inferiority RCTs
• New Tmt vs Old (‘Standard’) Tmt’
• New Tmt has ‘similar’ efficacy..
• Sponsored by Pharma ( of NEW Tmt..! )
Child with Primary (‘latent’) Tuberculosis
Primary (‘latent’) Tuberculosis
‘Standard’ Tmt = 9 months Isoniazid (INAH)( ‘chemoprophylaxis’ )
Primary ( ‘latent’ ) Tuberculosis
* ‘New’ approach = 4 mo Rifampin
* Advantage of shorter course
-> better Adherence -> better Efficacy
BUT…RIFAMPIN -> Adverse Events
SAFETY: 4 months Rifampin vs 9 months Isoniazid.
Funded by Canadian Institutes of Health Research
“..Non-inferiority, open-label RCT.”
Primary Outcome: Adverse effect -> Cessation Tmt
Secondary Outcome: Any Adverse effect
* ASSUMPTIONS..
1. Similar Efficacy ( INAH vs RIF )
2. Better Adherence with Rifampin..
3. BUT .. Adverse Events with Rifampin
But..? How much ‘trade off’ acceptable -> ie, more side effects with Rifampin
-> for better adherence..
From literature rate AEs on Isoniazid = 6%
..Expect higher rate AEs with Rifampin
Question for you..
5
4
Rate AEs on Isoniazid = 6%
Q: How much of an INCREASE in Adverse Effects ‘acceptable’ with RIF (*RD)…?
A. 2% ( ie, rate AEs with RIF -> 8% )B. 5% ( -> 11% ) C. 10% ( -> 16% )
*RD = Risk Difference (Absolute RD)
Rate AE Isoniazid = 6%
The authors chose an additional 5% (RD)…
A. 2% B. 5% ( rate AEs with Rif <11% ) C. 10%
ie, ‘Non-inferiority Boundary’ = 5% difference
ie, Maximum ‘acceptable’ increase side effects on Rifampin.
If CI95% < 11% (RIF) = Non-inferior wrt AE ( vs 6% INAH)
If ‘superiority’ RCT..
Risk Difference ( Adverse Events )
RIF better RIF worse (Less side effects) (More side effects)
0%
Non-inferiority boundary (‘Trade Off’)...
Absolute Risk Difference (ARD)
RIF better RIF worse (Less side effects) (More side effects)
0% 5%
Non-inferiority boundary (‘Trade Off’)...
Absolute Risk Difference (ARD)
RIF better RIF worse (Less side effects) (More side effects)
0% 5%
Zoneof Non-inferior
?Possible Result #1 = ISONIAZID is ‘superior’( wrt ‘Safety’)
0% 5%
Risk Difference
RIF WorseRIF Better
CI95%
?..Result #2 = RIF ‘superior’ wrt ‘Safety’ ( Unlikely..! )
0% 5%
Risk Difference
RIF WorseRIF Better
?..Result #3 = ‘Underpowered’
0% 5%
Risk Difference
RIF WorseRIF Better
?..Result #4 = RIF ‘Non-inferior’..!
0% 5%
Risk Difference
RIF WorseRIF Better
RESULTS:
Severe Adverse Effects -> Stop Treatment
-> Rifampin = 0% vs Isoniazid = 0%..!!
Secondary Outcome: -> Minor AEs = 8.1% (RIF) vs 8.5% (INAH)
Risk Diff = -0.3% ( CI95%: -3.3% to 2.7% )
RESULTS ( 20 OM : Minor AEs)..RD = -0.3% ( CI95%: -3.3% to 2.7% )RIF = ‘Non-inferior’ ( ie, <5% RD )
0% 5%
Risk DifferenceRIF WorseRIF Better
Adherence:Rifampin = 86.5% vs Isoniazid = 77.1%
Difference (adjusted) = 13.6% better on RIF(CI955= 7.9 to 19.3); p<0.05
Checklist (Crit Appraisal): Non-inferiority RCT
Critical Appraisal: Non-inferiority RCTs
Step 1. RCT Valid.?
* Prognostic balance at Start ?Concealed Randomization (n = 829);
Risk (prognostic) factors balanced (table 1)
* Prognostic balance Maintained ?Open label (*Adjudicators of AEs = Blinded)
Min Loss Follow-Up (2%) ; ‘Modified’ ITT (?)
Appraisal of Non-inferiority RCTs
Step 1. RCT Valid..
Step 2. Non-inferiority established..?
Did investigators guard against
Unwarranted conclusion of non-inferiority?
Quality criteria wrt Non-inferiority…
1. Non-inferiority boundary defined & justified.?
YES…
Tho, used adult rates AEs (9 mo Isoniazid)
Quality criteria wrt Non-inferiority…
1. Non-inferiority boundary defined & justifiedYES. Adult rates AE (on 9 mo Isoniazid)
2. Sample size calc took boundary into account..?
YES…
80% power detect non-inferiority boundary
Quality criteria wrt Non-inferiority…
2. Sample size calc took boundary into accountYES. 80% power non-inferiority boundary
3. Both ‘ITT’ & ‘per-protocol’ analyses..?
?..YES / ..? NO
“Mod ITT”(?) -> ‘Per-protocol’-> same (?!)
( ‘justified’: Min loss F/Up & Good adherence )
Quality criteria wrt Non-inferiority…
3. Both ITT & per-protocol analyses?..YES ..Per-protocol-> same as “Mod ITT”
4. Was the effect of ‘Standard’ Tmt retained?
NO..!…AE rate on Isoniazid very low (0%)..!Expected rate AEs = 6%
No obvious explanation (?child vs adult)Dosages for both were ‘standard’
Quality criteria wrt Non-inferiority…
4. Was the effect of ‘Standard’ Tmt retained?NO…AE rate on Isoniazid very low (0%)..!No obvious explanation (?child vs adult)
5. Results include CI95%..?
YES.
CI95% = −3.3% to 2.7% ( RD = -0.3% )
ie, < Non-inferiority boundary (ie, <5%)
Question for you..
7
6
Q. What is the effect on SAFETY of longer duration Tmt with Isoniazid (ie, 9 m vs 4 mo Rif) ?
? ..Promotes -> ‘Non-inferiority’ of RIF…
Yes No Unsure
Q. Effect on SAFETY of Longer Duration Tmt with Isoniazid (9 vs 4 mo Rif) ?
YES -> Likely MORE AEs with 9 mo Isoniazid
-> ‘Non-inferior’ wrt AEs with 4 mo RIF..!
…“to account for the longer treatment period & more follow-up
visits in isoniazid group, we estimated the average percentage of visits in which minor symptoms (such as stomach upset, poor appetite, or fatigue were reported).”
Question for you..
7
9
Q. Are you convinced by author’s conclusion of 'Non-inferiority' ..?
0/10‘TotallyUNCONVINCED’
10/10‘Totally
CONVINCED’
..difficultes in design, conduct, analysis & interpretation..!
Syst Review: ‘Quality of Non-inferiority RCTs’JAMA (2006);295:1147–1151
n=162 N.I. RCTs.. Using 4 Quality criteria:1. Non-inferiority boundary defined & justified 2. Sample size calculation took boundary into account3. Both ITT & per-protocol analyses reported4. Results include CI95%
* Only 20.3% (of 162) fulfilled all 4 criteria..!
* And… 12.1% of high quality trials -> -> misleading conclusions..!
*ie, Medical journals with impact factor >10
Esp.. Lancet & New England Journal of Medicine (>75%) (also JAMA, BMJ, & Annals of Internal Medicine)
Published between January 2010 -> May 2015
n = 168 N.I. trials included.
“Non-inferiority trials: Systematic review of reporting in *MAJOR general medical journals.”
Rehal S, et al. BMJ Open (2016)
Results: n = 168 N.I. trials
79% (132/168) trials concluded ‘Non-inferiority’…!
Only 46% (77/168) -> Justification of ‘N.I. margin’.
“Non-inferiority trials: Systematic review of reporting in major medical journals.”
Rehal S, et al. BMJ Open (2016)
Results: n = 168 N.I. trials
35% (65/168) -> conducted & report only one analysis-> most commonly ITT analysis.
59% (99/168) ignored “missing data” ..!
Conclusion (in high impact Med Js):“Reporting of non-inferiority trials is POOR…!”
“Non-inferiority trials: Systematic review of reporting in major medical journals.”
Rehal S, et al. BMJ Open 2016;6:e012594.
Be suspicious of Non-inferiority RCTs..!
No Journal, paper, or author is above suspicion of ‘SPIN’
.. How to ‘FAKE’ -> Non-inferiority..!( esp..wrt ‘Safety’ )
1. Population..
Exclude high risk patients
Enroll non-adherant patients
Enroll currently tolerating ‘unsafe’ drug
2. Intervention / Comparator
Low dose of ‘unsafe’ intervention
Protective co-interventions
Shorter duration of ‘unsafe’ drug
.. How to ‘FAKE’ -> Non-inferiority..!(esp..wrt ‘Safety’)
..How to ‘FAKE’ -> Non-inferiority..!
3. Outcome..
Unjustified ‘Non-inferiority Boundary’
Less important 10 outcome measure
Surrogate outcome measure
..How to ‘FAKE’ -> Non-inferiority..!
4. Study design / execution
Poor (or un-measured) adherence
Large 'Loss to Follow-up’
Small sample size / Low event rates
Learning Objectives
1. Recognise the ‘new’ RCT designs
2. Appreciate the roles of ‘new’ RCTs
3. How to appraise a Non-inferiority RCT
-> Take care with…claims of ‘Non-inferiority’
Take Home Message