clinical appraisal on risk & harm study · preclinical toxicology testing on animals...
TRANSCRIPT
22/04/57
1
Clinical Appraisal onRisk & Harm Study
CEB-rama.org
Section for Clinical Epidemiology and Biostatistics
Definition of Risk Study
CC-EBM
Most often used to express the probability thata particular outcome will occur following a particular exposure is “ RISK STUDY”
22/04/57
2
Principles
• Are the results valid ? ( Need to assessment !!)
• What are the results ?• Application or How can I apply
this result to our patient care ?
CEB-rama.org
Principles • Are the results valid ? ( Need to assessment !!)
• Cohort study– Similar prognosis factors ?– Method to detect outcome similar ?– Sufficient of follow up ?
• Case-control study– Case vs. Control have circumstances that would lead to exposure– Same circumstances and methods to determine the exposure
• Cross-sectional study• What are the results ?
• Strong association between exposure and outcome• How precise was the estimate of risk
• Application ?• Study patients similar to patients in my practices• Long sufficient follow up ?• Similar exposure with my patients• What is magnitude of risk ?• Are there any benefits that are know to be associated with exposure?
CEB-rama.org
22/04/57
3
Example• Does Soy milk increase the risk of
developing peanut allergy in children ?
CEB-rama.org
Validity• Similar prognosis
– RCT : Limitation or unethical
– Observational Studies• Cohort study
– Similar prognosis factors ?– Method to detect outcome similar ?– Sufficient of follow up ?
• Case-control study– Case vs. Control have circumstances that would lead to exposure– Same circumstances and methods to determine the exposure
• Adjusted analysis
CEB-rama.org
22/04/57
4
Validity
• Ascertainment of exposure– How to measure the exposure
– Recall bias ?
CEB-rama.org
Validity• Outcome measurement
– The same way
no Accident
no Accident
CEB-rama.org
22/04/57
5
Validity– Follow Up
• Drop-out ?
• Follow up complete ?
no Accident
no Accident
CEB-rama.org
• Similar prognosis
• Exposure ascertainment
• Outcome measurement
• Complete FU
What is the range of validity ? (Scoring in your mind)
0 50 100
0 100%
CEB-rama.org
22/04/57
6
Result
• Magnitude of association
Exposure Outcome
Relative riskOdds ratio
CEB-rama.org
Results
• How precise of the association
Relative riskOdds ratio
1 1 95% CI
P-value 0.001 0.09 0.103
3
CEB-rama.org
22/04/57
7
Application
• Our patients = Patients in this review
• Is FU long enough?
• Magnitude of risk: BIG??? vs. SMALL
• Should I stop the exposure?
CEB-rama.org
Number needed to harm
• NNH:number of patients you need to treat to harm one of them for any odds ratio (OR)
• your patient’s expected event rate for this adverse event if they were not exposed to this treatment (PEER):
• NNH = PEER (OR-1)+1PEER (OR-1) x (1-PEER)
CEB-rama.org
22/04/57
8
NNH table
CEB-rama.org
Should I stop the exposure?
• NNH = 1
• NNH = 50
• NNH = 100
• NNH = 500
ItchingSkin infection
LimpingPain CVADeath
CEB-rama.org
22/04/57
9
Application
• Patients’ preference• Alternative treatment?
CEB-rama.org
Example of ClinicalAppraisal of Risk Study
CEB-rama.org
22/04/57
10
Risk Drugs ( Risk vs Harm)• Glucosamine in OA
• High efficacy of ATB to treatment tonsillitis (Zithromax, Cephalosporin vs. S.pyrogenis)
• ASA/Anticoagulant in CVH
• Bromhexine in Asthma
• Nifedipine vs. Hypertensive crisis
• Cimetidine Rx PU compared with Ranitidine(More drug interaction metformin, Chloroquin)
• Clopidogrel vs low dose ASA rxnoncardioembolic TIA หรือ stroke
CEB-rama.org
Lessons & Learnt form the withdrawal of Rofecoxib
CEB-rama.org
22/04/57
11
Question• 65 years old• Female• OA knee• Received Vioxx
(Rofecoxib) for 2 years• Worry about CVS side-
effect
Would you change analgesic drug?CEB-rama.org
•Forming answerable clinical questions
•Searching for the best evidence answer
•Critical appraisal
3 skills for handling evidence:
CEB-rama.org
22/04/57
12
Categories of Clinical Questions
CEB-rama.org
4 parts of clinical question
• Patient or Problem P
• Intervention or exposure I
• Comparison C
• Outcome O
CEB-rama.org
22/04/57
13
Converting a clinical problem into a clinical question1. Patient
Among OA female patients…
2. InterventionWould treating with Vioxx
3. Comparison interventioncompared with other NSAIDs…
4. Outcome : increased CV events
CEB-rama.org
What EBM?
“Expertise in integrating
1. Best research evidence
2. Clinical Circumstance
3. Patient values
in clinical decisions”
Haynes, Devereaux, & Guyatt, 2002CEB-rama.org
22/04/57
14
CC-EBMMonitor the changeMonitor change
CC-EBM
22/04/57
15
Critical Appraisal of the Critical Appraisal of the EvidenceEvidence
Cardiovascular events associated with Rofecoxib in a colorectal
adenoma chemoprevention trial
CEB-rama.org
CEB-rama.org
22/04/57
16
HarmUsers’ Guides for an article about harm
CEB-rama.org
Are the results valid?Are the results valid?
• Did experimental and control groups begin the study with a similar prognosis?
• Did the investigators demonstrate similarity in all known determinants of outcome; did they adjust for differences in the analysis?
Yes (Table 1)
CEB-rama.org
22/04/57
17
CEB-rama.org
Are the results valid?
• Were exposed patients equally likely to be identified in the two groups?
• Did experimental and control groups retain a similar prognosis after the study started?
Yes (RCT)
CEB-rama.org
22/04/57
18
CEB-rama.org
Are the results valid?
• Were the outcomes measured in the same way in the groups being compared?
Yes
CEB-rama.org
22/04/57
19
CEB-rama.org
Are the results valid?
• Was follow-up sufficiently complete?– Loss to follow-up ?
Yes (complete 3 years of treatment)
CEB-rama.org
22/04/57
20
What is the range of validity ?(in your mind)
0 50 100
CEB-rama.org
What is the range of validity ? (in your mind)
0 50 100
CEB-rama.org
22/04/57
21
What are the results?
• How strong in the association between exposure and outcome? – Hazard Ratio– Relative risk
• How precise is the estimated of the risk?– Narrow 95% confidence interval
CEB-rama.org
CEB-rama.org
22/04/57
22
CEB-rama.org
How can I apply the results to patient care?
• Were the study patients similar to the patient under consideration in my practice?
Racial difference, colorectal adenoma
• Was the duration of follow-up adequate?Yes
• What was the magnitude of the risk? Hazard ratio 2.8, Relative risk 1.9-4.5
• Should I attempt to stop the exposure?Yes
CEB-rama.org
22/04/57
23
Rofecoxib story
CEB-rama.org
Biosynthesis of eicosanoids
CEB-rama.org
22/04/57
24
CC-EBMCEB-rama.org
Rofecoxib development
• 1987 Two enzymes-cyclooxygenaseCOX-1 and COX-2
• 1999 First two drugs were approved by FDA (Food and Drug Administration)
celebrex: CLASSrofecoxib: VIGOR
• 2004 Merck withdrew rofecoxibAPPOVe trial
CEB-rama.org
22/04/57
25
Conceptual work
Preclinical toxicology testing on animals
Investigational new drug application
FDA acceptance of IND
Drug development
CEB-rama.org
FDA acceptance of IND
Phase I Include < 100 people,
to establish safety
Clinical trials
Phase II Include 100-1000 people,
to establish efficacy
Phase III Include >1000 people, to confirm short and
long term efficacy
CEB-rama.org
22/04/57
26
Phase III Include >1000 people, to confirm short and
long term efficacy
New drug application
FDA approval for marketing
Post marketing surveillance
CEB-rama.org
Clinical trial
Phase I Clinical pharmacology /
toxicology, safetysafetysafetysafety, dose fining
Phase II Initial clinical investigation for
treatment effect, pilot studypilot studypilot studypilot study
Phase III Full-scale evaluation of treatment
effect (efficacyefficacyefficacyefficacy)
Phase IV Post-market surveillance
(effectivenesseffectivenesseffectivenesseffectiveness)
CEB-rama.org
22/04/57
27
Efficacy study•Efficacy trial : Does receiving treatment
work under ideal conditions?
•Efficacy is established by restricting patients in a study to those who will cooperate fully with medical advice.
CEB-rama.org
Effectiveness study•Effectiveness trial: Does offering
treatment help under ordinary circumstances?
•The extent to which a specific intervention, procedure, regimen, or service, when deployed in the field.
CEB-rama.org
22/04/57
28
Efficiency
•The effects or end results achieved in relation to the effort expended in terms of money, resources and time
•The measure of the economy (or cost in resources) with which a procedure of known efficacy and effectiveness is carried out
CEB-rama.org
Preclinical development
1,000 chemicals
Phase I
100 chemicals
Phase II
10 drugs
Phase III
1 drugs
Duration (years)
2.5
1.5
2.5
3
Chance of reaching the market (%)
IV80
50
20
10
5
Market approval
CEB-rama.org
22/04/57
29
Evaluation of drug safetyType of Adverse Drug Reaction
Type AExpected/known mechanism, often dose-relatedEg. bradycardia from beta-blocker
CEB-rama.org
Evaluation of drug safetyType of Adverse Drug Reaction
Type AExpected/known mechanism, often dose-relatedEg. bradycardia from beta-blocker
Type BUnexpected/ unclear mechanism, unclear causationEg. allergic reactions
CEB-rama.org
22/04/57
30
Evaluation of drug safetyType of Adverse Drug Reaction
Type AExpected/known mechanism, often dose-relatedEg. bradycardia from beta-blocker
Type BUnexpected/ unclear mechanism, unclear causationEg. allergic reactions
Type CUnexpected/relation detected statisticallyEg. coronary deaths from rofecoxib
CEB-rama.org
Hierarchy of Evidence
• Systematic reviews and meta-analysis• Randomized controlled trial• Cohort study• Case-control study• Cross-sectional study• Case series• Case report
CEB-rama.org
22/04/57
31
Experimental
Observational
RCTQuasi-experimental
Descriptive
Analytic
Case seriesCross-sectional
Cross-sectionalCohortCase-control
Assign exposure
Natural exposureNo comparison gr.
Comparison gr.
CEB-rama.org
CC-EBM
Randomized controlled trials
Populationof patientsWith the Condition Sample
Improved
Not improved
Improved
Not improved
Outcomes
Experimental
Comparison
Random sampling
R
22/04/57
32
Loopholes in ADR evaluation
Phase ADR type Hierarchy of usefulness
Pre-clinic A Animal studies
Phase I A, B Cohort (before-after)
CS
Phase II A, B RCT (Cohort>CC>CS)
Phase III A, B, C RCT/Meta (Cohort>CC>CS)
Phase IV A, B, C CS>Cohort>CC>RCT/Meta
CC=case-control, CS=case seriesCEB-rama.org
CC-EBM
Sample size considerationADR incidence Background incidence n
1/100 0
1/10,000
1/1,000
1/100
360
520
730
2,000
1/1,000 0
1/10,000
1/1,000
1/100
3,600
7,300
20,300
136,400
1/5,000 0
1/10,000
1/1,000
1/100
18,200
67,400
363,000
3,255,000
22/04/57
33
Rofecoxib development
• 1987 Two enzymes-cyclooxygenaseCOX-1 and COX-2
• 1999 First two drugs were approved by FDA (Food and Drug Administration)
celebrex: CLASSrofecoxib: VIGOR
• 2004 Merck withdrew rofecoxibAPPOVe trial
CEB-rama.org
September 30, 2004
Dr Peter Kim, president of Merck Research Laboratories, said that the company was considering withdraw rofecoxib (Vioxx) from the market.
CEB-rama.org
22/04/57
34
CC-EBM
CC-EBM
22/04/57
35
VIGOR studyVioxx Gastrointestinal Outcomes Research
• 8076 patients (over 40 yrs) with RA• Rofecoxib vs. Naproxen • Median F/U: 9 months• Exclude pts with stroke in the last 2
years or MI/CABG in the last year or those who required ASA
CEB-rama.org
Critical appraisal
• Short-term trial• The exclusion of high-risk patients• Methodologic inattention to CVS events
Minimized the possibility of evidence of CVS harm
CEB-rama.org
22/04/57
36
ResultsVIGOR study
• Primary endpoint: complicated GI endpoints (perforation, obstruction, or severe bleeding)0.6/100 pt-yr (ROf) vs. 1.4/100 pt-yr (Nap)
• MI0.53/100 pt-yr (ROf) vs. 0.13/100 pt-yr (Nap)
CEB-rama.org
CC-EBM
VIGOR studyNEJM 2000;343:1520-28
22/04/57
37
Treatment effect
• Attributable risk reduction (ARR)Ic-IRx
• Number needed to treat (NNT)1/ARR
• Number needed to harm (NNH)1/IRx- Ic
CEB-rama.org
The power of EBM
• NNT = 1/0.014-0.006 = 125• NNH = 1/0.053-0.0013 = 250
CEB-rama.org
22/04/57
38
The power of EBM
• NNT = 1/0.014-0.006 = 125• NNH = 1/0.053-0.0013 = 250
For every 250 pts treated , 2 severe GI events are avoided but 1 MI is caused
CEB-rama.org
The power of EBM
• Lacked of placebo group• An increased cardiovascular risk with
rofecoxib?• A protective effect of naproxen?
CEB-rama.org
22/04/57
39
Why?
• 11 of the 13 principal authors have had financial associations with Merck.
CEB-rama.org
What they do next?
• Should be the study that address primary question of cardiovascular toxicity
CEB-rama.org
22/04/57
40
What they do next?
• Should be the study that address primary question of cardiovascular toxicity
• Instead, the efficacy trials designed to investigate the prevention of recurrent colonic polyps was launched, with monitoring of CVS events of safety
CEB-rama.org
Observational study
• What if, a large case-control study conducted to test hypothesis whether or not the increase was from rofecoxib rather than waiting the results of RCT not designed to find the adverse effect.
• Sooner• Less cost• Not need large sample as in RCT
CEB-rama.org
22/04/57
41
Can we trust meta-analysis?
CEB-rama.org
Fallacy perpetuatedMeta-analysis
Konstam MA et al. Circ 2001;104:2280CEB-rama.org
22/04/57
42
Fallacy perpetuatedMeta-analysis
• Conclusions— This analysis provides no evidence for an excess of CV events for rofecoxib relative to either placebo or the non-naproxen NSAIDs that were studied. Differences observed between rofecoxib and naproxen are likely the result of the antiplatelet effects of the latter agent.
Konstam MA et al. Circ 2001;104:2280CEB-rama.org
Fallacy perpetuatedMeta-analysis
• 5 of 7 authors were Merck employees• Pooled composite endpoint of all-cause
mortality, non-fatal MI, and non-fatal stroke
• Trials included were all Merck sponsored studies and were weighted with unpublished in-house phase II/III studies
CEB-rama.org
22/04/57
43
Independent meta-analysis
• Lancet 2004;364:2021• Cumulative meta-analysis• Data from
- Cochrane- MEDLINE- EMBASE- CINAHL
CEB-rama.org
CC-EBM
22/04/57
44
CC-EBM
Meta-analysis of RCT comparing rofecoxib with control
CC-EBM
Cumulative meta-analysis of RCT
comparing rofecoxib with control
22/04/57
45
Mechanism
• COX-2 inhibitor- Lack of antiplatelet effect (inhibiting
production of prostacyclin)- Without changing the production of
the prothrombotic product, thromboxane.
CEB-rama.org
Government ineptitude
• FDA approves drugs on the proviso that companies collect, and report all ADRs.
• Companies often promise post-marketing clinical trials but in practice, more than half are never undertaken.
• FDA under pressure to speed up approvals but no money for surveillance or analysis.
CEB-rama.org
22/04/57
46
Lessons1. Pharmaceutical companies cannot be trusted2. Earlier critical appraisal might have helped3. No substitute for independent external review4. RCTs are not the best way to detect harm5. Observational studies should not be forgotten6. Healthy skepticism is good
CEB-rama.org
Dr Graham, associate director in the FDA's Office of Drug Safety, said an estimated 88 000 to 139 000 Americans had heart attacks and strokes as a result of taking rofecoxib.
The number, he said, far exceeds earlier disasters such as the 5000 to 10000 children born in the 1960s with birth defects related to thalidomide
CEB-rama.org
22/04/57
47
Do no harm
CEB-rama.org
CC-EBM
Why Evidence-Based Medicine Practice?
22/04/57
48
Systematic search for best evidence
Clinical Question
Assessment of validity
Assessment of applicability
Clinical circumstance and patient values
Decision
EBM AS A CYCLE
CEB-rama.org
What EBM is not:
• Cookbook medicine• Overrules experience/expertise• Always about RCT’s• Always cost-minimizing
CEB-rama.org
22/04/57
49
Section for Clinical Epidemiology and Biostatistics
Thank you
CEB-rama.org