1

2

Levels of evidence

Ia Meta-analysis or systematic review of RCTs

Ib Randomised trial

IIa Controlled non-randomised study

IIb Cohort study

III Case-control study

IV Descriptive study

Non-systematic review

Consensus report

Leading article

3

Survey of 85 physicians (36% from internal medicine)”Very important” in influencing my prescribing

Own training and experience 88%

Scientific papers 62%

Advice from colleagues 48%

Detail men 20%

Drug adds 4%

Patient preference 2%

(Avorn J. Am J Med 1982;73:4-8)

4

”Is impaired cerebral blood flow a major cause of senile dementia?”

Yes 71%

No 14%

No opinion 15%

32% found cerebral vasodilators useful in managing confused geriatric patients

5

Survey of 85 physicians (36% from internal medicine)”Very important” in influencing my prescribing

Own training and experience 88%

Scientific papers 62%

Advice from colleagues 48%

Detail men 20%

Drug adds 4%

Patient preference 2%

(Avorn J. Am J Med 1982;73:4-8)

6

Drug adds

287 advertisements for anti-hypertensive or lipid-lowering drugs

- 125 promotional claims with references

- 23 refs. unretrievable (data on file, monographs)

- 45 of 102 claims not supported by reference

(Lancet 2003;361:27)

7

Randomised trials

Unclear randomisation method:

- effect exaggerated by 30%, on average

No blinding:

- effect exaggerated by 14%, on average

(BMJ 2001;323:42-6)

8

Outcome reporting bias

102 RCTs approved by the Copenhagen & Frederiksberg Ethics Review Committee 1994-95and subsequently published

Incompletely reported outcomes for meta-analysis:

50% for efficacy, 65% for safety

9

Outcome reporting bias

Unreported outcomes 86% of trialists denied unreported outcomes despite evidence in publications & protocols

Only half of the trialists responded to the question

10

Outcome reporting bias

Full outcome reporting is associated with p<0.05 Odds ratio 2.4 (1.4 - 4.0) for efficacyOdds ratio 4.7 (1.8 - 12) for safety

Are primary outcomes consistent between protocols and publications?

Discrepancy in primary outcomesProportion (%) of trialswith inconsistencies

Changes to protocol-defined outcome 53% (40/76)

New publication-defined outcome 33% (21/63)

Change in power calculation outcome 29% (10/38)

ANY INCONSISTENCY 63% (52/82)

Conclusions

Trial outcomes are often inadequately reported for inclusion in meta-analyses

Reporting of outcomes is biased to favour p<0.05

Primary outcomes are omitted, changed, or newly-introduced in over 60% of trials

Protocols should be publicly available

13

14

‘Positive’ studies are more likely to be published than ‘negative’ studies

Hazard ratio for 130 clinical trials

‘positive’ (P<0.05) vs ‘negative’ (P>0.10)

3.13 (1.76 to 5.58), P=0.0001 Median time to publication 4.7 vs 8 years

15

Cochrane Reviews

Freely available from

www.cochrane.dk

16

Based on Cochrane Reviews

when possible

17

NSAIDs

“Systematic reviews of RCTs have found no important differences in effect between different NSAIDs or doses but have found differences in toxicity related to increased doses and possibly to the nature of the NSAID itself.”

“The only meta-analysis that found one drug to be more effective than another was funded by the manufacturer”

Clinical Evidence 1999;2

18

Clinical Evidence(NSAIDs)

We have favoured systematic reviews that have not been sponsored or authored by industry ...

it is easy to seemingly follow the rules for systematic reviews and yet adopt inclusion and exclusion criteria that omit inconvenient studies. In fact, it is hard to find a systematic review sponsored by, or co-authored by, industry that concludes that the company’s product is not better than those of its competitors.

19

Celecoxib

Conclusion, industry supported meta-analysisCelecoxib ... has significantly improved gastrointestinal safety and tolerability (BMJ Sept 2002)

Conclusion, Cochrane ReviewFor an individual with RA the potential benefits of celecoxib need to be balanced against the uncertainty that the short-term reduced incidence of upper GI complications are maintained in the long-term and its increased cost in comparison to traditional NSAIDs.

20

Industrisponsorerede forsøg med lægemidler

Ofte problemer med:• Design• Data-analyse• Afrapportering• Konklusion

21

Forsøg med psykofarmaka

Gamle præparater:

- Alt for høj dosis, alt for hurtig dosisøgning.

Nye præparater tilsyneladende lige så gode,

med langt færre bivirkninger.

Men: udbredt manipulation med dosis, data-analyse og afrapportering.

(J Nerv Ment Dis 2002;190:583)

(BMJ 2003;326:1171)

22

Behandling for skizofreni

Olanzapin 17.000 kr/år

Haloperidol 2.400 kr/år

Forbrug i primærsektoren

Olanzapin 216 mio kr i 2002

Haloperidol 4 mio kr i 2002

”the new drugs have no unequivocal advantages for first line use” (BMJ 2000;321:1371)

23

Industrisponsoreret forskning versus offentligt sponsoreret forskning

Chancen for et positivt udfald 4 gange så stor for forsøg eller systematiske oversigter over flere forsøg.

(BMJ 2003;326:1167)

Chancen for en positiv konklusion 5 gange så stor for forsøg, trods samme effekt.

(JAMA 2003;290:921)

24

Cochrane Reviews

25

Hvilken forskning mangler vi?

- Sammenligninger med gamle, billige præparater, og på en fair måde (offentlig finansiering)

- Sammenligninger med andet end lægemidler

Forebyggelse af ikke-insulinkrævende sukkersyge:

- metformin: 31% effekt

- motion og vægttab: 56% effekt(N Engl J Med 2002;346:393)

- Forskning i skadevirkninger

26