Overall and subgroup analysis• If the OVERALL results show highly significant

evidence of a worthwhile effect of treatment, but a few subgroups of the overview unexpectedly indicate no benefit (which could well happen by chance), then the appropriate question is whether there is good evidence that this life-saving treatment should be denied to these patients.

• REVERSAL of the usual demand that there should be proof of worthwhile benefit.

Courtesy of Dr. K. Wheatley

Meta-analysis vs. randomized controlled trials: internal validity vs. generalizibility

• Have complimentary roles– RCT, large adequately powered

• If our desire is to assess the efficacy of treatment (i.e. understand a measure of benefit of the treatment under ideal conditions of a clinical trial using narrow defined eligibility criteria)

– Meta-analysis (of totality of evidence)• If our goal is to obtain reliable estimate about the

treatment effectiveness (i.e. understand the extent to which a given treatment can produce a beneficial effect under variety of circumstances and eligibility criteria)

Meta-analysis vs. randomized controlled trials

Small CTs To study mechanisms

Meta-analyses of small RCTs

To generate hypotheses for more reliable RCTs

Large RCts

Meta-analyses of large RCTs

To obtain reliable overall answers under specificconditions of a trial

To obtain a typical and unbiased and generalizible estimate of treatment effect and to explore interactionsamong subgroups

Literature-based vs. individual patient data meta-analysis?

• IPD MA gold-standard

• LMA may be misleading– Data extraction, patient exclusion,

length of follow-up, method analysis may be less accurate in LMA

Lancet 1993;341:418-22; Stat Med 1998;14:2057-2079

IV Ethical obligations to account of what’s already known

• To avoid unnecessary trials if reliable knowledge already exists

• Conversely, to determine if there is true uncertainty about relative values of competing treatment alternatives – A new trial should be conducted if there is a substantial

uncertainty which of the trial treatments would benefit the patient better

• Requirement that equipoise (uncertainty principle) is met

Ethical obligation of building systematically on what is already known

• Clinical trials should be preceded by a systematic review and should be reported with a discussion of assessing the trial’s results in the context what is already known– Ethical requirement for updating systematic

reviews

• UK, Denmark, Holland now mandates search or conduct of SR before a new clinical trial is done

JAMA 1998;280:280-282;Lancet 2001:358:1648

V Knowledge resources

“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials.”

Archie Cochrane

Cochrane Database of Systematic Reviews -

The Cochrane Collaboration - an international network of individuals and institutions committed to preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of health care interventions.

Cochrane Systematic Reviews (2,796) (January 2003)

Database of Abstracts of Reviews of Effectiveness (3,875)

Registry of Randomized Controlled Trials (353,809)

How many systematic reviews are needed to “cover” whole medicine?

• 10,000 systematic reviews to provide broad coverage of most health care topics

Clarke M, personal communication

Cochrane Centres

South African

Australasian

Chinese

Brazilian

Nordic

German

San Antonio

ItalianIberoamericanFrenchDutch

UK

Canadian

New England

San Francisco

Cochrane Systematic reviews

• Cochrane reviews have been shown to be methodologically superior to non-Cochrane systematic reviews

BMJ 2000;320:537-40, JAMA 1998;280:278-80

Cochrane Cancer Network with Update Software Ltd

The Cancer Library

Courtesy of Dr. Chris Williams

Meta-analyses in radiation oncology

• 100 meta-analyses in the Cochrane Database of Systematic Reviews– 22 Cochrane Reviews

– 78 DARE reviews

• MEDLINE (Clinical Queries) search– 616 systematic reviews

Meta-analyses in radiation oncology: an example of reliable review with long-term

(20 years) follow-up

• Favourable and unfavourable effects on long-term survival of radiotherapy for early breast cancer: an overview of the randomised trials

Early Breast Cancer Trialists' Collaborative Group*

Lancet 2000; 355: 1757 70 (20 May 2000 )

Proportional effects on all-cause mortality

in 40 trials of radiotherapy

Lancet 2000; 355: 1757 70 (20 May 2000 )

Absolute effects of radiotherapy

on cause-specific survival

Absolute benefits and hazards

Part VIEvidence and decision-making

Clinical Decision Making

Evidence from research

Patient circumstances

Preferences,values andrights

Courtesy of Dr. G. Lyman

Reporting data on benefits and harms

• If evidence on benefits and harms are not reported or is of poor quality, one has to wonder how physicians make decisions and recommendations for their patients

Eddy D. JAMA 1990;264:1737-39

Reporting data on benefits and harms: RCTs in myeloma

• Survival outcomes

111/136 (82%)• Survival beyond 5 years

15/111 (14%)• Treatment-related mortality

33/136 (24%)• Non-fatal adverse events

91/136 (67%)

Annals Oncol 2001;12:1611-1617

Reporting harms in RTOG randomized trials

86.4%91.5%

74.6%

102030405060708090

100

Overall survival Toxicity reported treatment-relatedmortality

OutcomesTotal number of studies: 59

Po

rce

nta

ge

of

stu

die

s

N= 51 N = 54 N = 44

HOW TO INTEGRATE BENEFITS AND RISKS OF AVAILABLE

THERAPEUTIC OPTIONS

• Should we always use the option with the best benefit/risk ratio?

Efficacy=80% Toxicity=10% E/R=8

Efficacy=20% Toxicity=1% E/R=20

Decision-making at the bedside

• Minimal conditions for treatment benefit at which therapy is worth considering is met when– Absolute benefits>absolute harms (adjusted for

the probability of bad event, e.g. relapse)

• Never administer treatment or order diagnostic test if treatment harm is greater than its efficacy

Integrating benefits and harms of radiation therapy of breast cancer

• Threshold for administering radiation therapy (RT):

probability of breast cancer recurrence (without RT)>

Deaths due to (RT) (%)

4.3%

51.4-46.6 (=4.8%) = 89.6% (actual relapse=30.1%)

Deaths due to breast cancer without RT- deaths due to breast cancer on RT