Searching for Evidence

Task 1

Find a full text document from a referenceThe ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group . Major outcomes in moderately hypercholesterolemic, hypertensive patients randomized to pravastatin vs usual care: the antihypertensive and lipid- lowering treatment to prevent heart attack trial (ALLHAT-LLT). JAMA. 2002;288:2998-3007.

Task 2

Find the best evidence concerning the use of aspirin for the primary prevention of cardiovascular disease in people with diabetes

ATHENS ACCOUNT

• To get a NHS Athens account, username and password. Register for free at: https://register.athensams.net/nhs/nhseng/

Task 3 NNT - Revision

From the diagram on the next slideCalculate:

•Relative Risk•Absolute Risk Reduction•Numbers Needed to Treat

Task 4 NNT Calculation

In the Statins Metaanalysis over a mean 4.3 years:Baseline risk of a major coronary event = 5.7%Relative Risk Reduction is 29.2%Calculate Absolute Risk Reduction andNumbers Needed to Treat (for 4.3 years)

Task 5

From the NNT found in the previous taskCalculate the drug cost to prevent one coronary

event using Simvastatin (£1.37 for 28 tablets)

Task 6

The JUPITER trial of Rosuvastatin 20mg vs placebo (in 17802 healthy men & women LDL<3.4 mmol/l, high sensitivtiy CRP 2.0+ mg/l)

reported:0.77 vs 1.36 events per 100 person-years HR 0.56 (0.46-0.69) p<0.00001Rosuvastatin 20mg costs £26.02 for 28 tabs What was the drug cost to prevent 1 event in this trial?

Systematic Reviews & Guidelines

Chris Lewis Apr 2009

Types of “paper” research evidence• Primary studies

– Case studies– Experiments– Surveys– Clinical Trials

• Secondary studies– Non-systematic reviews– Systematic reviews

• Meta-analyses • Guidelines • Decision analyses

• Economic analyses

Advantages DisadvantagesEvidence-based Guideline

Summarises all relevant research about all possible interventions for a clinical problem. Explores benefits and harms.

May become out-of-date quickly.Expert opinion often fills gaps in evidence.

Systematic Review Summarises all research about an intervention.

Usually only one of several possible interventions is considered. May not explore benefits vs harms.

Primary Study Very specific information Not comprehensive

Types of evidence

Systematic Review• A systematic review is a literature review • focused on a single question • that tries to identify, appraise, select and

synthesize • all high quality research evidence relevant to that

question.

Meta-analysis

A systematic review in which the data from the primary studies is sufficiently similar that they can be analyzed as if they were the same trial.

They provide higher statistical power to detect an effect and a more powerful estimate of true effect size than individual studies.

How to conduct a systematic review

FAST critical appraisal

4 FAST Questions for appraising systematic reviews:• Finding: Did they find all relevant studies?• Appraisal: Did they select good studies?• Synthesis: When the studies are put together

what do they mean?• Transferability: Are the conclusions applicable

to my patient?

Critical Appraisal Q1 – Formulate Question

Did the study address a focused clinical question?

• Population• Intervention • Comparator• Outcome

Should be found in the title, abstract or introduction

Critical Appraisal Q2 – Find all relevant studies

Were the inclusion/exclusion criteria appropriate?

• Clearly defined before the literature search• Should specify PICO• May specify type of study (eg RCT)

Should be found in the methods section

Critical Appraisal Q3 – Find all relevant studies

Is it unlikely that important relevant studies were missed?

Search strategy should include:• Major databases: Medline, EMBASE, Cochrane• Reference lists from relevant studies• Inquiry into unpublished studies• Foreign language publications• MESH terms and text words

Methods section should describe search strategyResults section should state number of studies retrieved

and number of studies excluded with reasons

Copyright restrictions may apply.

Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313.

Literature review flowchart

Publication Bias

• ‘Negative’ studies less likely to be published than ‘ positive’ studies

• In a follow-up of 737 studies submitted to the ethics committee at the Johns Hopkins hospital positive studies were 2.5 times more likely to be SUBMITTED than negative studies (Dickersin, JAMA, 1992)

• All trials registered at inception• Unethical not to make results available

Critical Appraisal Q4 – Assess studies

Were the included studies valid?• Should describe how quality of included studies was

assessed• Criteria for quality assessment should be pre-

determined• Criteria for quality assessment should be appropriate

to the question asked

Methods section should describe assessment process and quality criteria

Results section should give information on quality of the individual studies

Critical Appraisal Q5 – Assess studies

Were assessments of studies reproducible?

• Assessed independently by 2+ reviewers• Level of agreement between reviewers• Procedure for dealing with disagreement

Methods should describe how and by who assessments were done

Results should show level of agreements

Critical Appraisal Q6 – Analysis/Results

Were the results similar from study to study?

• Ideally results should be similar (homogeneous)

• Significance of heterogeneity may be assessed• Reasons for heterogeneity should be explored

Results section.Forest plot should show chi2 test for heterogeneity

Meta-analysis (Forest) plot

1. How many studies are there?2. How many studies favour treatment?3. How many studies are statistically

significant?4. Which is the largest study?5. Which is the smallest study?6. What is the combined result?

Assessing heterogeneity

“Eyeball” test: In the Forest plot a vertical line running through the combined OR should cross the horizontal lines (95% CI) of all the individual studies

Assessing heterogeneity

Chi2 test or Cochran Q:• If chi2 is statistically significant (p<0.1) there is

definite heterogeneity• If chi2 not statistically significant (p>0.1) and

Q/df >1 there is possible heterogeneity• If chi2 not statistically significant (p>0.1) and

Q/df >1 heterogeneity is very unlikely

Explaining heterogeneity

• Variation in population studied• Variation in intervention• Variation in outcome measures• Variation in study methods

Copyright restrictions may apply.

Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313.

Plotted relative risk ratios (RRs) (95% confidence intervals [CIs]) for major coronary events

Copyright restrictions may apply.

Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313.

Plotted relative risk ratios (RRs) (95% confidence intervals [CIs]) for major cerebrovascular events

Critical appraisal – Question 7

Are conclusions supported by data?

Biased interpretation or emphasis of results is surprisingly common

Best to form your own conclusions from the data before reading authors conclusions

Then consider the reasons for any differences between your conclusions and the authors’

Critical Appraisal Q8

Are the results clinically relevant?

Assuming we have a statistically significant benefit from treatment:

• Is the size of the benefit worthwhile?• Are the results applicable to my patient?

Funding Bias?

• Funding / employment of authors• Drug company sponsorship

Results

Statins reduce risks by:• Coronary events 29.2% (16.7-39.8%) p<.001• Cerebrovascular events 14.4% (2.8-24.6%)

p0.02• Revascularisations 33.8% (19.6-45.5%) p<.001• Cardiovascular death 22.6% (0.56-1.08) p.13• All deaths 0.92(0.84-1.01) p.09

Inclusion Criteria / Baseline characteristics

Trial Age Mean age

Sex% male

GP / Hosp Chol mmol/l Other

WOSCOPS 45 - 64 55 100 GP (W.Scotland)

F LDL >4.5 <6.0

CVD 16%Smoke 44%

AFCAPS/TexCAPS

M45-73F55-73

58 85 Texas?Hosp

TC 4.65-6.82Trig<=4.52

102800 screened6600 randomised

PROSPER 70-82 75 42 ?Hosp (Scot/Ire/Nethrlnds)

TC 4.0-9.0 Smo/HT/Diab

ALLHAT-LLT

55+ 66 51 GP(N.America)

LDL >=3.1 <=4.9 Trig <3.95

Hypertension+1 other RF

ASCOT-LLA

40 - 79 63 81 GP (UK/Scand)

TC <=6.5 Hypertension+3 other RF’s

HPS 40 - 80 NA NA Hosp (UK) F TC >=3.5 Diabetes

CARDS 40 - 75 61 68 GP & Hosp (UK/Ireland)

LDL <=4.14 F Trig <=6.78

Diabetes

CARDS required at least 1 of retinopathy, albuminuria, smoking, hypertension

Influence of DiabetesTrial RR Major Coronary Events % Diabetic

WOSCOPS 1995 0.70 (0.58-0.85) 1.0

AFCAPS/TexCAPS 1998 0.60 (0.43-0.83) 3.8

PROSPER 2002 0.91 (0.71-1.15) 12.2

ALLHAT-LLT 2002 0.91 (0.79-1.04) 34.4

ASCOT-LLA 2003 0.65 (0.50-0.83) 24.3

HPS 2003 0.57 (0.41-0.79) 100

CARDS 2004 0.53 (0.35-0.82) 100

Systematic Reviews

Advantages:• Larger numbers with greater statistical power• Robustness across differing trial populations,

drugs within a classDisadvantages:• Small, but consistent, biases may produce an

invalid conclusion of real effect

Resources

• Oxford Centre for Evidence-based Medicine http://www.cebm.net/

• The Cochrane Collaboration http://www.cochrane.org/

Formula for caluclating NNT from OR if Patient Expected Event Rate is known

GUIDELINES

The AGREE InstrumentAppraisal of Guidelines for REsearch & EvaluationThe AGREE Collaboration – June 2001www.agreecollaboration.org

The AGREE Instrument

23 items organised in six domainsEach item is rated on a 4-point score scale1. Strongly disagree2. Disagree3. Agree4. Strongly agree

Overall assessment• Strongly recommend• Recommend (with provisos/alterations)• Would not recommend• Unsure

Domain 1: Scope & Purpose

1. Overall objective(s)2. Clinical question(s)3. Target population

should all be specifically described

Domain 2: Stakeholder Involvement

4. Guideline development group includes individuals from all relevant professional groups

5. Patients’ views & preferences sought6. Target users clearly defined7. Piloted by end users

Domain 3: Rigour of development

8. Systematic search for evidence9. Criteria for selecting evidence described10.Methods for formulating recommendations

described11.Health benefits, unwanted effects and risks

considered in formulating recommendations12.Explicit link between recommendations and

evidence13.External review by experts before publication14.Procedure for update provided

Domain 4: Clarity & Presentation

15.Recommendations specific & unambiguous16.Management options clearly presented17.Key recommendations easily identifiable18.Guideline supported by tools for application

Domain 5: Applicability

19.Potential organisational barriers in applying the recommendations discussed

20.Cost implications considered21.Key review criteria for monitoring/audit

presented

Domain 6: Editorial Independence

22.Editorially independent from funding body23.Conflicts of interest of guideline

development members recorded