evidence-based medicine and critical appraisal ben rehman director london medicines information...

Evidence-Based Medicine and Evidence-Based Medicine and Critical AppraisalCritical Appraisal

Ben RehmanBen RehmanDirectorDirector

London Medicines Information ServiceLondon Medicines Information ServiceNorthwick Park HospitalNorthwick Park Hospital

IntroductionIntroduction Evidence-Based MedicineEvidence-Based Medicine

• What is it? What is it? • Why do we need it?Why do we need it?• How do we do it?How do we do it?

Critical appraisal—RCTsCritical appraisal—RCTs Brief introduction to other study designs including Brief introduction to other study designs including

types of pharmacoeconomic studiestypes of pharmacoeconomic studies

• How should we use it?How should we use it? At patient level?At patient level? As part of broader decision making and resource As part of broader decision making and resource

optimisation? optimisation?

Example critical appraisal workshopExample critical appraisal workshop

Evidence-Based Medicine—Evidence-Based Medicine—What is it?What is it?

the conscientious, systematic, explicit, and the conscientious, systematic, explicit, and judicious use of current best evidence in judicious use of current best evidence in patient carepatient care

best research evidence replaces or best research evidence replaces or supports current practice supports current practice

integration of evidence and patient or integration of evidence and patient or population values population values

optimising health gainoptimising health gain• clinical evidenceclinical evidence• pharmacoeconomicspharmacoeconomics

Evidence-Based Medicine—Evidence-Based Medicine—Why do we need it?Why do we need it?

Health professionals face problems Health professionals face problems ensuring best possible patient care:ensuring best possible patient care:• information overloadinformation overload• inadequate traditional sourcesinadequate traditional sources• opinion based approach flawedopinion based approach flawed• disparity between experience and up-to-date disparity between experience and up-to-date

knowledgeknowledge Healthcare systems have finite resourcesHealthcare systems have finite resources But… it’s not a complete panacea!But… it’s not a complete panacea!

Evidence-Based Medicine—Evidence-Based Medicine—How do we do it?How do we do it?

1.1. Form an answerable questionForm an answerable question

2.2. Obtain current best evidenceObtain current best evidence

3.3. Critically appraise the evidenceCritically appraise the evidence

4.4. Integrate appraised evidence with Integrate appraised evidence with clinical expertise and patient valuesclinical expertise and patient values

5.5. Evaluate our successes and failuresEvaluate our successes and failures

Evidence-Based Medicine—Evidence-Based Medicine—How do we do it?How do we do it?

Formulate question

Integrate evidence and values—implement

changes

Evaluate successes and failures—audit

Track down best evidence

Critically review quality

How do we do EBM?How do we do EBM?

1.1. Form an answerable questionForm an answerable question Should contain:Should contain:

• Intervention you are interested inIntervention you are interested in• Patient or population you are interested Patient or population you are interested

in (clearly for drug interventions you in (clearly for drug interventions you require the indication here too)require the indication here too)

• Outcomes you are interested inOutcomes you are interested in So for the example paper, our So for the example paper, our

original question might have been:original question might have been:• Is adjunctive risperidone effective in Is adjunctive risperidone effective in

patients with frequently relapsing patients with frequently relapsing bipolar I disorder?bipolar I disorder?


2.2. Obtain current best evidenceObtain current best evidence Group discussionGroup discussion

• What are good sources of evidence?What are good sources of evidence? What have you used in the past and found What have you used in the past and found

useful?useful?

• What makes them good?What makes them good? What criteria do you apply?What criteria do you apply?

• How can we find good evidence?How can we find good evidence? What portals and sources do we have What portals and sources do we have

available?available?


2.2. Obtain current best evidenceObtain current best evidence Discussion point: good sources of Discussion point: good sources of

evidence for EBMevidence for EBM


2.2. Obtain current best evidenceObtain current best evidence An appropriate search strategy is An appropriate search strategy is

importantimportant We should be aware of the hierarchy We should be aware of the hierarchy

of evidence:of evidence:• Systematic reviewsSystematic reviews• Meta-analysesMeta-analyses• Randomised-controlled trialsRandomised-controlled trials• Cohort studiesCohort studies


2.2. Obtain current best evidenceObtain current best evidence

Its a guide—it’s not Its a guide—it’s not necessarily this necessarily this simple! Consider:simple! Consider:• A large RCT vs. a A large RCT vs. a

meta-analysis of small meta-analysis of small RCTs?RCTs?

• A meta-analysis of A meta-analysis of observational studies?observational studies?

Hierarchy becomes Hierarchy becomes complicatedcomplicated

The hierarchy of The hierarchy of evidence:evidence:


3.3. Critical appraisalCritical appraisal Consider the following conceptsConsider the following concepts

• Internal validityInternal validity Does it prove what it set out to prove?Does it prove what it set out to prove?

• External validityExternal validity Are the results true in the wider world?Are the results true in the wider world?

• BiasBias Is there systematic error?Is there systematic error?

PopulationSample

Outcome

Outcome

Experimental intervention

Control intervention

Randomisation

Time

PopulationSample

Outcome

Outcome

Experimental intervention

Control intervention

Randomisation

Time


3.3. Critical appraisal—placebo Critical appraisal—placebo controlled RCTs controlled RCTs


3.3. Critical appraisal—placebo Critical appraisal—placebo controlled RCTscontrolled RCTs

Are this study’s results valid?Are this study’s results valid?• Did the trial address a clearly focused issue?Did the trial address a clearly focused issue?

Consider the intervention and the outcomes Consider the intervention and the outcomes consideredconsidered

• Is an RCT an appropriate method to answer Is an RCT an appropriate method to answer this question?this question?

• Was the assignment of patients to treatment Was the assignment of patients to treatment randomised?randomised?

Why is it important?Why is it important? Acceptable vs dubious methods?Acceptable vs dubious methods? Is the process well described?Is the process well described? Was the process concealed?Was the process concealed?



Are this study’s results valid?Are this study’s results valid?• Were the groups similar at the start of the Were the groups similar at the start of the

trial?trial? Could any differences have affected outcome?Could any differences have affected outcome? Was any method used to balance randomisation Was any method used to balance randomisation

(stratification)?(stratification)?• Was follow-up sufficient?Was follow-up sufficient?

LengthLength CompletenessCompleteness

• Was there sufficient power?Was there sufficient power? Were there enough participants to minimise the play Were there enough participants to minimise the play

of chance?of chance?• Was an intention-to-treat analysis performed?Was an intention-to-treat analysis performed?



Are the results of this individual study Are the results of this individual study important for us?important for us?• Define the population studiedDefine the population studied

Inclusion and exclusion criteriaInclusion and exclusion criteria

• EndpointsEndpoints Measurement of outcomeMeasurement of outcome

• Clinical relevanceClinical relevance• Surrogate and composite markersSurrogate and composite markers• Validity Validity

Primary vs. secondaryPrimary vs. secondary

• Balance of beneficial outcomes against side-Balance of beneficial outcomes against side-effectseffects

Are the results of this individual study Are the results of this individual study important for us?important for us?• What is the magnitude of the treatment effect?What is the magnitude of the treatment effect?

How is it described?How is it described?• Proportions of people, a measurement (mean or Proportions of people, a measurement (mean or

median differences), a survival curve?median differences), a survival curve? How is it expressed?How is it expressed?

• With proportions see terms like relative risk reduction, With proportions see terms like relative risk reduction, absolute risk reduction, number needed to treatabsolute risk reduction, number needed to treat

• Is what is being expressed clear?Is what is being expressed clear?• Is it clinically significant?Is it clinically significant?





Definitions for measures of Definitions for measures of association and effectivenessassociation and effectiveness• Control event rateControl event rate

the rate at which an outcome occurs in the control the rate at which an outcome occurs in the control populationpopulation

• Experimental event rateExperimental event rate the rate at which an outcome occurs in the the rate at which an outcome occurs in the

experimental populationexperimental population

Definitions for measures of association Definitions for measures of association and effectivenessand effectiveness• Absolute risk (AR)Absolute risk (AR)

Probability an individual will experience a specified Probability an individual will experience a specified outcome during a specified periodoutcome during a specified period

Range of 0 to 1, or expressed as a percentageRange of 0 to 1, or expressed as a percentage

• Relative risk (RR)Relative risk (RR) How many times more likely (RR > 1) or less likely (RR < 1) How many times more likely (RR > 1) or less likely (RR < 1)

an event is to happen in one group compared with another an event is to happen in one group compared with another

• Absolute risk reduction (ARR)Absolute risk reduction (ARR) Absolute difference in risk between experimental and Absolute difference in risk between experimental and

control groups control groups

• Relative risk reduction (RRR)Relative risk reduction (RRR) Proportional reduction in risk between experimental and Proportional reduction in risk between experimental and

control participants in a trial control participants in a trial



Definitions for measures of Definitions for measures of association and effectivenessassociation and effectiveness• Number needed to treat (NNT)Number needed to treat (NNT)

A measure of treatment effectiveness A measure of treatment effectiveness Measures the people who need to be treated with an Measures the people who need to be treated with an

intervention over a period of time to prevent an intervention over a period of time to prevent an additional adverse outcome or achieve an additional additional adverse outcome or achieve an additional beneficial outcome beneficial outcome

Reciprocal of ARR Reciprocal of ARR



Definitions for measures of association Definitions for measures of association and effectivenessand effectiveness• Odds ratio (OR)Odds ratio (OR)

Measure of treatment effectiveness—likelihood of event Measure of treatment effectiveness—likelihood of event happening in experimental group vs. control grouphappening in experimental group vs. control group

Effects being measured may be adverse (e.g. death or Effects being measured may be adverse (e.g. death or disability) or desirable (e.g. survival) disability) or desirable (e.g. survival)

If events rare OR analogous to relative risk (RR); as If events rare OR analogous to relative risk (RR); as event rates increase the OR and RR divergeevent rates increase the OR and RR diverge

• When interpreting remember:When interpreting remember: Closer OR is to 1 Closer OR is to 1 smaller the difference in effect smaller the difference in effect

between experimental intervention and control between experimental intervention and control interventionintervention

OR > 1 ≡ effects of treatment more than controlOR > 1 ≡ effects of treatment more than control OR < 1 ≡ effects of treatment less than control OR < 1 ≡ effects of treatment less than control OR 95% CI includes 1 ≡ effect cannot be deemed OR 95% CI includes 1 ≡ effect cannot be deemed

statistically significantstatistically significant





Absolute vs. relative risk reduction as Absolute vs. relative risk reduction as measures—an examplemeasures—an example• Pros and cons of each measurePros and cons of each measure

Clinical significance?Clinical significance? Proportional difference?Proportional difference? Ease of interpretation?Ease of interpretation?



An example—4S study (Scandinavian An example—4S study (Scandinavian Simvastatin Survival Study)Simvastatin Survival Study)11 • PurposePurpose

Evaluate effect of cholesterol lowering with simvastatin Evaluate effect of cholesterol lowering with simvastatin on mortality and morbidity in patients with coronary on mortality and morbidity in patients with coronary heart disease (CHD) heart disease (CHD)

• ParticipantsParticipantsPatients with stable angina or history of myocardial Patients with stable angina or history of myocardial infarction more than 6 months previouslyinfarction more than 6 months previouslySerum cholesterol > 6.2mmol/lSerum cholesterol > 6.2mmol/lExcluded patients with arrhythmias and heart failureExcluded patients with arrhythmias and heart failureRun in of 8 weeks of dietary therapyRun in of 8 weeks of dietary therapyIf cholesterol still raised (5.5-8.0 mmol/L) randomised to If cholesterol still raised (5.5-8.0 mmol/L) randomised to receive simvastatin (20mg esculating to 40mg) or receive simvastatin (20mg esculating to 40mg) or placebo placebo

1.1. Lancet. 1994 Nov 19;344(8934):1383-9Lancet. 1994 Nov 19;344(8934):1383-9

4S study4S study• Randomisation and allocationRandomisation and allocation

4444 patients randomised to double-blind 4444 patients randomised to double-blind treatment with simvastatin or placebotreatment with simvastatin or placebo

2223 patients placebo 2223 patients placebo

2221 were simvastatin2221 were simvastatin

Mean follow-up was 5.4 yearsMean follow-up was 5.4 years



4S study4S study• ResultsResults• All cause mortality: All cause mortality:

256 patients (12%) in placebo group dead vs 182 (8%) 256 patients (12%) in placebo group dead vs 182 (8%) in simvastatin groupin simvastatin group

• Coronary deaths:Coronary deaths:

189 coronary deaths in the placebo group vs. 111 in the 189 coronary deaths in the placebo group vs. 111 in the simvastatin groupsimvastatin group

• Morbidity (coronary event):Morbidity (coronary event):

622 patients (28%) in placebo group vs. 431 (19%) in 622 patients (28%) in placebo group vs. 431 (19%) in the simvastatin group the simvastatin group

• So, on the face of it, the results look extremely So, on the face of it, the results look extremely promising. How do the measures of association we promising. How do the measures of association we discussed earlier play out?discussed earlier play out?





OUTCOME

Dead Alive

Control group (placebo) n = 2223 256 1967

Experimental group (simvastatin) n = 2221 182 2039

Control event rate = 256 = 0.115 (11.5%)2223

Control odds of event = 256 = 0.1301967

Experimental event rate = 182 = 0.082 (8.2%)2221

Experimental odds of event = 182 = 0.0892039

4S study results—all cause mortality4S study results—all cause mortality




Relative risk reduction CER – EER = 0.115 – 0.082 = 0.29 (29%)RRR CER 0.115

Absolute risk reduction CER – EER = 0.115 – 0.082 = 0.033 (3.3%) ARR

Number needed to treat _1_ = _1_ = 30.1NNT ARR 0.033

Relative risk EER = 0.082 = 0.71 RR CER 0.115

Odds ratio = odds of event in experimental group = 0.089 = 0.69OR odds of event in control group 0.130



NNT examples—when should we adopt therapy?NNT examples—when should we adopt therapy?

Streptokinase + aspririn v. placebo (ISIS 2)

prevent 1 death at 5 weeks

20

tPA v. streptokinase (GUSTO trial)

save 1 life with tPA usage

100

Simvastatin v. placebo in IHD (4S study)

prevent 1 event in 5y

15

Treating hypertension in the over-60s

prevent 1 event in 5y

18

Aspirin v. placebo in healthy adults

prevent MI or death in 1 year

500

InterventionIntervention OutcomeOutcome NNNNTT

Discussion point: numbers needed to Discussion point: numbers needed to treattreat• When do you think therapy should be When do you think therapy should be

adopted? adopted? • Clearly there is no one answer to this Clearly there is no one answer to this

but what might be the considerations?but what might be the considerations?



Statistical approaches to uncertaintyStatistical approaches to uncertainty• Why do they exist?Why do they exist?

Cannot include all individuals in a population Cannot include all individuals in a population in trialin trial

Need to quantify uncertaintyNeed to quantify uncertainty

• p values and confidence intervals are p values and confidence intervals are the measures used, but what are they the measures used, but what are they and what are the differences?and what are the differences?



Statistical approaches to uncertaintyStatistical approaches to uncertainty• p valuesp values

assess the significance of the difference between assess the significance of the difference between a sample estimate and a hypothesised valuea sample estimate and a hypothesised value

tell us the probability that an observed effect tell us the probability that an observed effect occurred by chance if in truth there is no effectoccurred by chance if in truth there is no effect

But… doesn’t quantify the size of an effect, nor But… doesn’t quantify the size of an effect, nor the directionthe direction

p<0.05 p<0.05 → commonly reject null hypothesis→ commonly reject null hypothesis ideally p<<<0.05 and trial reports the actual ideally p<<<0.05 and trial reports the actual

valuevalue



Statistical approaches to uncertaintyStatistical approaches to uncertainty• Confidence intervalsConfidence intervals

Range of values around the sample estimate Range of values around the sample estimate (i.e. the value found in the study)(i.e. the value found in the study)

The range has specified probability (usually The range has specified probability (usually 95%) so CI acts as hypothesis test for the 95%) so CI acts as hypothesis test for the rangerange

Should be seen with most measures of Should be seen with most measures of associationassociation



Statistical approaches to uncertainty—Statistical approaches to uncertainty—please remember!please remember!• statistical significance statistical significance ≠≠ clinical significance… clinical significance…

what is actually being measured?what is actually being measured?• sample size is still importantsample size is still important

larger sample = less uncertainty = narrower CI larger sample = less uncertainty = narrower CI and smaller observed effect considered significantand smaller observed effect considered significant

• there can still be errorthere can still be error rejecting null hypothesis when it’s true (type 1)rejecting null hypothesis when it’s true (type 1) Not rejecting the null hypothesis when it’s false Not rejecting the null hypothesis when it’s false

(type 2)(type 2)



Increasingly used by pharma. Increasingly used by pharma. Designed to show a new treatment is Designed to show a new treatment is

not inferior to standard treatment by not inferior to standard treatment by a predefined clinically acceptable a predefined clinically acceptable endpointendpoint

Rely on assumptions that can be Rely on assumptions that can be hard to validatehard to validate


3.3. Critical appraisal—RCTs of Critical appraisal—RCTs of therapeutic equivalencetherapeutic equivalence

Trials based on the concept that:Trials based on the concept that:• new treatment is non-inferiornew treatment is non-inferior• but would exhibit therapeutic efficacy if but would exhibit therapeutic efficacy if

a placebo controlled RCT could/was a placebo controlled RCT could/was performedperformed

• new treatment offers ancillary benefitsnew treatment offers ancillary benefits Note: non-inferior so efficacy Note: non-inferior so efficacy

determined by effect against placebo determined by effect against placebo not comparatornot comparator



Estimation of non-inferiority marginEstimation of non-inferiority margin• Use statistical and clinical reasoning to Use statistical and clinical reasoning to

determine what is non-inferiordetermine what is non-inferior• If there are a variety of placebo If there are a variety of placebo

controlled RCTs this will be similar to a controlled RCTs this will be similar to a meta-analysis meta-analysis

Non-inferiority margin determined as Non-inferiority margin determined as a fraction (f) and specifies an a fraction (f) and specifies an acceptable magnitude for treatment acceptable magnitude for treatment effect that must be preservedeffect that must be preserved



What to look forWhat to look for• These trials should be designed in a manner These trials should be designed in a manner

consistent with the historical placebo-controlled consistent with the historical placebo-controlled trialstrials

• The active comparator should be well established The active comparator should be well established with predictable quantifiable, and consistent effectswith predictable quantifiable, and consistent effects

• What constitutes non-inferiority should be What constitutes non-inferiority should be determined prior to initiation of trialdetermined prior to initiation of trial

• Protocol deviations and poor adherence may have a Protocol deviations and poor adherence may have a larger impact on quality than in conventional trialslarger impact on quality than in conventional trials

• Analysis should be by intention-to-treat AND on-Analysis should be by intention-to-treat AND on-treatment, and should also report absolute AND treatment, and should also report absolute AND relative effects relative effects



So when appraising these trials ask So when appraising these trials ask yourself:yourself:• Was the active control previously shown to be effective?Was the active control previously shown to be effective?• Were study patients and outcome variables similar to Were study patients and outcome variables similar to

those in the original trials that established the efficacy of those in the original trials that established the efficacy of the active control?the active control?

• Were both regimens applied in optimal fashion?Were both regimens applied in optimal fashion?• Was the appropriate null hypothesis tested?Was the appropriate null hypothesis tested?• Was the equivalence margin specified before the study?Was the equivalence margin specified before the study?• Was the trial large enough?Was the trial large enough?• Was the analysis intention-to-treat AND on-treatment, Was the analysis intention-to-treat AND on-treatment,

and should also report absolute AND relative effects?and should also report absolute AND relative effects?• PLUS usual assessment of size and precision of PLUS usual assessment of size and precision of

treatment effect!treatment effect!



Many other types of study in the Many other types of study in the hierarchy e.g. systematic reviews, hierarchy e.g. systematic reviews, case-control studies, cohort studiescase-control studies, cohort studies

Details of appraisal not included Details of appraisal not included today but worth thinking about most today but worth thinking about most in terms of:in terms of:• Internal validityInternal validity• BiasBias• External validityExternal validity


3.3. Critical appraisal—other Critical appraisal—other study designsstudy designs



Cohort studyCohort study

PopulationSample

Exposed

Not exposed

Outcome

Outcome

Time

Case control studyCase control study



Population

CasesExposed

Not exposed

Time

ControlsNot exposed

Exposed

Study

Important in decision makingImportant in decision making Considers costs and consequences of Considers costs and consequences of

alternate courses of actionalternate courses of action Scope depends on level of decision Scope depends on level of decision

making they informmaking they inform Costs always measured the same Costs always measured the same

way (although scope may vary), way (although scope may vary), measurement of consequence varies measurement of consequence varies according to study typeaccording to study type


3.3. Economic evidenceEconomic evidence

Cost minimisation analysesCost minimisation analyses• analyse difference in costs where there analyse difference in costs where there

is no difference in outcomeis no difference in outcome• narrow focusnarrow focus

Cost-effectiveness analysesCost-effectiveness analyses• outcome expressed in natural units (e.g. outcome expressed in natural units (e.g.

validated single marker)validated single marker)• again fairly narrow scope—generally 2 again fairly narrow scope—generally 2

interventions for single indicationinterventions for single indication


3.3. Critical appraisal—what is Critical appraisal—what is economic evidence?economic evidence?

Cost-utility analysesCost-utility analyses• Costs and consequences but Costs and consequences but

consequence measured as utility consequence measured as utility • Utility = a value of health state rather Utility = a value of health state rather

than a natural marker (usually QALY)than a natural marker (usually QALY)• QALY is a measure of a person’s length QALY is a measure of a person’s length

of life weighted by a valuation of their of life weighted by a valuation of their health-related quality-of-life (HRQL) over health-related quality-of-life (HRQL) over that periodthat period

• Can compare incongruent interventionsCan compare incongruent interventions



Cost utility analysisCost utility analysis• Used by NICEUsed by NICE

Consider outcome in terms valuable to patients i.e. Consider outcome in terms valuable to patients i.e. life expectancy and qualitylife expectancy and quality

Uses a standard measure (utility)Uses a standard measure (utility) Comprehensive consideration of costsComprehensive consideration of costs NICE state that:NICE state that:

Technologies can be considered to be cost Technologies can be considered to be cost effective if their health benefits are greater effective if their health benefits are greater than the opportunity costs measured in terms than the opportunity costs measured in terms of the health benefits associated with of the health benefits associated with programmes that may be displaced to fund the programmes that may be displaced to fund the new technology. In other words, the general new technology. In other words, the general consequences for the wider group of patients consequences for the wider group of patients in the NHS are considered alongside the effects in the NHS are considered alongside the effects for those patients who may directly benefit for those patients who may directly benefit from the technology of interest.from the technology of interest.



Cost-benefit analysesCost-benefit analyses• Broadest possible scope!Broadest possible scope!• Allocation of resources between Allocation of resources between

difference sectors of economydifference sectors of economy• Maximising social welfareMaximising social welfare• Rely on assigning cost to healthcare Rely on assigning cost to healthcare

intervention—incredible complicated in intervention—incredible complicated in practice, particular in NHS where practice, particular in NHS where individuals don’t pay at point of deliveryindividuals don’t pay at point of delivery



A lot is published! But texts I’ve found A lot is published! But texts I’ve found useful include:useful include:• Straus SE, Richardson WS, Paul Glasziou, Haynes RB. Straus SE, Richardson WS, Paul Glasziou, Haynes RB.

Evidence-based Medicine: How to Practice and Teach Evidence-based Medicine: How to Practice and Teach EBM, Third Edition. Churchill Livingstone: Edinburgh, EBM, Third Edition. Churchill Livingstone: Edinburgh, 20052005

• Guyatt GH, et al. Users’ Guides to the Medical Literature: Guyatt GH, et al. Users’ Guides to the Medical Literature: II. How to use an article about therapy or prevention. A. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA 1993; 270: Are the results of the study valid? JAMA 1993; 270: 2598-26012598-2601

• Guyatt GH, et al. Users’ Guides to the Medical Literature: Guyatt GH, et al. Users’ Guides to the Medical Literature: II. How to use an article about therapy or prevention. B. II. How to use an article about therapy or prevention. B. What are the results and will they help me in caring for What are the results and will they help me in caring for my patients? JAMA 1994; 271: 59-63my patients? JAMA 1994; 271: 59-63

• Evidence Based Medicine (EBM) journal notebook seriesEvidence Based Medicine (EBM) journal notebook series—available online —available online

• Clinical evidence online. Excellent terminology glossary Clinical evidence online. Excellent terminology glossary and explanationsand explanations


3.3. Critical appraisal—further Critical appraisal—further readingreading

Is our patient/population so different from Is our patient/population so different from those in the study that its results cannot those in the study that its results cannot apply?apply?

Is the treatment feasible for us?Is the treatment feasible for us? What are the potential benefits and harms What are the potential benefits and harms

from therapy?from therapy? What are our patients’ values and What are our patients’ values and

preferences for both the outcome and preferences for both the outcome and side-effects?side-effects?

Is there any economic analysis supporting Is there any economic analysis supporting therapy?therapy?

How do we use EBM?How do we use EBM?

4.4. Applying results of research to Applying results of research to our individual patient or populationour individual patient or population

The scenarioThe scenario• You work in a mental health trust and You work in a mental health trust and

have been asked to consider the off-label have been asked to consider the off-label use of risperidone in bipolar disorder for use of risperidone in bipolar disorder for the D&TC following a request to use this the D&TC following a request to use this for a number of patientsfor a number of patients

• Your literature search identifies 4 RCTs of Your literature search identifies 4 RCTs of which this is the most substantive studywhich this is the most substantive study

• The requesting consultant suggests a The requesting consultant suggests a licensed application has been filed with licensed application has been filed with the eMEA but the outcome is unknownthe eMEA but the outcome is unknown

Example critical appraisal Example critical appraisal workshopworkshop

Your roleYour role• You now need to appraise this study and You now need to appraise this study and

reach a conclusion as to its validity with reach a conclusion as to its validity with reference to the circumstance describedreference to the circumstance described

• You can use the CASP tool to helpYou can use the CASP tool to help• You should also identify further steps to You should also identify further steps to

take in helping prepare for the D&TC take in helping prepare for the D&TC meetingmeeting

Example critical appraisal Example critical appraisal workshopworkshop

Example critical appraisal of RCT Example critical appraisal of RCT workshopworkshop

Working through each point from the CASP tool:Working through each point from the CASP tool:

1.1. Did the study ask a clearly-focused question?Did the study ask a clearly-focused question?

2.2. Was this a randomised controlled trial (RCT) and Was this a randomised controlled trial (RCT) and was it appropriately so?was it appropriately so?

3.3. Were participants appropriately allocated to Were participants appropriately allocated to intervention and control groups?intervention and control groups?

4.4. Were participants, staff and study personnel Were participants, staff and study personnel ‘blind’ to participants’ study group?‘blind’ to participants’ study group?

5.5. Were all of the participants who entered the Were all of the participants who entered the trial accounted for at its conclusion?trial accounted for at its conclusion?

6.6. Were the participants in all groups followed Were the participants in all groups followed up and data collected in the same way?up and data collected in the same way?

7.7. Did the study have enough participants to Did the study have enough participants to minimise the play of chance?minimise the play of chance?

8.8. How are the results presented and what is How are the results presented and what is the main result?the main result?

9.9. How precise are these results?How precise are these results?

10.10. Were all important outcomes considered so Were all important outcomes considered so the results can be applied?the results can be applied?

Example critical appraisal of Example critical appraisal of RCT workshopRCT workshop


Good pointsGood points Potential Potential drawbacksdrawbacks

Overall, what are the main messages?

What are going to be your main What are going to be your main points (related to this study) when points (related to this study) when you’re preparing your paper for the you’re preparing your paper for the D&TC?D&TC?


evidence-based medicine and critical appraisal ben rehman director london medicines information...

Documents