Critically Appraised Topic (CAT)

Paul Cronin B.A., M.B. B.Ch. B.A.O., M.S.,

M.R.C.P.I., .F.R.C.R.

Department of Radiology, Division of Cardiothoracic Radiology,

University of Michigan, Ann Arbor, Michigan

AUR 60th Annual Meeting San Antonio Hill Country

San Antonio, Texas March 19-22, 2012

Introduction

• Difficult to publish Case Reports • IRB • Critically appraised topic - CAT

– Standardized summary of research evidence

– Organized around a clinical question – Providing a critique of the research – A statement of the clinical relevance

Critically Appraised Topic - CAT

• 275 – 17

• 4 – Seminars in Roentgenology – 2009 • 5 - Abdominal Imaging - 2008 • 8 - Canadian Association Radiology Journal

– 2001-2005

• Mini meta

Critically Appraised Topic - CAT

• Advantages – Quick – No IRB – Publishable – 4 journal pages

Critically Appraised Topic

1. Asking a focused and answerable question that translates uncertainty to an answerable question

2. Searching for the best available evidence

3. Critically appraising the evidence for validity and clinical relevance

Critically Appraised Topic

4. Applying the results to clinical practice

5. Evaluation of performance

Step 1: Ask an Answerable Question

• Important to – The patient, clinician, and/or learners – The patients’ well-being – The clinicians’ knowledge needs – Likely to recur in clinical practice – Answerable in the time available

Step 1: Ask an Answerable Question

• PICO – P – Population / problem – I – Intervention / index test – C – Control / reference test – O – Outcome

CAT study question PIOC format question Patient or Problem Intervention Comparison

Intervention

Outcome

Whether low-dose CTC would

perform as well as optical

colonoscopy (OC) for screening

patients for CRC?

Is low-dose CT colonography

equivalent to optical

colonoscopy in identifying >5

mm colonic polyps?

Screening

population of

patients with polyps

CT colonography Optical

colonoscopy

Accuracy in

diagnosis of >5 mm

colonic polyps /

colorectal cancer

Whether ultrasound performs

better than MRI in the diagnosis

of rotator cuff tears?

In patients with rotator cuff tears,

how does US compare to MRI for

diagnosis?

Patients with rotator

cuff tear

Ultrasound MRI Accuracy in

diagnosis of rotator

cuff tears

Whether coronary CTA performs

comparably to invasive coronary

angiography for identifying

potentially or probably

hemodynamically significant

native coronary artery disease,

defined as luminal diameter

stenosis of at least 50%?

In patients with known or

suspected coronary artery

disease, how does coronary CTA

compare to invasive coronary

angiography for identifying ≥

50% luminal diameter coronary

artery stenosis?

Patients with known

or suspected

coronary artery

disease

Coronary CT

angiography

Catheter coronary

angiography

Accuracy in

diagnosis of ≥ 50%

luminal diameter

coronary artery

stenosis

How does computerized

tomography angiography (CTA)

performs in comparison with

magnetic resonance

angiography (MRA) in the

detection of symptomatic carotid

stenosis?

In patients with symptomatic

carotid stenosis, how does CTA

compare with MRA for

diagnosis?

Patients with

symptomatic

carotid artery

stenosis

Carotid CT

angiography

Carotid MR

angiography

Accuracy in

diagnosis of carotid

artery stenosis

What is the current role of RFA

and how does RFA compare to

surgical resection for treatment

of colorectal liver metastases?

In patients with colorectal liver

metastases how does

percutaneous radiofrequency

ablation compare with surgical

resection or other ablative

techniques?

Patients with

colorectal liver

metastasis

Percutaneous

radiofrequency

ablation

Surgical resection Annual recurrence

or mortality rate

Step 2- Search for the Best Current Evidence

1. Identify terms to fit the PICO question

2. Search for secondary sources 3. Search for primary sources

PIOC format question Patient or Problem Intervention Comparison Intervention Outcome

Is low-dose CT colonography

equivalent to optical

colonoscopy in identifying

clinically meaningful colonic

polyps?

Polyp

OR

Colorectal Neoplasm

OR

Colonic Neoplasm

AND Computed Tomography

Colonography

OR

Colonography, Computed

Tomography

AND

Low-Dose

AND Colonoscopy AND Diagnosis

OR

Sensitivity and Specificity

In patients with rotator cuff

tears, how does US compare to

MRI for diagnosis?

Rotator Cuff

OR

Shoulder

OR

Tendon

OR

Injury

OR

Pathology

AND Ultrasonography

OR

Ultrasound

AND Magnetic resonance Imaging

OR

MRI

OR

Contrast enhanced

MRI

AND Diagnosis

OR

Sensitivity and Specificity

In patients with known or

suspected coronary artery

disease, how does coronary

CTA compare to invasive

coronary angiography for

identifying ≥ 50% luminal

diameter coronary artery

stenosis?

Coronary artery disease AND Tomography, x-ray computed

OR

Tomography, x-ray

computed/methods

AND Angiography

OR

Coronary angiography

OR

Coronary angiography

methods

AND Coronary stenosis

OR

Diagnosis

OR

Sensitivity and specificity

In patients with symptomatic

carotid stenosis, how does CTA

compare with MRA for

diagnosis?

Carotid artery stenosis AND Tomography, x-ray computed

OR

Tomography, x-ray

computed/methods

AND MRI / MRA

OR

Magnetic resonance Imaging

OR

Contrast enhanced

MRI

AND Diagnosis

OR

Sensitivity and Specificity

In patients with colorectal liver

metastases how does

percutaneous

radiofrequency ablation

compare with surgical resection

or other ablative techniques?

“liver neoplasm”

OR

“Liver neoplasm/secondary”

AND catheter ablation AND liver neoplasm / surgery AND Efficacy

OR

Recurrence

OR

Mortality

Step 2- Search for the Best Current Evidence

• Primary Study Designs – Randomized double-blind clinical (or

controlled) trial - RCT – Observational studies

• Prospective cohort studies • Retrospective cohort studies • Case-control studies • Case series

Step 2- Search for the Best Current Evidence

• Levels of Evidence – The Oxford center for evidence based

medicine has graded study designs in a hierarchy of evidence

Level Etiology Diagnosis Intervention Prognosis

Ia Systematic Review with

homogeneity of Level Ib

studies

Systematic Review with

homogeneity of Level Ib

studies

Systematic Review with

homogeneity of Level Ib

studies

Systematic Review with

homogeneity of Level Ib

studies

Ib Randomized controlled trial Validating cohort study with

good reference standard

Randomized controlled trial Cohort study with good (>80%)

follow up

Ic Absolute SP-ins and SN-outs4

IIa Systematic Review with

homogeneity of Level IIb

studies

Systematic Review with

homogeneity of Level IIb

studies

Systematic Review with

homogeneity of Level IIb

studies

Systematic Review with

homogeneity of Level IIb

studies

IIb Cohort study (prospective) Exploratory cohort study with

good reference standard

Cohort study (prospective) Cohort study (retrospective)

IIc Outcomes research Outcomes research Outcomes research Outcomes research

IIIa Systematic Review with

homogeneity of non

consecutive cohort or case

control studies

Systematic Review with

homogeneity of Level IIIb

studies

Systematic Review with

homogeneity of case control

studies

Systematic Review with

homogeneity of case control

studies

IIIb Non consecutive cohort study

or case control study

Non consecutive studies; or

without consistently applied

reference standard

Case control study

IV Poor quality cohort study or

poor quality case control

study or case series

Case series or poor or non

independent reference

standard

Poor quality cohort study or

poor quality case control

study or case series

Poor quality cohort study or

poor quality case control study

or case series

V “expert” opinion without

critical appraisal

“expert” opinion without

critical appraisal

“expert” opinion without

critical appraisal

“expert” opinion without

critical appraisal

Step 2- Search for the Best Current Evidence

• The Evidence Pyramid

Step 2- Search for the Best Current Evidence

• Sources of Evidence for 1st Literature – Studies

• PubMed • MEDLINE • EMBASE • ISI Web of Knowledge • MD Consult • Google Scholar

Step 2- Search for the Best Current Evidence

• Sources of Evidence for 2nd Literature – Syntheses

• PubMed Clinical • Cochrane Library • Turning Research into Practice (TRIP) • National Institute of Clinical Excellence

(NICE) • SUMSearch

Step 2- Search for the Best Current Evidence

• Sources of Evidence for 2nd Literature – Syntheses

• Cumulative Index to Nursing and Allied Health Literature (CINAHL)

• National Guidelines Clearinghouse (NGC) • National Library for Health • Scottish Intercollegiate Guidelines Network

(SIGN)

Step 2- Search for the Best Current Evidence

• Sources of Evidence for 2nd Literature – Synopses

• The American College of Physicians (ACP) Journal Club

• Evidence Based Medicine Online – Systems

• Clinical Evidence • Up to Date

Step 3- Appraise the Literature • Critically Appraising Primary Literature

Section and Topic

Item

#

TITLE/ABSTRACT /

KEYWORDS

1 Identify the article as a study of diagnostic accuracy (recommend

MeSH heading ‘sensitivity and specificity’).

INTRODUCTION

2 State the research questions or study aims, such as estimating

diagnostic accuracy or comparing accuracy between tests or across

participant groups.

METHODS

Participants 3 Describe the study population: The inclusion and exclusion criteria,

setting and locations where the data were collected.

4 Participant recruitment: Was recruitment based on presenting

symptoms, results from previous tests, or the fact that the participants

had received the index tests or the reference standard?

5 Participant sampling: Was the study population a consecutive series

of participants defined by the selection criteria in item 3 and 4? If not,

specify how participants were further selected.

6 Data collection: Was data collection planned before the index test

and reference standard were performed (prospective study) or after

(retrospective study)?

Test methods 7 The reference standard and its rationale.

8 Technical specifications of material and methods involved including

how and when measurements were taken, and/or cite references for

index tests and reference standard.

9 Definition of and rationale for the units, cutoffs, and/or categories of

the results of the index tests and the reference standard.

10 The number, training, and expertise of the persons executing and

reading the index tests and the reference standard.

11 Whether or not the readers of the index tests and reference standard

were blind (masked) to the results of the other test and describe any

other clinical information available to the readers.

Statistical methods 12 Methods for calculating or comparing measures of diagnostic

accuracy, and the statistical methods used to quantify uncertainty

(e.g., 95% confidence intervals).

13 Methods for calculating test reproducibility, if done.

RESULTS

Participants 14 Report when study was done, including beginning and ending dates

of recruitment.

15 Clinical and demographic characteristics of the study population

(e.g., age, sex, spectrum of presenting symptoms, comorbidity,

current treatments, recruitment centers).

16 The number of participants satisfying the criteria for inclusion that did

or did not undergo the index tests and/or the reference standard;

describe why participants failed to receive either test (a flow diagram

is strongly recommended).

Test results

17 Time interval from the index tests to the reference standard, and any

treatment administered between.

18 Distribution of severity of disease (define criteria) in those with the

target condition; other diagnoses in participants without the target

condition.

19 A cross tabulation of the results of the index tests (including

indeterminate and missing results) by the results of the reference

standard; for continuous results, the distribution of the test results by

the results of the reference standard.

20 Any adverse events from performing the index tests or the reference

standard.

Estimates 21 Estimates of diagnostic accuracy and measures of statistical

uncertainty (e.g., 95% confidence intervals).

22 How indeterminate results, missing responses, and outliers of the

index tests were handled.

23 Estimates of variability of diagnostic accuracy between subgroups of

participants, readers or centers, if done.

24 Estimates of test reproducibility, if done.

DISCUSSION 25 Discuss the clinical applicability of the study findings.

STARD

Item Yes No Unclear

1. Was the spectrum of patients representative of the patients who will receive

the test in practice?

( ) ( ) ( )

2. Were selection criteria clearly described? ( ) ( ) ( )

3. Is the reference standard likely to correctly classify the target condition? ( ) ( ) ( )

4. Is the time period between reference standard and index test short enough to

be reasonably sure that the target condition did not change between the two

tests?

( ) ( ) ( )

5. Did the whole sample or a random selection of the sample, receive verification

using a reference standard of diagnosis?

( ) ( ) ( )

6. Did patients receive the same reference standard regardless of the index test

result?

( ) ( ) ( )

7. Was the reference standard independent of the index test (i.e. the index test

did not form part of the reference standard)?

( ) ( ) ( )

8. Was the execution of the index test described in sufficient detail to permit

replication of the test?

( ) ( ) ( )

9. Was the execution of the reference standard described in sufficient detail to

permit its replication?

( ) ( ) ( )

10. Were the index test results interpreted without knowledge of the results of the

reference standard?

( ) ( ) ( )

11. Were the reference standard results interpreted without knowledge of the

results of the index test?

( ) ( ) ( )

12. Were the same clinical data available when test results were interpreted as

would be available when the test is used in practice?

( ) ( ) ( )

13. Were uninterpretable/ intermediate test results reported? ( ) ( ) ( )

QUADAS

Step 3- Appraise the Literature • Interpreting Results in Diagnostic

Radiology – Sensitivity – Specificity – Positive predictive value – Negative predictive value – Positive (LR+) – Negative (LR-)

Measure Meaning Interpretation

Diagnostic Literature

Sensitivity Patients with the disease with positive test results

("true positives")/ patients with disease

Negative test result in a highly sensitive test

can rule out the disease if prevalence is

relatively low

Specificity Patients without the disease with negative results

("true negatives")/ well people (no disease)

Positive test result in a highly specific test

can rule in the disease if prevalence is

relative high

Positive predictive

value (PPV)

Patient with true positive test results / all patients

with positive test result (true positives + false

positives)

This value is affected by the prevalence in

the population studied. If a test has a high

positive predictive value (>95%) in your

population, one may confidently commence

treatment

Negative predictive

value (NPV)

Patients with true negative results / all patients with

negative test result (true negatives + false

negatives)

This value is affected by the prevalence in

the population studied. If a test has a high

negative predictive value (>95%) in your

population, one may be able to safely

withhold treatment

Likelihood ratio (LR) Likelihood of a given test result in patients with

disease / the likelihood of the same result in

patients without disease

LR=0 (negative LR): excludes the disease

LR= infinity (positive LR): confirms the

disease

LR=1 the test result is uninformative as the

result is equally likely in the two groups

Step 3- Appraise the Literature • Statistical Measures for Therapeutic

Studies – Hazard ratio - HR – Relative risk - RR – Attributable risk - AR – Number needed to treat / harm - NNT / NNH

Measure Meaning Interpretation

Therapeutic Literature

Relative risk (RR) Risk of developing disease (or outcome)

in the treatment group/ risk of developing

disease (or outcome) in the control group

RR=1: no difference between

groups

RR<1 treatment reduces the risk of

the disease

RR>1 treatment increases the risk

of the disease

Absolute risk (AR) Difference in the rates of events

calculated as the rate in the control group

minus the rate in the experimental group

AR =0 no difference between

groups

AR positive: treatment is beneficial

AR negative: treatment is harmful

Number needed to

treat (NNT)

The number of patients needed to be

treated for 1 of them to benefit from the

treatment

Number needed to

harm (NNH)

The number of patients needed to be

treated for 1 patient to experience an

Step 3- Appraise the Literature • Critically Appraising Secondary Literature

Study Design Quality of Evidence Lower if Higher if

Randomized Trial → High Risk of bias

-1 Serious

-2 Very serious

Inconsistency

-1 Serious

-2 Very serious

Indirectness

-1 Serious

-2 Very serious

Imprecision

-1 Serious

-2 Very serious

Publication bias

-1 Likely

-2 Very likely

Large effect

+1 Large

+2 Very Large

Dose response

+1 Evidence of gradient

All plausible confounding

+1 Would reduce a

demonstrated effect or

+1 Would suggest a

spurious effect when

results show no effect

Moderate

Observational study → Low

Very low

GRADE

Section/topic # Checklist item Reported

on page #

TITLE

Title 1 Identify the report as a systematic review, meta-analysis, or both.

ABSTRACT

Structured summary 2 Provide a structured summary including, as applicable: background; objectives;

data sources; study eligibility criteria, participants, and interventions; study

appraisal and synthesis methods; results; limitations; conclusions and implications

of key findings; systematic review registration number.

INTRODUCTION

Rationale 3 Describe the rationale for the review in the context of what is already known.

Objectives 4 Provide an explicit statement of questions being addressed with reference to

participants, interventions, comparisons, outcomes, and study design (PICOS).

METHODS

Protocol and

registration

5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web

address), and, if available, provide registration information including registration

number.

Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report

characteristics (e.g., years considered, language, publication status) used as

criteria for eligibility, giving rationale.

Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact

with study authors to identify additional studies) in the search and date last

searched.

Search 8 Present full electronic search strategy for at least one database, including any

limits used, such that it could be repeated.

Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in

systematic review, and, if applicable, included in the meta-analysis).

Data collection

process

10 Describe method of data extraction from reports (e.g., piloted forms, independently,

in duplicate) and any processes for obtaining and confirming data from

investigators.

Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding

sources) and any assumptions and simplifications made.

Risk of bias in

individual studies

12 Describe methods used for assessing risk of bias of individual studies (including

specification of whether this was done at the study or outcome level), and how this

information is to be used in any data synthesis.

Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means).

Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done,

including measures of consistency (e.g., I2) for each meta-analysis.

Risk of bias across

studies

15 Specify any assessment of risk of bias that may affect the cumulative evidence

(e.g., publication bias, selective reporting within studies).

Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses,

meta-regression), if done, indicating which were pre-specified.

RESULTS

Study selection 1

7

Give numbers of studies screened, assessed for eligibility, and included in the

review, with reasons for exclusions at each stage, ideally with a flow diagram.

Study

characteristics

1

8

For each study, present characteristics for which data were extracted (e.g., study

size, PICOS, follow-up period) and provide the citations.

Risk of bias within

studies

1

9

Present data on risk of bias of each study and, if available, any outcome level

assessment (see item 12).

Results of individual

studies

2

0

For all outcomes considered (benefits or harms), present, for each study: (a)

simple summary data for each intervention group (b) effect estimates and

confidence intervals, ideally with a forest plot.

Synthesis of results 2

1

Present results of each meta-analysis done, including confidence intervals and

measures of consistency.

Risk of bias across

studies

2

2

Present results of any assessment of risk of bias across studies (see Item 15).

Additional analysis 2

3

Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses,

meta-regression [see Item 16]).

DISCUSSION

Summary of

evidence

2

4

Summarize the main findings including the strength of evidence for each main

outcome; consider their relevance to key groups (e.g., healthcare providers, users,

and policy makers).

Limitations 2

5

Discuss limitations at study and outcome level (e.g., risk of bias), and at review-

level (e.g., incomplete retrieval of identified research, reporting bias).

Conclusions 2

6

Provide a general interpretation of the results in the context of other evidence, and

implications for future research.

FUNDING

Funding 2

7

Describe sources of funding for the systematic review and other support (e.g.,

supply of data); role of funders for the systematic review.

PRISMA

1. Was an ‘a priori’ design provided?

The research question and inclusion criteria should be established

before the conduct of the review.

Yes

No

Can’t answer

Not applicable

2. Was there duplicate study selection and data extraction?

There should be at least two independent data extractors and a

consensus procedure for disagreements should be in place.

Yes

No

Can’t answer

Not applicable

3. Was a comprehensive literature search performed?

At least two electronic sources should be searched. The report must

include years and databases used (e.g. Central, EMBASE, and

MEDLINE). Key words and/or MESH terms must be stated and where

feasible the search strategy should be provided. All searches should

be supplemented by consulting current contents, reviews, textbooks,

specialized registers, or experts in the particular field of study, and by

reviewing the references in the studies found.

Yes

No

Can’t answer

Not applicable

4. Was the status of publication (i.e. grey literature) used as an

inclusion criterion?

The authors should state that they searched for reports regardless of

their publication type. The authors should state whether or not they

excluded any reports (from the systematic review), based on their

publication status, language etc.

Yes

No

Can’t answer

Not applicable

5. Was a list of studies (included and excluded) provided?

A list of included and excluded studies should be provided.

Yes

No

Can’t answer

Not applicable

6. Were the characteristics of the included studies provided?

In an aggregated form such as a table, data from the original studies

should be provided on the participants, interventions and outcomes.

The ranges of characteristics in all the studies analyzed e.g. age,

race, sex, relevant socioeconomic data, disease status, duration,

severity, or other diseases should be reported.

Yes

No

Can’t answer

Not applicable

7. Was the scientific quality of the included studies assessed and

documented?

‘A priori’ methods of assessment should be provided (e.g., for

effectiveness studies if the author(s) chose to include only

randomized, double-blind, placebo controlled studies, or allocation

concealment as inclusion criteria); for other types of studies

alternative items will be relevant.

Yes

No

Can’t answer

Not applicable

8. Was the scientific quality of the included studies used

appropriately in formulating conclusions?

The results of the methodological rigor and scientific quality should

be considered in the analysis and the conclusions of the review, and

explicitly stated in formulating recommendations.

Yes

No

Can’t answer

Not applicable

9. Were the methods used to combine the findings of studies

appropriate?

For the pooled results, a test should be done to ensure the studies

were combinable, to assess their homogeneity (i.e. Chi-squared test

for homogeneity, I²). If heterogeneity exists a random effects model

should be used and/or the clinical appropriateness of combining

should be taken into consideration (i.e. is it sensible to combine?).

Yes

No

Can’t answer

Not applicable

10. Was the likelihood of publication bias assessed?

An assessment of publication bias should include a combination of

graphical aids (e.g., funnel plot, other available tests) and/or

statistical tests (e.g., Egger regression test).

Yes

No

Can’t answer

Not applicable

11. Was the conflict of interest stated?

Potential sources of support should be clearly acknowledged in both

the systematic review and the included studies.

Yes

No

Can’t answer

Not applicable

AMSTAR

Original AGREE Item AGREE II Item

Domain 1. Scope and Purpose

1. The overall objective(s) of the guideline is (are)

specifically described.

The overall objective(s) of the guideline is (are) specifically

described.

2. The clinical question(s) covered by the guideline is (are)

specifically described.

The health question(s) covered by the guideline is (are)

specifically described.

3. The patients to whom the guideline is meant to apply are

specifically described.

The population (patients, public, etc.) to whom the guideline is

meant to apply is specifically described.

Domain 2. Stakeholder Involvement

4. The guideline development group includes individuals

from all the relevant professional groups.

The guideline development group includes individuals from all

the relevant professional groups.

5. The patients’ views and preferences have been sought.

The views and preferences of the target population (patients,

public, etc.) have been sought.

6. The target users of the guideline are clearly defined.

No change

7. The guideline has been piloted among end users.

Delete item. Incorporated into user guide description of item

19.

Domain 3. Rigour of Development

8. Systematic methods were used to search for evidence.

Systematic methods were used to search for evidence.

Renumber to 7.

9. The criteria for selecting the evidence are clearly

described.

The criteria for selecting the evidence are clearly described.

Renumber to 8.

NEW Item 9. The strengths and limitations of the body of

evidence are clearly described.

10. The methods for formulating the recommendations are

clearly described.

The methods for formulating the recommendations are clearly

described.

11. The health benefits, side effects, and risks have been

considered in formulating the recommendations.

The health benefits, side effects, and risks have been

considered in formulating the recommendations.

12. There is an explicit link between the recommendations

and the supporting evidence.

There is an explicit link between the recommendations and the

supporting evidence.

13. The guideline has been externally reviewed by experts

prior to its publication.

The guideline has been externally reviewed by experts prior to

its publication.

14. A procedure for updating the guideline is provided.

A procedure for updating the guideline is provided.

Domain 4. Clarity of Presentation

15. The recommendations are specific and unambiguous.

The recommendations are specific and unambiguous.

16. The different options for management of the condition

are clearly presented.

The different options for management of the condition or

health issue are clearly presented.

17. Key recommendations are easily identifiable.

Key recommendations are easily identifiable.

Domain 5. Applicability

18. The guideline is supported with tools for application.

The guideline provides advice and/or tools on how the

recommendations can be put into practice.

AND Change in domain (from Clarity of Presentation) AND

renumber to 19

19. The potential organizational barriers in applying the

recommendations have been discussed.

The guideline describes facilitators and barriers to its

application.

AND change in order – renumber to 18

20. The potential cost implications of applying the

recommendations have been considered.

The potential resource implications of applying the

recommendations have been considered.

21. The guideline presents key review criteria for

monitoring and/ or audit purposes.

The guideline presents monitoring and/ or auditing criteria.

Domain 6. Editorial Independence

22. The guideline is editorially independent from the funding

body.

The views of the funding body have not influenced the content

of the guideline.

23. Conflicts of interest of guideline development members

have been recorded.

Competing interests of guideline development group

members have been recorded and addressed.

AGREE

Steps Primary research Secondary research Critical appraisal issue

(Systematic review /

Meta-analysis)

Q Formulate a question Formulate a question Does the research ask a

(PI CO) (PI CO) clearly focused question

(PI CO) and use it to direct

the search and select

articles for inclusion?

F Find the best evidence Find the best evidence Did the search find all the

best evidence?

A Appraise the included Appraise the included Have the studies been

studies studies critically appraised?

S Synthesise the results Have the results been

(summary tables and synthesized with

plots) appropriate summary

tables and plots?

Apply the results Apply the evidence

Step 3- Appraise the Literature

• Results of a Meta-analysis – Have the studies been critically

appraised (using appropriate quality criteria)?

– Have the results been synthesized with appropriate summary tables and plots?

Step 3- Appraise the Literature

• Summarizing the Results of your Literature Review – http://www.radiography.com/pub/

Section A To calculate diagnostic accuracy parameters from a 2x2 table Enter numbers of patients in the highlighted cells: Test Positive Negative Disease + 85 11 96 Disease - 22 37 59

n = 155

Calculated Diagnostic Accuracy Parameters:

Sample size = 155

Prevalence = 0.619 Lower CI Limit Upper CI Limit

Sensitivity = 0.885 95% CI(Sens)= 0.822 0.949

Specificity = 0.627 95% CI(Spec)= 0.504 0.751 PPV = 0.794 NPV = 0.771

LR + result*= 2.375 *Interpreting Likelihood Ratios:

LR - result*= 0.183 0 excludes disease,infinity excludes normality LR > 10 or < 0.1 = large change in likelihood LR 5-10 or 0.1-0.2 = moderate change LR 2-5 or 0.2-0.5 = small change LR of 1 = no change

Section B

To calculate the predictive values of this test in local population: Enter local disease prevalence (0-1) 0.15 2*2 Table recalculated (new prevalence, n =100) Test Positive Negative Disease + 13 2 Disease - 32 53 Local PPV = 0.295 Local NPV = 0.969

Section C Bayesian Analysis (for any test with known sensitivity and specificity): 3 Variables required - enter data in the highlighted cells: Sensitivity (0-1) 0.885 Specificity (0-1) 0.627 Pretest prob. (0-1) 0.15

Posttest prob. pos result= 0.295

Posttest prob. neg result= 0.031

Section D

Estimation of 95% CI; known sample size, disease prevalence, sensitivity and specificity* 4 variables required - enter data in highlighted cells: Total sample size 100 Prevalence (0-1) 0.5 Lower Limit Upper Limit Sensitivity (0-1) 0.95 95% CI 0.890 1.010 Specificity (0-1) 0.8 95% CI 0.689 0.911

*NB: Read limitations of CI estimations for small sample sizes and sensitivity /specificity >0.9 or <0.1

Section E Calculation of Likelihood Ratios from known Sensitivity and Specificity 2 variables required - enter data in highlighted cells: Sensitivity (0-1) 0.885 Specificity (0-1) 0.627 LR +ve test = 2.373 LR -ve test = 0.183

Step 4: Apply

• Test result means to an individual patient – Transform sensitivity/specificity to

likelihood of having the disease – Pre-test probability → Post test

probability – Fagan’s nomogram – Graph of conditional probabilities

Graph of Conditional Probability

Step 5: Evaluate

• Evaluate - applied to or alter clinical practice – Ongoing developments in technology – Evaluate technical parameters /

generations of equipment used • Limitations of available literature

– Need for better quality studies / secondary studies

Name Link

PubMed. US National Library of Medicine and National Institutes of Health. http://www.ncbi.nlm.nih.gov/sites/entrez.

Medline / Ovid SP. http://gateway.ovid.com/

EMBASE http://www.embase.com/.

Google Scholar http://scholar.google.com/.

ISI Web of Knowledge http://apps.isiknowledge.com/UA_GeneralSearch_input.do?product=UA&search_mode=General

Search&SID=4BKm4mOcfDnofbKjHBI&preferencesSaved=.

MD Consult http://www.mdconsult.com/php/120885574-2/homepage.

Cochrane Collaboration http://www.cochrane.org.

The Cumulative Index to Nursing and Allied Health Literature http://www.ebscohost.com/cinahl/.

National Institute of Clinical Excellence http://www.nice.org.uk/.

Scottish Intercollegiate Guidelines Network http://www.sign.ac.uk/.

SUMSearch search engine http://sumsearch.uthscsa.edu/.

The National Library for Health http://www.library.nhs.uk/Default.aspx.

National Guidelines Clearinghouse http://www.guideline.gov/.

PubMed Clinical Queries http://www.ncbi.nlm.nih.gov/entrez/query/static/clinical.shtml.

The American College of Physicians Journal Club http://www.acpjc.org/.

BMJ Evidence Based Medicine http://ebm.bmj.com/.

Bandolier electronic journal. http://www.medicine.ox.ac.uk/bandolier/aboutus.html.

The Turning Research Into Practice Database http://www.tripdatabase.com/index.html.

UpToDate http://www.uptodate.com/home/index.html.

Dynamed Clinical Reference Tool http://www.ebscohost.com/dynamed/.

Physicians Information and Education Resource http://pier.acponline.org/index.html.

BMJ Clinical Evidence http://clinicalevidence.bmj.com/ceweb/index.jsp.

The Database of Abstracts of Reviews of Effects http://mrw.interscience.wiley.com/cochrane/cochrane_cldare_articles_fs.html.

The Cochrane Central Register of Controlled Clinical Trials (CENTRAL) http://www.mrw.interscience.wiley.com/cochrane/cochrane_clcentral_articles_fs.html.

The National Library of Medicine http://www.nlm.nih.gov/.

Netting the Evidence http://www.shef.ac.uk/scharr/ir/netting/.

The Center for Evidence Based Radiology at the Brigham and Women’s` hospital http://www.brighamandwomens.org/cebi/default.aspx

Centre for Reviews and Dissemination at the University of York http://www.york.ac.uk/inst/crd/.

The Centre for Health Evidence at the University of Alberta http://www.cche.net/.

The Centre for Evidence Based Medicine at Oxford http://www.cebm.net/.

The Centre for Evidence Based Medicine at the University Health Network, Toronto http://www.cebm.utoronto.ca/practise/evaluate/index.htm#top

The Blue Cross and Blue Shield Association Technology Evaluation Center http://www.bcbs.com/blueresources/tec/.

Critically Appraised Topic (CAT)

Paul Cronin B.A., M.B. B.Ch. B.A.O., M.S.,

M.R.C.P.I., .F.R.C.R.

Department of Radiology, Division of Cardiothoracic Radiology,

University of Michigan, Ann Arbor, Michigan