ebm 2013 diagnosis

7/29/2019 eBm 2013 Diagnosis

1/58

Critical AppraisalHow to Use an Article About a

Diagnostic Test

Madeleine Grace M. Sosa, MD

FPPS,FPNA,FCNSP MSClinical Epidemiology


2/58

General Objective

To apply the various rules of evidence in

arriving at decisions regarding the usefulness

of a diagnostic test


3/58

Specific Objectives

To discuss the rationale for each of the users

guides pertaining to the validity of claims on

accuracy To define and differentiate the concepts of

validity and reliability, as they pertain to theresults of a diagnostic test

To learn the definitions of and calculations forsensitivity, specificity, positive and negativepredictive value, likelihood ration, pretest andpost test probability, pre-test and post-test odds


4/58

Specific Objectives

To recognize the advantages and

disadvantages of using likelihood ratios,sensitivities, specificities and predictive values

in measuring the accuracy of diagnostic tests

To learn how to estimate posttest probabilitiesgiven the pre-test probability and a measure

of accuracy


5/58

Clinical Scenario

Your nieces classmate Marco was admitted in a hospital while

on vacation in the province. Apparently he presented with

high grade fever of 5 days duration with associated abdominal

pain. He remained well except for generalized body malaise

and slightly decreased appetite. His clinical signs are said tobe suggestive of typhoid fever or dengue fever. Investigations

included a CBC and platelet, which showed normal findings

except for a wbc of 4. Chest x-ray and urinalysis were normal.

Blood for culture was extracted. Typhidot-M was alsoextracted and reported to be positive.


6/58

Cont

Quite anxious and wanting to quickly arrive at

a specific diagnosis and knowing that you are

a physician, Marcos father who is a podiatrist

asked your help. He wanted to know how

sensitive is the Typhidot-M test in the

diagnosis of typhoid fever compared to a

blood culture, which will take another 5 daysto report.


7/58

P

I

C

O

Architecture of a focused question: a

4-part review question

+ study designRichardson et al. The well-built clinical question: a key to evidence-based decisions. ACP Journal Club 1995;A-12

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997;127:380-7.


8/58

CATmaker


9/58

Search Strategies


10/58

Search Strategies


11/58

Search Strategies


12/58


13/58

Evaluation of a simple and rapid dipstick

assay for the diagnosis of typhoid fever inIndonesia by Hussein Gasem et al,

published in the Journal of MedicalMicrobiology in 2001


14/58

Evaluating and applying the results of

a diagnostic tests

APPRAISING DIRECTNESS

APPRAISING VALIDITY APPRAISING OF RESULTS

ASSESSING APPLICABILITY

INDIVIDUALIZING RESULTS


15/58

I. Evaluating directness

Does the study provide a direct enough answer to

your clinical question in terms of

P: patient population with a certain disease

I/E: the exposure( or treatment) to beadministered

O: the outcome( or condition ) that the treatment

are intended to prevent or promote M: Methodology


16/58

Evaluating Directness

Clinical Question Research Question

P

E/I

C

O

M


17/58

Appraising diagnostic tests: 3 easy steps

1. Are the results valid?

2. What are the results?

3. Will they help me look

after my patients?

Appropriate spectrum of patients?

Does everyone get the gold standard?

Is there an independent, blind or objective

comparison with the gold standard?


18/58

II. Appraising Validity

Validity Questions

1. Was the reference standard and acceptable

one ?

2. Was the reference standard interpreted

independently from the test in question?


19/58


20/58


21/58





after my patients?

Sensitivity, specificity

Likelihood ratios

Predictive values


22/58

III. Appraising Results

What likelihood ratios were associated

with the range of possible test results ?


23/58

APPRAISING RESULTS?

What likelihood ratios were associated with

the range of possible results?

Measures of accuracy sensitivity, specificity,

positive predictive value, negative predictive value

Estimate the likelihood ratios for each of the

possible results

Estimate the pre-test and post-test probability


24/58


25/58

2 by 2 table: sensitivity

Disease

Test

+ -

+

-

Sensitivity = a / a + c

Proportion of people with the

disease who have a positive

test result.

So, a test with 84%

sensitivity.means that the

test identifies 84 out of 100

people WITH the disease

a

True

positives

c

Falsenegatives


26/58

2 by 2 table: sensitivity

Disease

Test

+ -

+

-

Sensitivity = 70/ 91= 0.769

Proportion of people with thedisease who have a positive

test result.

So, a test with 76.9%

sensitivity.means that the

test identifies 70 out of 91

people WITH the disease

70

21


27/58

2 by 2 table: specificity

Disease

Test

+ -

+

-

b

False

positives

d

True

negatives

Specificity = d / b + d

Proportion of people without

the disease who have a

negative test result.


28/58


Disease

Test

+ -

+

-

b

False

positives

d

True

negatives

Specificity = d / b + d




So, a test with 75% specificity

will be NEGATIVE in 75 out

of 100 people without the

disease


29/58


Disease

Test

+ -

+

-

b

7

d

109




So, a test with 94% specificity

will be NEGATIVE in 109 out

of 116 people without the

disease

Specificity=109/116=0.94


30/58

Reference standard

Test result Disease Present Disease Absent Total

Disease present True Positive

( a )

False Positive

( b )

A + B

Disease absent False Negative( c )

True Negative( d )

C + D

Sensitivity = a /( a+c )

Specificity = d /( b+d )

LR for positive test result = [ a /(a+c) / [b / ( b+d) ]

LR for negative test result = [ c /(a+c) / [d / ( b+d) ]


31/58


32/58


33/58

Blood Culture

Dipstick test Disease Present Disease Absent Total

Positive True Positive(a)

( 70 )

False Positive(b)

( 4 )

A + B

Negative False Negative

(c )

( 21 )

True Negative

(d)

( 32 )

C + D

Sensitivity = a /( a+c )

Specificity = d /( b+d )

LR for positive test result = [ a /(a+c) / [b / ( b+d) ]

LR for negative test result = [ c /(a+c) / [d / ( b+d) ]


34/58

RESULTS ( page 174, 2nd column)


35/58

What are the results?

Are the likelihood ratios for the test results

presented or data necessary for their

calculation provided? Usefulness of the test = determined by the

accuracy with which it identifies the target disease

Accuracy measure ---LIKELIHOOD RATIO


36/58

Likelihood ratios

Can be used in situations with more than 2

test outcomes

Direct link from pre-test probabilities to post-test probabilities


37/58

Likelihood ratios

Positive likelihood ratio (LR+)

How much more likely is a positive test to be found in a person with thedisease than in a person without it?

LR+ = sens/(1-spec)

Negative likelihood ratio (LR-)

How much more likely is a negative test to be found in a person withoutthe condition than in a person with it?

LR- = (1-sens)/(spec)


38/58

What do likelihood ratios mean?

LR>10 = strong

positive test

result

LR


39/58

2 x 2 table: positive likelihood ratio

Disease

Test

+ -

+

-

100-SENSI

SENSI 100-SPECI

SPECI

LR+ = a/a+c / b/b+d

or

LR+ = sens/(1-spec)

How much more often a positive test occurs

in people with compared to those withoutthe disease


40/58


41/58

BLOOD CULTURE

TEST REST Disease Present Disease Absent Row Total

Positive A B A+B

Intermediate C D C+D

Negative E F E+FColumn Total ( A+C+E) ( B+D+F)

Odds ratio of (+) = (A/B) / ( A+C+E)/ ( B+D+F)

Odds ratio of intermediate = (C/D)/ ( A+C+E)/ ( B+D+F)

Odds ratio of negative = (E/F) ( A+C+E)/ ( B+D+F)


42/58

Positive Likelihood Ratios

Dipstick Result With Typhoid

Fever

Without

Typhoid Fever

+ Likelihood Ratio

Negative 21 32 (21/32)/( 91/36)

+1 18 1

+2 38 1

+3 11 1

+4 3 1

TOTAL 91 36

A i i di i 3


43/58





after my patients?

Can I do the test in my setting?

Do results apply to the mix of patients I see?

Will the result change my management?

Costs to patient/health service?


44/58

IV. ASSESSING APPLICABILITY

Are there biologic issues that may affect

accuracy of the test?

Are there socio-economic issues that may

affect accuracy of the test?


45/58

V. INDIVIDUALIZING THE RESULTS

How will the results affect the probability

of disease in your patient? ( Estimate the

individualized post-test probability of

your patient)

Is this test useful for your patient?

l b l


46/58

Determine Applicability

Individualizing the results

1. Estimate the Pre-test Probability

2. Calculate the post-test Probability

Step1: Convert the pre-test probabilitypre-test

odds

Step2: Multiply the pre-Test odds by the LR of the

expected lab results = Post-test Odds

Convert the Post-test odds =Post-testprobability


47/58

Pretest probability and Odds

Pretest Prob : 70% (70/100)

Pretest Odds : 2.33 ( 70/30)

Odd: X /1-X

x


48/58

Posttest odds and probabiliy

Posttest odds = LR x pretest odds

Posttest probability = X/ 1+X

x


49/58

Will the result change management?

No action Test Action

(e.g. treat)

Probability of disease0% 100%

Testing

thresholdAction

threshold


50/58



(e.g. treat)


Testing

thresholdAction

threshold


51/58

Will the results help me in caring for my

patient?

Will the reproducibility of the test result and

its interpretation be satisfactory in my setting?

Are the results applicable to my patient?

Will the results change my management?

Will patients be better off as a result of the

test?


52/58

Will the results change my management?

Test threshold

Treatment threshold

When the probability of the target disorder lies

between the test and treatment thresholds,

further testing is mandated

%


53/58

Post test 20%

? Appendicitis:

McBurney tenderness LR+

= 3.4

Pre test 5%

%

Bayesian

reasoning

Fagan nomogram


54/58



(e.g. treat)


Testing

thresholdAction

threshold


55/58

Normogram

Pre test

Post test


56/58

Will the reproducibility of the test result and itsinterpretation be satisfactory in my setting?

The value of the test depends on its ability to yieldthe same results when re-applied to stable

patients Poor reproducibility can results from problems

with the test itself or in cases where the test willrequire interpretation


57/58

Are the results applicable to my patient?

Similar setting

Patient meets all the study inclusioncriteria

Patient does not violate any of the

exclusion criteria


58/58

Will patients be better off as a result of the

test?

Does it adds information beyond thatotherwise available?

Does it leads to change in management thatis ultimately beneficial to the patient?

The value of an accurate test will be

undisputed when the target disorder, if leftundiagnosed, is dangerous, the test hasacceptable risks and effective treatmentexists

ebm 2013 diagnosis

Documents