ek-5 20jahghn dr.cita_dx_200111 (sri-laptop's conflicted copy 2014-01-28) (rachmi-macbook-air's...

8/11/2019 EK-5 20Jahghn Dr.cita_DX_200111 (Sri-Laptop's Conflicted Copy 2014-01-28) (Rachmi-MacBook-Air's Conflicted Copy 2014-03-01)

1/83

How to Appraise a Diagnostic Test?

Dr. Cita Rosita Sigit Prakoeswa, dr, SpKK(K)Department of Dermato Venereology Dr Soetomo Hospital

Faculty of Medicine, Airlangga University, SurabayaTropical Disease Center, Airlangga University, Surabaya


2/83

What is diagnosis ?

Increase certainty aboutpresence/absence ofdisease

Disease severity Monitor clinical course

Assess prognosisrisk/stage within diagnosis

Plan treatment Screening

Epidemiology

Knottnerus, BMJ 2002


3/83

By the end of this session,

you should be able to.

describe and illustrate key measures of

diagnostic test performance

represent diagnostic test performance in 2

different ways


4/83

EBM Process

Patient

Encounter

Formulating the

Clinical Question

Searching the

Evidence

Appraising the

Evidence

Diagnosis

Therapy

Prognosis

Etiology

Patient

Intervention

Comparison

Outcome

Hierarchy of evidence

Pre appraised resources

Drawing conclusion

That impact on practice

DOES

POEM

(Lang, 2000) 4


5/83

What should I do

about this condition

or problem?

What causethe problem?

Does this person

have the condition

or problem?

Who will get

the condition

or problem?

How common

is the problem?

What are the

type of problem?

INTERVENTION

PROGNOSIS/RISK FACTORS

DIAGNOSIS

PROGNOSIS FACTORS

FREQUENCY & RATE

PHENOMENA / THOUGHTS

CLINICALQUESTION

5


6/83

ACQ Diagnosis (PICO)

Patient /

Problem /

Population

Intervention

(Index)Comparison Outcome

In an otherwise

healthy 7-year-

old boy with

sore throat

how does the

clinical exam

compare to

throat culture

in diagnosing

GAS infection?


7/83

ResearcherInvolvement

Longitudinal

Cross-sectional

ResearchGoal

ResearchApproach

Controlled?Randomized?

ResearchFocus Clinical Manifestation / Diagnosis / Prognosis / Therapy / Review

13 2 4

7


8/83

Hierarchy of study designs

8


9/83


10/83

Basic Principles (1)

Ideal diagnostic testsright answers:

(+) results in everyone with the disease and

( - ) results in everyone else

Usual clinical practice:

The test be studied in the same way it would

be used in the clinical setting

Observational study, and consists of: Predictor variable (test result)

Outcome variable (presence / absence of the

disease)


11/83

Basic Principles (2)

Sensitivity, specificity

Prevalence, prior probability, predictive values

Likelihood ratios Dichotomous scale, cutoff points (continuous

scale)

Positive (true and false), negative (true & false) ROC (receiver operator characteristic) curve


12/83

What is the reason that there aremany parameters in diagnostic test?

PrevalenceSensitivity (%)Specificity (%)LR+

LR-PPV (%)NPV (%)Pre-test OddsPost-test OddsPre-test Probability (%)Post-test Probability (%)

Disease

(+)

Disease

(-)Total

Test

(+)

True pos

a

False

pos

b

a+b

Test

(-)

False

neg

c

True neg

dc+d

Total a+c b+d

a+b+

c+d


13/83

METHOD 1:

NATURAL FREQUENCIES TREE

Population

1.000


14/83

IN EVERY 1.000 PEOPLE, 200 WILL HAVE THE DISEASE

Disease +

200

Disease -

800

Population

1.000

If these 1000 people are representative of the population atrisk, the assessed rate of those with the disease (20%)

represents the PREVALENCEof the diseaseit can also beconsidered the PRE-TEST PROBABILITY of having the disease


15/83

Sensitivity

The proportion of people who truly

have a designated disorder who are

so identified by the test. Sensitive tests have few false

negatives.

When a test with a high Sensitivity isNegative, it effectively rules out the

diagnosis of disease. SnNout


16/83

Disease +200

Disease -800

Test +190

Test -10

Population

1.000

In other words, thesensitivity is

190/200=95%

Test Alergi dengan Uji KulitSensitivitas 95 %, artinya:

SnNout: bila hasil uji kulitnya (-): 95% out (dia bukan penderita alergi )

Sensitivity


17/83

The proportion of people who are

truly free of a designated disorder

who are so identified by the test.

Specific tests have few false positives

When a test is highly specific, a

positive result can rule inthe

diagnosis. SpPin

Specificity


18/83

Disease +

200

Disease -

800

Test +

190

Test -

10

Population1000

Test Alergi dengan Uji KulitSpesifitas 96 % artinya:

SpPin: bila hasil uji kulitnya (+): 96% in (dia penderita alergi)

Test +

32

Test -

768

In other words, thespecificity is 768/800

= 96%

Specificity


19/83

CASESNON-CASES

Sensitivity & Specificity

Negative Positive

Degree of positivity on test

%o

fGroup

DISEASED

NON-DISEASED

Test cut-offFALSE

NEGATIVES

FALSEPOSITIVES

Numeric? (complex)


20/83


Sensitivity and Specificity are usuallyconsidered properties of the test ratherthan the setting, and are thereforeusually considered to remain constant.

However, sensitivity and specificity are

likely to be influenced by complexity ofdifferential diagnosesand a multitude ofother factors(cf spectrum bias).


21/83


Positive & Negative Predictive Value

For sensitivity and specificity, thereference variable (denominator) is theDISEASE

For predictive value, the reference

variable (denominator)is the TEST


22/83


23/83

POSITIVEPREDICTIVE

VALUE = 190/222=86 %

Disease +

200

Disease -

800

Test +

32

Test -

768

Test +

190

Test -

10

Population

1000

This is also the POST-

TEST PROBABILITY ofhaving the disease

Positive Predictive Value

Test Alergi dengan Uji KulitPPV 86 % artinya bila hasil uji kulitnya (+): kemungkinan dia

menderita alergi adalah 86%


24/83

Disease +

200

Disease -

800

Test +

32

Test -

768

Test +

190

Test -

10

Population

1000

Negative Predictive Value

NEGATIVEPREDICTIVEVALUE = 768/778=99%

Test Alergi dengan Uji KulitNPV 99 % artinya bila hasil uji kulitnya (-): kemungkinan dia

tidak menderita alergi adalah 99 %


25/83

Positive & Negative

Predictive Value

The Positive Predictive Value of a testwill vary (according to the prevalenceof the condition in the chosen setting)


26/83

Predictive value & changing prevalence

Disease +

200

Disease -

9.800

Population

10.000

Prevalence reduced by an orderof magnitude from 20% to 2%


27/83

Disease +

200

Disease -

9.800

Test +

392

Test -

9.408

Test +

190

Test -

10

Population

10.000Sensitivity and

Specificityunchanged

Predictive value & changing prevalence


28/83

POSITIVE

PREDICTIVEVALUE = 33%

Positive predictive value

at low prevalence

Disease +

200

Disease -

9.800

Test +392

Test -

9.408

Test +190

Test -

10

Population

10.000Previously, PPV

was 86%


29/83

NEGATIVEPREDICTIVE

VALUE >99%

Disease +

200

Disease -

9.800

Test +

392

Test -

9.408

Test +

190

Test -

10

Population

10.000Previously, NPV

was 99%

Negative predictive value

at low prevalence


30/83

Prediction of low prevalence events

Even highly specific tests, when applied

to low prevalence events, yield a high

number of false positive results

Because of this, under such

circumstances, the Positive Predictive

Value of a test is low

However, this has much less influence

on the Negative Predictive Value


31/83

Likelihood Ratio

Relative likelihood that a given test would be

expected in a patient with (as opposed to one

without) a disorder of interest.

probability (%) of the test resu ltin patients without disease

LR=probability (%) of a test resu ltin patients with disease
http://eta.cche.net/usersguides/hg/calculators/c1C-2.asp?A=50&B=50&C=50&D=50&H=ONhttp://eta.cche.net/usersguides/hg/calculators/c1C-2.asp?A=50&B=50&C=50&D=50&H=ON


32/83

Likelihood

The likelihood thatsomeone with thedisease will have apositive testis

190/200 or 95%This is the same asthe sensitivity

Disease +

200

Test +

190

Test -

10

Population

1000


33/83

The likelihood thatsomeonewithoutthe disease willhave a positive testis 32/800 or 4%This is the same asthe (1-specificity)

Disease -

800

Test +

32

Test -

768

Population

1000

Likelihood


34/83

LIKELIHOOD OF POSITIVE TESTIN THE ABSENCE OF THE DISEASE

SENSITIVITY

1- SPECIFICITY= = 23.8

LIKELIHOOD OF POSITIVE TEST

GIVEN THE DISEASE=LIKELIHOODRATIO (LR)

A Likelihood Ratio (LR) of 1.0

indicates an uninformative test (occurs when sensitivity and specificityare both 50%)

The higher the Likelihood Ratiothe better the test (other factors being equal)

0.95

0.04=

Test Alergi dengan Uji KulitLR+=23,8, artinya bila hasil uji kulitnya (+): hasil (+) ini dapat terjadi 23,8kali lebih besar terjadi pada penderita alergi dibandingkan dengan yang

bukan penderita alergi

Likelihood Ratio


35/83

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 1

PRE-TEST PROBABILITY

POST-TE

A (90%)

B (70%)

C (50%)

A : Sensitivity =Specificity = 0.9LR+ = 9.0

B : Sensitivity =Specificity = 0.7LR+ = 3.0

C : Sensitivity =

Specificity = 0.5LR+ = 1.0

P

OST-TESTPROBA

BILITY

Sensitivity & Specificity; PositivePredictive Value; Prevalence & LR


36/83

DISEASE

Yes No Total

3 7Yesa b

10a+b

c dNo 1 89 90 c+d

4 96 100

TEST

Total a+c b+d a+b+c+d

METHOD 2:

TRADITIONAL 2x2 TABLES


37/83

DISEASE

Yes No Total

3 7Yesa b

10a+b

c dNo 1 89 90 c+d

4 96 100TEST

Totala+c b+d a+b+c+d

SENSITIVITY

SENSITIVITY

The proportion of people with the diagnosis(N=4) who are correctly identified (N=3)

Sensitivity = a/(a+c) = 3/4 = 75%

FALSENEGATIVES


38/83

DISEASE

Yes No Total

3 7Yesa b

10a+b

c dNo 1 89 90 c+d

4 96 100TEST


SPECIFICITY

SPECIFICITY

The proportion of people without the diagnosis(N=96) who are correctly identified (N=89)

Specificity = d/(b+d) = 89/96 = 93%

FALSEPOSITIVES


39/83

DISEASE

Yes No Total

3 7Yes a b

10 a

+b

c dNo 1 89 90 c+d

4 96 100TEST


PRE-TEST ODDS

In the sample as a whole, the odds of having thedisease are 4 to 96 or 4% (the PRE-TEST ODDS)


40/83

DISEASE

Yes No Total

3 7Yesa b

10a+b

c dNo 1 89 90 c+d

4 96 100TEST


POST-TEST ODDS

In those who score positive on the test, the odds of havingthe disease are 3 to 7 or 43% (the POST-TEST ODDS)

In the sample as a whole, the odds of having the diseaseare 4 to 96 or 4% (the PRE-TEST ODDS)


41/83

DISEASEYes No Total

3 7Yesa b

10a+b

c d

No 1 89 90 c+d

4 96 100TES

T


POST-TEST ODDS

In those who score positive on the test, the odds of havingthe disease are 3 to 7 or 43% (the POST-TEST ODDS)

In the sample as a whole, the odds of having the disease

are 4 to 96 or 4% (the PRE-TEST ODDS)

In those who score negative on the test, the odds of having

the disease are 1 to 89 or approximately 1%


42/83

BAYES THEOREM

POST-TEST ODDS =

LIKELIHOOD RATIO x PRE-TEST ODDS


43/83

LIKELIHOOD RATIO AND PRE-

AND POST-TEST PROBABILITIES

For a given test with a givenlikelihood ratio, the post-

test probability will dependon the pre-test probability(that is, the prevalence ofthe condition in the sample

being assessed)

SENSITIVITY ANALYSIS OF


44/83


A DIAGNOSTIC TEST

Value 95% CI

Pre-testprobability

35% 26% to 44%



45/83

Applying the 95% confidence

intervals above to the nomogram,the post-test probability is likely to

lie in the range 55-85%

Value 95% CI

Pre-testprobability

35% 26% to 44%

Likelihoodratio

5.0 3.0 to 8.5


A DIAGNOSTIC TEST

The diagonal line (representing Sensitivity=0.5


46/83

RECEIVER OPERATING CHARACTERISTIC CURVE

Overall shape ispredicted by thereciprocal relationshipbetween sensitivity andspecificity

The closer the curve getsto Sensitivity=1 andSpecificity=1, the betterthe overall performanceof the test

g ( p g yand Specificity=0.5) represents performance nobetter than chance

Hence the area under thecurve gives a measure ofthe tests performance

FALSE POSITIVE RATE (1-Specificity)


47/83

0

100

1-Specificity

Sensitivit

y

AREA UNDER ROC CURVES

0

100

1-Specificity

S

ensitivity

Sensitivity and specificity both100% - TEST PERFECT

Sensitivity and specificity both50% - TEST USELESS

AREA=1.0

AREA=0.5The area under a ROCcurve will be between0.5 and 1.0


48/83

0

100

1-Specificity

Sensitivity

Area = 0.7 (between0.5 and 1.0)

Consider (hypothetically) two patients drawn

randomly from the DISEASE+ and DISEASE- groupsrespectively

If the test is used to guess which patient is from theDISEASE+ group, it will be right 70% of the time

AREA UNDER ROC CURVES

APPLYING A DIAGNOSTIC TEST


49/83

APPLYING A DIAGNOSTIC TEST

IN DIFFERENT SETTINGS

The Positive Predictive Value of a test will vary

(according to the prevalence of the condition in

the chosen setting)

Sensitivity and Specificity are usually consideredproperties of the testrather than the sett ing, and

are therefore usually considered to remain

constant

However, sensitivity and specificity are likely to

be influenced by complexity of differential

diagnoses and a multitude of other factors (cf

spectrum bias)


50/83


51/83

Diagnosis test & clinical setting


52/83




53/83



54/83

Interpreting Diagnostic Studies

VIA - RaMMbo


55/83

Validity

Selection? VALIDITYQUESTION


56/83

Participants

Index group (IG) &Gold standard

Comparison Group (CG)

Outcome

IG

C

G

+- DC

+ -BA

Representative?

VALIDITY

Reproducible

Maintain?

Measurementsblind subjective? OR

objective?

QUESTION:

Di ti A St d


57/83

Diagnostic Accuracy Study:Basic Design

Series of patients

Index test

Reference standard

Blinded cross-classification

Recruitment:


58/83

Recruitment:Was diagnostic test evaluated is representative

spectrum of patient?

Series of patients

Index test

Reference standard


Maintenance:


59/83

Maintenance:Was the endpoint of the reference standard

obtained for all subjects?

Series of patients

Index test

Reference standard


Measurement:


60/83

Measurement:Were the assesors kept blind to the results of eachtest and/or were the reference standard endpoint

objective

Series of patients

Index test

Reference standard



61/83

Selected Patients

Index test

Reference standard


Spectrum Bias


62/83

Series of patients

Index test

Reference standard


Verification Bias


63/83

Series of patients

Index test


Ref. Std A Ref. Std. B

Differential Reference Bias


64/83

Series of patients

Index test

Reference standard

Unblinded cross-classification

Observer Bias


65/83

Importance

What should I do


66/83

INTERVENTION

ETIOLOGY/RISK FACTORS

DIAGNOSIS

PROGNOSIS & PREDICTION

FREQUENCY & RATE

PHENOMENA / THOUGHTS

I

M

P

O

RT

A

NC

E

about this condition

or problem?

What causethe problem?

Does this person

have the condition

or problem?

Who will get

the condition

or problem?

How commonis the problem?

What are the

type of problem?66


67/83

CLINICAL TRIAL

PROGNOSIS

DIAGNOSTIC

RRR, ARR, NNT

p & CI

Survival curve

RR / ORp & CI

Sn,Sp,LH,PPV,NPV

p & CI

I

M

P

O

RT

A

NC

E 67


68/83

Applicability

PICO & Applicability


69/83

PICO & Applicability

Your question

(PICO)

Study What do theResult mean?

How well was

study done?

Validity

Importance

Applicability

69


70/83

Diagnostic tests


71/83

Diagnostic tests

Is not about finding absolute truth, butabout limiting uncertainty

establishes both the necessity and the

logical base for introducing probabilities,pragmatic test-treatment thresholds ..

Start thinking about

what youre going to do with the results of thediagnostic test, and

whether doing the test will help your patients


72/83


73/83

CRITICALAPPRAISALDIAGNOSTICTEST

Critical appraisal diagnostic test


74/83

Critical appraisal diagnostic test

Use worksheet (VIA; RAMMbo)

STARD

Use supporting softwares CAT Maker
http://localhost/var/www/apps/Local%20Settings/Temp/catmaker/CATMAKER.EXEhttp://localhost/var/www/apps/Local%20Settings/Temp/catmaker/CATMAKER.EXE


75/83


76/83

Validity (1)


77/83

y ( )Apakah penelitian uji diagnostik dilakukan secara tersamar dengan baku

emas yang benar ?

Validity (2)Apakah uji diagnostik dilakukan terhadap pasien dengan spektrum

penyakit atau kelainan yang memadai sehingga dapat diterapkan dalampraktek sehari-hari?

Validity (3)Apakah pemeriksaan dengan baku emas dilakukan tanpa memandanghasil pemeriksaan dengan uji diagnostik ?

Important


78/83

pBerapa Sn, Sp, LR+, LR-, PPV, NPV, Pre-test probability, Post-test

probability, Pre-test Odds, Post-test Odds ?

Applicable (1)


79/83

Applicable (1)Apakah uji diagnostik tersebut tersedia, terjangkau dan akurat?

Applicable (2)Apakah kita bisa memperkirakan pre-test probability (prevalens)penyakit pada pasien kita ?

Applicable (3)Apakah post-test probability yang dihitung akan mengubah tatalaksana

pasien kita?

Applicable (4)Apakah secara keseluruhan uji diagnostik tersebut bermanfaat bagi

pasien ?

STARD initiative (25 items)


80/83

Section and and

topic

Title, abstract, and

keywords

Introduction

Methods

Participants

Test methods

Statistical methods

Results Participants

Test results

Estimates

Discussions

STARD initiative (25 items)Standards for Reporting of Diagnostic Accuracy

Bossuyt PM, Reitsma JB, Bruns, DE, Gatsonis CA, Glasziou PP et al. BMJ 2003,326:41-6


81/83

1stcomponent of STARD


82/83

2nd component of STARD


83/83

Does early diagnosis really lead to improvedsurvival, or quality of life, or both?

Are the early diagnosed patients willing partners inthe treatment strategy?

Is the time and energy it will take us to confirm thediagnosis and provide (lifelong) care well spent?

Do the frequency and severity of the target disorderwarrant this degree of effort and expenditure?

Guides for deciding whether a screening or

early diagnostic maneuver does more goodthan harm:

ek-5 20jahghn dr.cita_dx_200111 (sri-laptop's conflicted copy 2014-01-28) (rachmi-macbook-air's...

Documents