Jost, C., Priyono, W., Bett, B., Poole, J . , Schoonman, L. , McLaws, M., Unger, F. , Mariner, J .

ASSESSING THE

ACCURACY OF A

CLINICAL OUTBREAK

DEFINITION FOR HIGHLY

PATHOGENIC AVIAN

INFLUENZA (HPAI)

�Background

�Key

concepts

�Methods

�Results

�Discussion

OUTLINE

�Emerged in

Guangdong Province,

China

�First detected in Hong

Kong, 1997

�Spread throughout

Asia, Europe, the

Middle East and

Africa

TYPE A H5N1 HPAI

(Sims, 2007; Morris and Jackson, 2006)

�First H5N1 outbreak 2003

�First reported to OIE in Feb 2004

�Confirmed in 31 of 33 provinces

H5N1 HPAI IN INDONESIA

(Azar et al., 2010)

Photo credit: Peter Roeder

� A set of cl inical criteria (symptoms, other epidemiological

parameters) for detecting diseased individuals or populations

� Indonesia MoA PDSR HPAI surveillance

� Operational Research in Indonesia for Highly Pathogenic Avian

Influenza (ORIHPAI)

� Early Detection, Response and Surveillance for Avian Influenza in

Africa (EDRSAIA)

� H7N7 Netherlands backyard chickens (Elbers, Koch and Bouma, 2005)

� Increased mortality or swollen head were the most sensitive clinical indicators (sensitivity 100%, specificity 20-32%)

� Addition of cyanosis increased the specificity of a diganosis (sensitivity 65-100%, specificity 68-80%)

CLINICAL CASE DEFINITION

� Accuracy: Proportion of subjects Accuracy: Proportion of subjects Accuracy: Proportion of subjects Accuracy: Proportion of subjects

correctly identified by the testcorrectly identified by the testcorrectly identified by the testcorrectly identified by the test

� Sensitivity (Se): Sensitivity (Se): Sensitivity (Se): Sensitivity (Se): Probability that a Probability that a Probability that a Probability that a

test correctly identifies infected test correctly identifies infected test correctly identifies infected test correctly identifies infected

subjects as positivesubjects as positivesubjects as positivesubjects as positive

� Specificity (Specificity (Specificity (Specificity (SpSpSpSp): ) : ) : ) : Probability that Probability that Probability that Probability that

a test correctly identifies NONa test correctly identifies NONa test correctly identifies NONa test correctly identifies NON----

infected subjects as negativeinfected subjects as negativeinfected subjects as negativeinfected subjects as negative

� Predictive value positive (PV+)Predictive value positive (PV+)Predictive value positive (PV+)Predictive value positive (PV+): : : :

Proportion of testProportion of testProportion of testProportion of test ----positive positive positive positive

subjects that are truly infectedsubjects that are truly infectedsubjects that are truly infectedsubjects that are truly infected

� Predictive value negative (PVPredictive value negative (PVPredictive value negative (PVPredictive value negative (PV---- ) : ) : ) : ) :

Proportion of testProportion of testProportion of testProportion of test ----negative negative negative negative

subjects that are truly not subjects that are truly not subjects that are truly not subjects that are truly not

infectedinfectedinfectedinfected

InfectedInfectedInfectedInfected Not Not Not Not

InfectedInfectedInfectedInfected

Test Test Test Test

PositivePositivePositivePositive

True

Positive

False

Positive

Test Test Test Test

NegativeNegativeNegativeNegative

False

Negative

True

Negative

DIAGNOSTIC ACCURACY

(Zepeda, pers. com.)

� Sample Frame

� 16 districts West and Central Java

� Level of assessment RT (lowest geopolitical unit in Indonesia)

� Routine veterinary officer visits or in response to disease report

� Vaccinated and unvaccinated areas

METHODS

� Outbreak case definition for HPAI in backyard and small-scale

commercial chicken flocks in Indonesia

� Step 1: High mortality rate (>80%), poultry dying within 12 hours of the onset of clinical signs = Sudden Death

� Step 2:

� Per acute death (or death within 4 hours), involvement of more than one household, with or without blue discoloration of the head and body = HPAI

� As above but death in >4 hours, with or without recovery of some birds = ND

METHODS

Population:

Sick Chickens

Level 1

Sudden Death

Level 2

HPAI-compatible

Level 2

ND-compatible

Level 2

Unknown

Level 1

Not Sudden Death

� Biological Sampling

� Tracheal samples from up to 25 affected chickens

� Standard World Health Organization (WHO) H5N1 sampling protocols

� Pooled 5 swabs per universal transport medium (UTM) tube containing antibiotics

� Maintained at 4°C until delivered to lab

� Stored at -80°C in lab

� Analysis (Wates DIC)

� Virus isolation, amplification and typing using Indonesia H5N1 and ND specific antibodies (Pusvetma)

� Up to 4 passages

� Outbreak considered positive if at least 1 tube positive

METHODS

� N = 297 outbreaks

� Average 18.6+7.8 outbreaks investigated per district

� 11.4% from vaccinated areas, 88.6% unvaccinated areas

� Sudden death diagnosed in 70.4% of outbreaks

� 77.0% were further diagnosed a HPAI-compatible

� 14.8% as VVND-compatible

� 8.1% as unknown

RESULTS

� Frequentist calculations

� Sudden death: Se 76.4%+5.9%, Sp 41.8+9.8%, PV+ 72.7+6.0%

� HPAI-compatible: Se 71.2+7.3%, Sp 62.3+7.7%, PV+ 64.6+7.4%

� Series diagnoses

� HPAI-compatible: Se 54.4+8.1%, Sp 78.0+6.6%

� VVND-compatible: Se 10.0+7.5%, Sp 94.3+3.0%

RESULTS

� Bayesian analysis HPAI (no gold standard)

� Case definition HPAI-compatible: Se 85.1+4.3%, Sp 78.2+4.5% PV+75.1+6.0%

� H5N1 virus isolation and typing: Se 88.1+4.4%, Sp 91.0+3.1%, PV+ 88.6+4.1%

� True prevalence H5N1 43.8+4.6%

� Bayesian analysis VVND (no gold standard)

� Case definition VVND-compatible: Se 69.1+10.6%, Sp 89.2+2.0%, PV+ 22.5+7.7%

� ND virus isolation and typing: Se 87.6+4.9%, Sp 90.9+1.9%, PV+ 30.8+10.7%

� True prevalence ND 4.5+1.7

RESULTS

� prevalence, � PV+, � PV-

� PV+, � Sp � PV-, � Se

DISCUSSION

PV +

Specificity %

PV -

Sensitivity % (Zepeda, pers. com.)

Predictive

value %

Prevalence %

� Bayesian analysis likely provides a more realistic estimation

of the accuracy of the level 2 clinical outbreak case

definitions

� Gold standard?

� Sensitivities of sudden death and HPAI-compatible diagnoses

within acceptable ranges reported in the literature for clinical

case definitions, but…

� For a study of the impact of control measures on disease

incidence, higher sensitivities would have been more

desirable

DISCUSSION

� For surveillance:

� Use a test with high Se and high PV- to: � Reduce the number of false negatives

� Avoid the introduction of a disease

� Use a test with high Sp and high PV+ to: � Confirm a diagnosis

� Avoid the unnecessary slaughter of animals

� Therefore, our HPAI case definition:

� Best used to avoid the introduction of a disease

DISCUSSION

� Test in parallel when:Test in parallel when:Test in parallel when:Test in parallel when:

� You wish to increase Se and PVYou wish to increase Se and PVYou wish to increase Se and PVYou wish to increase Se and PV----

� All tests must be negativeAll tests must be negativeAll tests must be negativeAll tests must be negative

� Test in series when:Test in series when:Test in series when:Test in series when:

� You wish to increase You wish to increase You wish to increase You wish to increase SpSpSpSp and PV+and PV+and PV+and PV+

� All tests must be positiveAll tests must be positiveAll tests must be positiveAll tests must be positive

� OOOOur HPAI case definition:ur HPAI case definition:ur HPAI case definition:ur HPAI case definition:

� Was applied in series (levels 1 and 2)Was applied in series (levels 1 and 2)Was applied in series (levels 1 and 2)Was applied in series (levels 1 and 2)

� Increased the specificity of the diagnosisIncreased the specificity of the diagnosisIncreased the specificity of the diagnosisIncreased the specificity of the diagnosis

� Sensitivity could be increased by removing level 1 of the diagnosisSensitivity could be increased by removing level 1 of the diagnosisSensitivity could be increased by removing level 1 of the diagnosisSensitivity could be increased by removing level 1 of the diagnosis

DISCUSSION

�Diagnostic tools that rely on antigen detection are only useful in active cases

�For acute viral infections of short duration (HPAI), studies based on antibody detection require prohibitively large sample sizes

�Because very few chickens survive and go on to develop antibodies

�Therefore, case definitions are an important diagnostic tool for surveillance and research of acute viral infections

LESSONS LEARNT

� Case definitions should be designed with the objective in

mind

� 100% accuracy does not exist!

� Estimating the diagnostic accuracy of the case definition used

is important for interpreting study results

LESSONS LEARNT

� Indonesia MoA

� Participating district VS

� Wates DIC

� ILRI

ACKNOWLEDGEMENTS