Chest radiology & CAD

Claudia Denkinger, MD PhD MSc DTMHHead of TB Programme,FINDFIND

19 June 2018, McGill University

The Problem – the missing millionsThe Problem – the missing millions

• Approximately 40% of all prevalent cases are not diagnosed and treatedtreated

• This is largely due to lack of2,700,000

g y• Access to TB diagnostics

& lack of testing due to lack of1 938 027

• & lack of testing due to lack of • Awareness of TB

100%72%

1,938,027

60%

1,629,9061,417,838

1,221,764

• Consideration of TB60%

53%45%

, ,

39%

1,049,237

Prevalent cases Reached TBDiagnostic centers

Diagnosed with TB Registered for treatment

Completed treatment

Recurrence-free survival

Subbaraman PLOS Med 2016 2

Why we should be considering chest X-ray (CXR)?Why we should be considering chest X-ray (CXR)?

Data from prevalenceprevalence surveys

Proportion screening symptom‐negative (X‐ray +)

Still large proportion of cases with symptoms that could be detected earlier

3

Still large proportion of cases with symptoms that could be detected earlier But typically 40−60% without reported symptoms – challenge for early detection

Slide courtesy K. Floyd, WHO

CXR has been identified as a triage and screening CXR has been identified as a triage and screening tool in WHO guidelines

• CXR IS A SENSITIVE TOOL FOR SCREENING FOR ACTIVE TB R f G id li t ti i f ti TB• Reference: Guidelines on systematic screening for active TB

• AN ABNORMAL CXR IS AN INDICATION FOR FULL DIAGNOSTIC EVALUATIONEVALUATION

• Reference: The International Standards of TB Care • CXR IS AN IMPORTANT TOOL FOR CHILDHOOD TB DIAGNOSIS

• Reference: Guidelines on childhood TB • CXR CAN IMPROVE THE EFFICIENCY OF XPERT MTB/RIF USE

• Reference: Implementation manual on Xpert MTB/RIF• Reference: Implementation manual on Xpert MTB/RIF • CXR CAN ASSIST THE DIAGNOSIS OF TB AMONG PEOPLE LIVING

WITH HIV• Reference: Consolidated guidelines on the use of ARV drugs for treating

and preventing HIV infection

WHO Factsheet http://www.who.int/tb/publications/Radiography_TB_factsheet.pdf 4

The concept of Triage testing The concept of Triage testing

• CXR is performed to evaluate an individual with signs and symptoms of TB, who is seeking care. 

• Triage use cases have also been described as patient• Triage use‐cases have also been described as patient initiated evaluations or passive case finding.

• Can improve cost‐effectiveness of diagnostic algorithm• And increase the number of patients tested

Khan ERJ 2016 5

The concept of Screening The concept of Screening

• Screening use cases are also sometimes referred to as health• Screening use‐cases are also sometimes referred to as health provider initiated screening or active case finding in certain high risk groups.g g p

• CXR is performed in individuals irrespective of whether they have symptoms and therefore can increase patients 

dtested• Can improve cost‐effectiveness of diagnostic algorithm

Khan ERJ 2016 6

Can CXR improve the of yield of Can CXR improve the of yield of screening algorithms?

Triage algorithms TP FP FN TN Cost per case

NNS

1. Cough ‐> SSM 107 100 393 99400 143 935

2 Cough > CXR > SSM 105 25 395 99475 320 9532. Cough ‐> CXR ‐> SSM 105 25 395 99475 320 953

3. Any symptom ‐> SSM 235 637 265 98863 296 426

4. Any symptom ‐> CXR ‐> SSM 230 159 270 99341 795 435

5. CXR ‐> SSM 299 498 201 99002 1842 335

6. Cough ‐> Xpert 161 50 339 99450 575 622

7. Cough ‐> CXR ‐> Xpert 158 12 342 99488 347 633

8. Any symptom ‐> Xpert 354 318 146 99182 1562 283

9. Any symptom ‐> CXR ‐> Xpert 347 80 153 99420 887 289

10 CXR > X t 451 249 49 99251 2065 222

WHO Systematic screening for active tuberculosis: principles and recommendations. 2013 7

10. CXR ‐> Xpert 451 249 49 99251 2065 222

The radiology landscape has evolved dramatically over the The radiology landscape has evolved dramatically over the past years

• CXR has become cheaper, quicker and can be done closer to where patients live (mobile CXR p (units)

• Available dedicated digital CXR systems based onAvailable dedicated digital CXR systems based on detector technology have the potential to simplify TB screeningTB screening

• The basic design of x-ray systems is similar for most manufacturersmost manufacturers

• The detector technology defines the quality and cost of a system.

FIND Digital Radiology landscape https://www.finddx.org/publication/digital‐radiology‐solutions‐for‐tb‐diagnostics‐in‐low‐to‐middle‐income‐countries/ 8

While technologies have improved –radiologists are missingWhile technologies have improved –radiologists are missing

• 100 radiologists per million people in the United Statespeople in the United States, Liberia has less than 10

di l i i fradiologists in a country of 3.5 million.

• High inter-reader variability

Can computer‐aided detectionCan computer aided detection overcome the issue with the shortage of radiologist?

Chen Lancet 2004Hussain J Glob Rad 2015Pic courtesy CAD4TB

9

CAD4TB: Computer Aided Detection for TB CAD4TB: Computer Aided Detection for TB

• What: • Software that automatically reads digital CXR imagesy g g• Software specifically trained to detect TB, currently version 5; version

6 will also include AI• Trained on thousands of X-rays from over 15 countries• Result is a score between 0-100 indicating the likelihood of TB.

A h t i t d i di ti th iti f th TB b lit• A heatmap is generated indicating the position of the TB abnormality

• Product:• CAD4TB (offline) or CAD4TBbox (cloud-based)

• Developed by: • Delft Imaging Systems & Diagnostic Image Analysis Group at

Radboud University Nijmegen Medical Center in The Netherlands

• Stage of development: CE marked, commercially available• Cost indication: $1.75 per scanCost indication: $1.75 per scan

10www.delft.care

Qure ai: qXR – Artificial intelligence for TB screening from Qure.ai: qXR – Artificial intelligence for TB screening from CXRs

• What: • Software that automatically reads digital CXR images• Software that automatically reads digital CXR images• Deep learning algorithm trained on 1.5 Million X-rays• Recognizes 15 clinically relevant CXR findingsg y g• Combination of 9 findings used to define a TB screening algorithm• Besides TB, also identifies other lung findings• Localization of suspected abnormalities are indicated• TB score: provides yes/no answer

• Product:• qXR

• Developed by: Qure.ai in India • Stage of development: CE marked commercially availableStage of development: CE marked, commercially available• Cost indication: <2 USD; price offered is dependent on volume,

geography number of sitesgeography, number of sites11www.qure.ai

Semantics: AI for TB screeningSemantics: AI for TB screening

• What: • Software that automatically reads digital CXR images• Software that automatically reads digital CXR images• Deep learning algorithm trained to recognize TB• Algorithm trained on thousands of X-rays from variety of countries (US,Algorithm trained on thousands of X rays from variety of countries (US,

China, India, Eastern Europe, South Africa)• Result is a score between 0-100 indicating the probability of TB.• Salience map is generated to visualize the pixels the algorithm considers to

be relevant for diagnosis

• Product:• AI for TB: Offline deployment as well as integrated cloud application possible

• Developed by: • Semantics in Texas

• Stage of development: not yet certified, research licence available• Cost indication: <$2 USD per patient• Cost indication: <$2 USD per patient

12www.semantic.md/ai-tb.

Other solutions on the horizon Other solutions on the horizon

Ch N tTierNet

ChexNet???? TierNet????

13

Is CAD ready for prime-time?Is CAD ready for prime-time?

• SR was performed on CAD4TB software: high sensitivity but at the cost of a low specificity for a triage use case i e sensitivities of >85% were associated withspecificity for a triage use-case, i.e. sensitivities of >85% were associated with specificities ranging from 23% to 69%; Th f i di fid l i di i• There were too few screening studies to confidently estimate diagnostic accuracy.

• Importantly, the review identified a number of limitations that could have resulted in a p y,biased assessment of diagnostic accuracy, and limited the generalisability of the findings:g

• Evidence base for recent versions, in particular, consists only of a small number of studies.• Another broad concern is that most studies have not evaluated the performance of CAD4TB when p

it is operationalised the way it will most likely to be used in the field (i.e. evaluated only on a subset of individuals who were symptomatic or in whom human readers had identified radiographs

b las abnormal

Khan ERJ 2016Pande IJTLD 2016 14

The ideal CAD solution – the CAD TPPThe ideal CAD solution – the CAD TPP

• May focus on TB but needs to identify other abnormalities that require further workup (e.g. Cardiomegaly)Cardiomegaly)

• Categorization of abnormal – likely or unlikely TB versus normal

• Thresholds of CAD should be predefined• Thresholds of CAD should be predefined • Should allow for minimally trained operators to utilize the system• Enable interpretation of images from different CXR devices • Be able to identify quality issues with the source filesBe able to identify quality issues with the source files• Should be able to transmit images for telemedicine consultations with experts if the need arises,

and do so through secure means that protect patient confidentialityand do so through secure means that protect patient confidentiality. • The interpretation should be less than 1 USD per image, which would amount to a monthly cost of

around 480USD if the software were utilised to evaluate on average 20 patients per day 6 daysaround 480USD if the software were utilised to evaluate, on average, 20 patients per day, 6 days per week.

Khan ERJ 2016 15

CAD evaluation in advance of a WHO review – ECAD projectCAD evaluation in advance of a WHO review – ECAD project

Immigration

Symptomatic screening

Immigration screeningCAD4TB 

Prevalence surveys

“CAD can be used for TB detection f h id ll f ll ifor research, ideally following a protocol that contributes to the required evidence base for guideline

Standardized panel of DICOM filesRapid comparative assessment of required evidence base for guideline

development.”p p

versions and new toolsAssessment across different use cases: triage versus screening

Qure.aitriage versus screeningReduction of biasIndependent evaluation from testIndependent evaluation from test developers (CXRs not used to train the software)Khan ERJ 2016

FIND is looking for collaboratorsFIND is looking for collaborators…

….who are willing to • contribute Chest-X-ray images and

clinical data from finalized projects toclinical data from finalized projects to the ECAD-TB project and

• Through this inform the evaluation of CAD solutions for WHO policyCAD solutions for WHO policy guidance

Info on FIND website:https://www.finddx.org/news/seeking‐collaborators‐to‐evaluate‐cad‐tools‐for‐

tb/tb/

FIND 17

CAD already in use – mostly in operational research but also in CAD already in use – mostly in operational research but also in large scale screening project

• Pakistan: 70 CAD4TB b• over 70 CAD4TB boxes

installed for screening • Plans to screen >2m

patients• Ghana:

• 51 X-ray systems in• 51 X-ray systems in hospitals, containers and screening vansscreening vans

• Utilized for screening

TB Reach; other 18

Th k YOU f li t i ! Any QUESTIONS?Thank YOU for listening!

And thank you for help with preparing the slides:• Sandra Kik, Consultant to FIND,

19