Evolution of Computer Evolution of Computer assisted diagnosis and assisted diagnosis and automation in Medical automation in Medical

PracticePractice

Dr Genevieve Warner Learmonth,Dr Genevieve Warner Learmonth,Honorary Senior Lecturer,Honorary Senior Lecturer,

Clinical Laboratory Sciences, Clinical Laboratory Sciences, University of Cape TownUniversity of Cape Town

History and developmentHistory and development

• Coulter Counter machine introduced in Coulter Counter machine introduced in 1968 to count erythrocytes (red blood 1968 to count erythrocytes (red blood cells)cells)

• In 1980’s attempts were made in UK to In 1980’s attempts were made in UK to develop a machine to detect develop a machine to detect abnormal abnormal cellscells ( and organisms) on pap smears. Led ( and organisms) on pap smears. Led by Dr Nassiem Husain.by Dr Nassiem Husain.

• Computer technology not advanced Computer technology not advanced enough at that time.enough at that time.

Neural Networks in image Neural Networks in image recognitionrecognition• In 1990’s neural networks were In 1990’s neural networks were

regarded as being the solution regarded as being the solution to the problem of image to the problem of image recognition. recognition.

• Mark Rutenberg at NASA was Mark Rutenberg at NASA was working at surveillance of the working at surveillance of the night skies to intercept SCUD night skies to intercept SCUD missiles in Gulf Warmissiles in Gulf War

• Dr Laurie Mango, a cytologist, Dr Laurie Mango, a cytologist, involved in possible automation involved in possible automation in diagnosis sat beside him at a in diagnosis sat beside him at a dinner party. dinner party.

• They discovered that they were They discovered that they were doing the same thing at doing the same thing at different magnitude ! different magnitude !

• PapNet was conceived and born. PapNet was conceived and born.

PapNetPapNet

• When I was asked to When I was asked to present Clinical trial present Clinical trial results for PapNet in results for PapNet in 1996, I learned about 1996, I learned about the use of neural the use of neural networks in image networks in image recognition, at Bristol recognition, at Bristol University Dept of University Dept of Engineering and Engineering and Mathematics.Mathematics.

• My interest was in image My interest was in image recognition of cells and recognition of cells and bacteria/ fungi bacteria/ fungi

SupportSupport

• I phoned PapNet in the I phoned PapNet in the USA, asked them if they USA, asked them if they could use the PapNet could use the PapNet machine to find TB machine to find TB bacilli on ZN stained bacilli on ZN stained sputum. They agreed, sputum. They agreed, but they were but they were disinclined to “waste disinclined to “waste time on it” as all their time on it” as all their funding was dedicated funding was dedicated to cervical Pap smear to cervical Pap smear screening.screening.

• My son proposed that My son proposed that that one day it would that one day it would be possible to be possible to recognise malaria, recognise malaria, leprosy, leprosy, meningococcus, meningococcus, organisms in drinking organisms in drinking water, fungi etc. using water, fungi etc. using similar computer similar computer imaging methodsimaging methods ! !

TB or not TB ?TB or not TB ?• Suddenly I realised Suddenly I realised

that as TB bacilli as that as TB bacilli as they were unique in they were unique in shape, size and shape, size and staining properties, staining properties, (literally a case of (literally a case of “TB or not TB”), “TB or not TB”), neural networks neural networks could be applied to could be applied to the search for TB the search for TB bacillibacilli

Then I met Kostas Veropoulos, a mathematical computer scientist who was “ looking for a project”

Mobile TB Diagnosis in South AfricaMobile TB Diagnosis in South AfricaVan / Lab visits peri urban areas. Van / Lab visits peri urban areas. Funded by Pick ‘n PayFunded by Pick ‘n Pay

13

Diagnosing TB

Diagnosis not easy due to confusion of clinical symptoms with other conditions.

Traditional diagnostic tests: microscopic examination of sputum smears

– most reliable way for rapid detection of infectious cases– allows diagnosis the same day– allows drug therapy to be started immediately

culturing specimens– increase the level of positive diagnosis– time consuming (takes 2-8 weeks) – not recommended for high

incidence rate areas

chest X-rays

The search for bacilli on The search for bacilli on conventional sputum smearsconventional sputum smears

• BoringBoring

• Labour intensiveLabour intensive

• Requires experienced Requires experienced

senior laboratory scientistssenior laboratory scientists

• High False Negative RateHigh False Negative Rate

• Low morale Low morale

• Recruitment of screening staff difficultRecruitment of screening staff difficult

• Remuneration of staffRemuneration of staff

The Project The Project

The slides, were examined by GML at the digital The slides, were examined by GML at the digital microscope with KV using digital imaging facility at microscope with KV using digital imaging facility at Bristol University in 1997. Bristol University in 1997.

5,000 individual bacilli were identified and images 5,000 individual bacilli were identified and images captured.captured.

Image processing performed.Image processing performed. LearningLearning Then system was challenged with 100 more images Then system was challenged with 100 more images

of typical TB bacilli.of typical TB bacilli. No funding available ---- no interest in TB.No funding available ---- no interest in TB.

23

The TB Project

Initiated in Computational Intelligence Group, Dept. of Engineering Mathematics, University of Bristol by Dr. GenevièveLearmonth, Dr. Konstantinos Veropoulos and Dr. Colin Campbell

Objective: to develop a semi-automated method for identification of tubercle

bacilli from sputum smears Method:

use of light/fluorescence microscopy and digital camera for capturing images of ZN/Auraminestained bacilli

use of image processing to enhance and extract features use of classification methods to identify TB bacilli

– Artificial Neural Networks (Back Propagation, Scaled Conjugate Gradient, Radial Basis Functions)

– Support Vector Machines (Kernel Adatron, Sequential Minimal Optimization)

– Statistical Methods (Minimum Distance, k-Nearest Neighbours)

Morphology of and types of TB Morphology of and types of TB bacillibacilli

Mycobacterium Tuberculosis(2–4μm x 0.3–0.5μm)

Mycobacterium bovis(1.5–1.9μm)

Mycobacterium avium(1.0–1.8μm x 0.5μm)

Mycobact. kansasii(usually 4–5μm x

0.3μm)

Test Example: ZN stained slideTest Example: ZN stained slide(sputum smear at 40x magnification after image (sputum smear at 40x magnification after image processing)processing)

Image processingImage processing

Test Example: Auramine stained slideTest Example: Auramine stained slide(sputum smear at 6(sputum smear at 63300x magnification after image x magnification after image

processing)processing) EDGE DETECTIONEDGE DETECTION

Results --- Results --- specificity for a single bacillusspecificity for a single bacillus

Classifier Accuracy (%)

overall sensitivity specificity

BP (10 hu) 87.02 87.11 86.90

SCG (7 hu) 87.50 91.20 82.69

KA (σ=0.4, C=1.0) 87.00 93.89 79.74

SMO (σ=0.4) 84.54 83.58 85.71

Auramine stain

Classifier Accuracy (%)

overall sensitivity specificity

BP (10 hu) 87.02 87.11 86.90

SCG (7 hu) 87.50 91.20 82.69

KA (σ=0.4, C=1.0) 87.00 93.89 79.74

SMO (σ=0.4) 84.54 83.58 85.71

ZN stain

ConclusionsConclusions

• Computer assisted diagnosis TB feasible Computer assisted diagnosis TB feasible with very encouraging resultswith very encouraging results

• Radical improvement in accuracyRadical improvement in accuracy– recognition of single bacillus > 80%, thus recognition of single bacillus > 80%, thus

recognition of multiple bacilli in a smear recognition of multiple bacilli in a smear yields a very high diagnostic accuracyyields a very high diagnostic accuracy

Cost EffectivenessCost Effectiveness

• Cost effective for population screeningCost effective for population screening– Cape Town: over 8,000 slides/month screened at Cape Town: over 8,000 slides/month screened at

$2/slide$2/slide– assuming 2 systems processing 200 slides/day each, assuming 2 systems processing 200 slides/day each,

click charge $25, estimated cost of machine at $20K, click charge $25, estimated cost of machine at $20K, then annual saving is 75% for client (then annual saving is 75% for client (estimation based estimation based on year 2000 priceson year 2000 prices))

– with computer components becoming cheaper monthly, with computer components becoming cheaper monthly, the above figures could change dramaticallythe above figures could change dramatically

• Socio-economically effectiveSocio-economically effective– disease more efficiently contained in adult populationdisease more efficiently contained in adult population

• 1 billion TB tests annually1 billion TB tests annually

FundingFunding

• TB is a disease of TB is a disease of povertypoverty

• Highly contagious Highly contagious airborne diseaseairborne disease

• Attacks Attacks immunosuppressed immunosuppressed personspersons

• Infancy, puberty, Infancy, puberty, pregnancy, illness, old pregnancy, illness, old ageage

• HIV infected persons are HIV infected persons are particularly susceptible particularly susceptible --- --- The Terrible TwinsThe Terrible Twins

9

Estimated TB Incidence Rates, 2000Source: Global Tuberculosis Report, WHO Report 2002

Presentations at CongressPresentations at Congress• Presented in Chicago at Analytical Presented in Chicago at Analytical

and Quantitative Cytology 1997and Quantitative Cytology 1997

• Presented in Hong Kong Presented in Hong Kong Cytopathology Congress 1998Cytopathology Congress 1998

Automated Identification of Tubercle Bacilli in Sputum. Veropoulos K, Learmonth G, Campbell C, Knight B, Simpson J, Journal of Analytical & Quantitative Cytology and Histology 1999 Vol.21, 4, 277 – 281

Publications and World wide Publications and World wide presentationspresentations• The Automated The Automated

identification of Tubercle identification of Tubercle bacilli using Image bacilli using Image Processing and Neural Processing and Neural Computing techniques.Computing techniques.

• K Veropoulos, C Campbell, K Veropoulos, C Campbell, G. Learmonth, B. Knight, J. G. Learmonth, B. Knight, J. Simpson, Simpson,

• Perspectives in Neural Perspectives in Neural Computing: Computing: ProceedingsProceedings of the 8th International of the 8th International Conference on Artificial Conference on Artificial Neural Networks, Skovde Neural Networks, Skovde Sweden. Sweden. 2-4 September 2-4 September 19981998. Volume 2 pp 797 - . Volume 2 pp 797 - 802. 802.

Springer Springer ISBN 3 540 76263 9ISBN 3 540 76263 9

• Image processing and Image processing and neural computing used in neural computing used in the diagnosis of the diagnosis of tuberculosistuberculosis

• Veropoulos, K. Campbell, Veropoulos, K. Campbell, C. Learmonth, G. C. Learmonth, G. Fac. of Eng., Bristol Univ.; Fac. of Eng., Bristol Univ.; 20 Oct 199820 Oct 1998

• Intelligent Methods in Intelligent Methods in Healthcare and Medical Healthcare and Medical Applications (Digest No. Applications (Digest No. 1998/514), IEE 1998/514), IEE Colloquium onColloquium on page(s): page(s): 8/1-8/4York, UK References 8/1-8/4York, UK References Cited: 10 INSPEC Accession Cited: 10 INSPEC Accession Number: 6128756 Number: 6128756

Lack of Interest and Funding in Lack of Interest and Funding in TB epidemics --- 1997/1998TB epidemics --- 1997/1998

• ““TB is not a problem in the UK or USA”TB is not a problem in the UK or USA”• Pharmaceutical Companies viewed any Pharmaceutical Companies viewed any

attempt to curtail the disease by rapid attempt to curtail the disease by rapid diagnosis/ identification of TB infected diagnosis/ identification of TB infected persons as a threat to their business of persons as a threat to their business of R&D of new anti TB drugs.R&D of new anti TB drugs.

• Very few foresaw the danger of TB Very few foresaw the danger of TB spreading hand in hand with the spreading hand in hand with the Immunosuppressed victims of HIV/AIDS Immunosuppressed victims of HIV/AIDS

• THE TERRIBLE TWINSTHE TERRIBLE TWINS

Futuristic Visions for Futuristic Visions for Computerised DiagnosisComputerised Diagnosis

Finding the bacillusFinding the bacillus

• Microscope:Microscope:

• SimpleSimple

• Purpose BuiltPurpose Built

• InexpensiveInexpensive

• PortablePortable

• User FriendlyUser Friendly

• Microscopes have not Microscopes have not changed much since van changed much since van Leeuwenhoek persuaded Leeuwenhoek persuaded Vermeer to use one in 1642 !Vermeer to use one in 1642 !

• The most commonly used The most commonly used microscope in labs today are microscope in labs today are expensive, complicated and expensive, complicated and have too many gadgets.have too many gadgets.

• For screening sputum for For screening sputum for bacilli a very simple bacilli a very simple microscope is required.microscope is required.

15

Typical configuration of a micrograph image analysis system

With a smart camera a computer system might not be neededreducing considerably the cost of an automated system

Computing and Image Computing and Image processingprocessing

• User FriendlyUser Friendly

• PC /LaptopPC /Laptop

• SimputerSimputer

• Hand Held Computer eg BlackberryHand Held Computer eg Blackberry

Image CaptureImage Capture and Global and Global TransmissionTransmission

• ““Smart” cameraSmart” camera• CoolscopeCoolscope• Built in Digital CameraBuilt in Digital Camera

• TRANSMISSION of IMAGES 1998 TRANSMISSION of IMAGES 1998 from home computer in Bristolfrom home computer in Bristol• Project Project “ Cybercytology”“ Cybercytology” with with • colleagues in Bangladeshcolleagues in Bangladesh• Global transmission of ImagesGlobal transmission of Images

• Cyber DiagnosisCyber Diagnosis• 22ndnd Opinion Opinion

What happened ?What happened ?

• No further work was No further work was done on this TB done on this TB project. project.

• KV awarded a PhD at KV awarded a PhD at Bristol University 1999Bristol University 1999

• KV went to the USA … KV went to the USA … University of Reno University of Reno

• Best Entrepreneur of the Best Entrepreneur of the Year award at Bristol Year award at Bristol University in 2001 to University in 2001 to GML and KVGML and KV

• In February 2006, a In February 2006, a phone call from NASA phone call from NASA

• Our work, found in their Our work, found in their archives; “would I assist archives; “would I assist them in driving this them in driving this initiative forward”initiative forward”

• The company The company Interscopic was born. Interscopic was born. The InterscopeThe Interscope was was conceived over Internet conceived over Internet calls and fertilised in calls and fertilised in CyberspaceCyberspace!!

The Future has arrivedThe Future has arrived

Welcome to the Welcome to the InterscopeInterscope