Automated Segmentation and Classification of Zebrafish Histology

Images forHigh-Throughput

PhenotypingBrian Canada

Academic Computing Fellow and PhD Candidate in Integrative Biosciences

Jake Gittlen Cancer Research Institute | Penn State College of Medicine

October 22, 2007

The zebrafish (danio rerio): A powerful functional genomics

tool

• Vertebrate• Develop tumors• Hundreds of eggs per clutch• Rapid, ex vivo development

– Most organ systems differentiated before 7 days post-fertilization

• Transparent embryos• Reverse genetics

– Morpholinos for gene “knock-down”

Zebrafish histology

Adult zebrafish (sagittal plane view) with

papilloma

Zebrafish larval array hht mutant 7dpf (days post-fertilization)

“High-Throughput”Zebrafish Histology

FixationEmbedding in agarose

Processing into paraffin

Sectioning, staining, mounting onto slides

ScanningDigitizationScoring and Annotation

The “rate-limiting step”

• What can be done to improve the speed and reliability of scoring images?

• Can we score abnormalities quantitatively?

Current efforts in automated zebrafish image

analysis• Stephen T.C. Wong and

colleagues at Harvard developed methods for quantitative assessment of neuron loss and automated detection of somites

• In principle, such automated methods should be scalable to allow high-throughput phenotyping

Liu T.L., “A quantitative zebrafish phenotyping tool for developmental biology and disease modeling ,” IEEE Signal Processing Magazine, Jan 2007.

Detection of Rohon-Beard sensory neuronsDetection of Rohon-Beard sensory neurons

Retinal cell detection for studying neurogenesis

Building on interdisciplinary expertise

Keith Cheng, MD, PhDZebrafish Functional

Genomics

James Z. Wang, PhDContent-Based Image Retrieval,

Automatic Image Annotation

SHIRAZ: System for Histological Image Retrieval

and Annotation for Zoopathology

[ ]IPL_Compactness = 9.8137IPL_Eccentricity = 0.9019

IPL_Solidity = 0.3086IPL_Contrast = 0.9375

IPL_Homogeneity = 0.0093LENS_COMPACTNESS = 1.1262LENS_eccentricity = 0.3530

……

Image segmentation Extract feature vector for each image

Repeat for allimages in database

ImagePre-processing

Use feature database to train model for image classification

(K-means clustering, Classification trees, Support Vector Machine, etc.)

Automatically classify

and annotate previously uncharacterized images

Creation ofVirtual Slides

SHIRAZ: System for Histological Image Retrieval

and Annotation for Zoopathology

• Prototype implemented in MATLAB for segmentation and classification of eye and gut images– Eye and gut tissues have a polar

or directional organization that is deformed or disrupted on mutation

• To our knowledge, we are the first group to publish material on automated zebrafish histology image analysis

– Canada, B.A., Thomas, G.K., Cheng ,K.C., Wang, J.Z., “Automated Segmentation and Classification of Zebrafish Histology Images For High-Throughput Phenotyping,” Proc IEEE-NIH Life Science Systems And

Applications (LISSA) Workshop 2007

Image pre-processing

Aperio T2 Scanner for Creation of

Virtual Slides

(120 slide capacity)

Manually crop eye and gut images from

selected larvae

To reduce computational costs, convert to grayscale 512 x 512

matrix (pad with white pixels if needed)

Take snapshot of selected

H&E-stained specimens in ImageScope

Example of wild-typeeye segmentation

Lens Ganglion Cell Layer (GCL)

Inner Plexiform Layer (IPL)

Inner Nuclear Layer (INL)

PhotoreceptorLayer (PRL)

Retinal Pigmented Epithelium (RPE)

Example of mutant eye segmentation

Eye feature extraction

• Filled area• Perimeter• Compactness• Eccentricity• Extent• Solidity• Fractal dimension

• Seven moment invariants

• Four gray level co-occurrence features: – Contrast – Correlation – Energy– Homogeneity

Yields vector of 92 features per eye image

Gut segmentationand feature extraction

30 features extracted per gut image, e.g.:

• Thickness and shape of the epithelial lining

• Polarity of the epithelial cells (position of nuclei relative to basement membrane)

• Number of distinct villi (folds) of the lumen

• Amount and “granularity” of cellular debris and mucous in lumen

Epithelial lining

Lumen Cell nuclei

Classification algorithm: CART (Classification And Regression

Trees)• Advantages:

– “White-box” model– Helps provide a sense

of objectivity and direction to histological assessment

• Disadvantages:– May not be as accurate

as other classification methods (e.g. SVM, GMM, ANN)

– “Splits” can only be performed on one dimension at a time (not really a problem in this case)

Preliminary Results

# of classes

Eye Images (n=79)

Gut Images (n=87)

10-fold CV

Leaveone out

10-fold CV

Leaveone out

Binary 90% 87% 86% 86%

Three classes

85% 85% 72% 71%

Five classes

72% 70% 56% 55%

Discussion and Conclusions

• Preliminary results are encouraging• Potential opportunities for improvement:

– Analyze different larval ages separately – Improve segmentation accuracy– Use color images instead of grayscale– Experiment with different classifiers (SVM, for example)– Minimize manual preprocessing– Increase overall size of datasets

• Future: – Direct integration into laboratory pipeline– Parallel image processing for higher throughput– Automatic image annotation and retrieval

Current collaborators

• Georgia Thomas, Graduate Student

• Keith Cheng, co-PI (Functional Genomics)

• James Z. Wang, co-PI (Info Science & Tech)

• Prof. Yanxi Liu (PSU Computer Science

dept.)

• Prof. Nancy Hopkins (MIT)