-the computational image analysis role
TRANSCRIPT
What are our digital images trying to tell us?- the Computational Image Analysis role
Dr Andrew SquelchCurtin Institute for Computation
Department of Exploration Geophysics
http://www.itqb.unl.pt/labs/bacterial-cell-biology
Cell biology –various microscopy sources e.g. FM, SIM
If only images could talk…
https://www.quora.com/What-is-the-difference-between-tablets-and-pills
Manufactured particles/objects –various image sources
https://www.zeiss.com/content/dam/Microscopy/Segments/Raw%20Materials/Oil%20and%20Gas/Rock%20Characterization/sandstone.jpg
Mineral particles/rock grains –Microscope / FIB-SEM/ microCT 2D & 3D
https://au.pinterest.com/toddsangels1/chapter-5-histology/
Histology/pathology –various imaging methods, e.g. SHG
If only images could talk…
Human interest –Digital photograph
http://www.channelinstruments.com/v_1/1249.aspx http://www.channelinstruments.com/v_1/1249.aspx
Canopy cover –Digital photograph (E&A)
http://www.stri.si.edu/sites/esp/tesp/plant_pictures/i_sp1391.mx.jpg
Plant / crop health –Scanned digital image
http://www.pnas.org/content/102/21/7760/F1.large.jpg
Motion / growth rates –Time series images
Seminar objectives
• Raise awareness about CIA
• Offer helping hand to new users of CIA
• Connect with researchers that have common interest in (some) aspects of CIA
• Canvas interest in CIA Research Group / SIG
• Cross-disciplinary CIA projects, e.g.
– Extractive metallurgy + Geophysics + Computer Science
Computational Image Analysis
“Seeing is believing : Quantifying is convincing”
“Image analysis in biology is moving from seeing and observing to quantifying
and modeling. Interpreting images as scientific measurements, rather than as
mere visualizations, brings the need for uncertainty quantification, error analysis,
statistical inference frameworks, etc. This raises a number of exciting theoretical
and algorithmic questions.”
(Sbalzarini, 2016)
Sbalzarini, I. F. 2016. Seeing is believing - quantifying is convincing: Computational image analysis in biology. W.H. De Vos et al. (eds.), Focus on Bio-Image Informatics, Advances in Anatomy, Embryology and Cell Biology 219. Springer International Publishing, Switzerland.
Computational Image Analysis
• Computer-assisted or automated analysis of digital images
• Repeatability cf manual annotation and analysis
• Batch processing large sets of digital images (inc. HPC)
• 2D, 3D and tracking in multi-channel and time-series (4D and 5D)
• Borrows from image processing, computer vision and machine learning
• Quantifying feature attributes in digital images– e.g. shape factors, size, orientation, waviness, growth, motion…
• Not always correct – need calibration/validation, error assessment
Open Source CIA software & toolkits
Fiji / ImageJ / ImageJ2https://fiji.sc/ https://imagej.net/
Icyhttp://icy.bioimageanalysis.org/
CellProfilerhttp://cellprofiler.org/
Blackspot (Python, PIL, Numpy, SciPy)https://www.ncbs.res.in/blackspot
Python (PIL, Numpy, SciPy, Scikit-image, OpenCV)https://www.python.org/
R Statistical computing languagehttps://www.r-project.org/
KNIME Analytics Platform (machine learning)https://www.knime.org/knime-analytics-platform
Many more plant analysis examples here: http://www.plant-image-analysis.org
VTK, ITK and 3D slicerhttps://www.kitware.com
Drishtihttps://github.com/nci/drishti
Commercial CIA software & toolkits
Amira (Life science) & Avizo (Materials Science)
https://www.fei.com/software/amira-avizo/
Imaris
http://www.bitplane.com/imaris/imarisMIPAR
http://www.mipar.us
Mimics
http://www.materialise.com/en/medical/software/mimics
ScanIP
https://simpleware.com/software/scanip/
MATLAB
https://au.mathworks.com/
What is Image Analysis / Quantification?
https://imagej.github.io/presentations/fiji-introduction/#/15/8
Are all clown noses the same size?
ImageJ Sample images
Are all clown noses the same size?
https://au.pinterest.com/pin/445574956854087709/
Are all clown noses the same size?
https://au.pinterest.com/pin/445574956854087709/
CIA Workflow elements
• Prepare, Identify, Analyse
CIA Workflow elements
• Prepare, Identify, Analyse
Measuring & Analysis
• Histogram
• Shape attributes
• Orientation
• Distribution
• Coherency
• Quantification
• Visualisation
Segmentation & Labelling
• Histogram
• Thresholding
• Edge detection
• Morphology operations
• Binarisation
• Fill holes
• Create mask
• Watershed
• Identify
• Labelling
• ROIs
• Tracking
Image processing
• Contrast equalization
• Background removal
• Filtering
• Denoising
• Sharpening
• Edge preserving
• Smoothing
• Registration
• Alignment
CIA Workflow elements
• Prepare, Identify, Analyse
Measuring & Analysis
• Histogram
• Shape attributes
• Orientation
• Distribution
• Coherency
• Quantification
• Visualisation
Segmentation & Labelling
• Histogram
• Thresholding
• Edge detection
• Morphology operations
• Binarisation
• Fill holes
• Create mask
• Watershed
• Identify
• Labelling
• ROIs
• Tracking
Image processing
• Contrast equalization
• Background removal
• Filtering
• Denoising
• Sharpening
• Edge preserving
• Smoothing
• Registration
• Alignment
CIA Workflow elements
• Prepare, Identify, Analyse
Measuring & Analysis
• Histogram
• Shape attributes
• Orientation
• Distribution
• Coherency
• Quantification
• Visualisation
Segmentation & Labelling
• Histogram
• Thresholding
• Edge detection
• Morphology operations
• Binarisation
• Fill holes
• Create mask
• Watershed
• Identify
• Labelling
• ROIs
• Tracking
Image processing
• Contrast equalization
• Background removal
• Filtering
• Denoising
• Sharpening
• Edge preserving
• Smoothing
• Registration
• Alignment
Workflow example
http://www.albany.edu/celltracking/
CIA examples from a few discipline areas
Plant leaf area - Blackspot
https://www.ncbs.res.in/sites/default/files/Black_Spot_User_Manual.pdf
Leaf area is calculated in square centimetres (cm2) using the number of ‘leaf’ pixels in the binary image and pixel resolution values
extracted from the image exif data
Canopy cover - Canopy Analyser
Automated Analysis of in Situ Canopy Images for the Estimation of Forest Canopy CoverKorhonen, L. and Heikkinen, J. 2009. Forest Science, Volume 55 (4), 323-334
Materials CT analysis -Avizo
https://en.wikipedia.org/wiki/Avizo_(software)
Rock porosity and permeability at scale
Threshold applied to segment microCT image (left) into binary image (right)
Liu, J., Regenauer-Lieb, K., Hines, C., Liu, K., Gaede, O. and Squelch, A. Improved estimates of percolation and anisotropic permeability from 3-D X-ray microtomography using stochastic analyses and visualization, Geochem. Geophys. Geosyst., 10, Q05010, doi:10.1029/2008GC002358. 2009.
Rock porosity and permeability at scale
Gray scale histogram –separates pores from grains
Liu, J., Regenauer-Lieb, K., Hines, C., Liu, K., Gaede, O. and Squelch, A. Improved estimates of percolation and anisotropic permeability from 3-D X-ray microtomography using stochastic analyses and visualization, Geochem. Geophys. Geosyst., 10, Q05010, doi:10.1029/2008GC002358. 2009.
Rock porosity and permeability at scale
Comparison of tensors of permeability and pore clusters
Local porosity distribution analysed for different sub-volumes: >800µm the value stabilises
Liu, J., Regenauer-Lieb, K., Hines, C., Liu, K., Gaede, O. and Squelch, A. Improved estimates of percolation and anisotropic permeability from 3-D X-ray microtomography using stochastic analyses and visualization, Geochem. Geophys. Geosyst., 10, Q05010, doi:10.1029/2008GC002358. 2009.
Fibre analysis - DiameterJ
https://imagej.net/DiameterJ
(Top) SEM image --> Segmented image --> Stylized Euclidean distance transform. (Bottom) A few of the graphs capable of being produced from data given by DiameterJ.
Nuclei size attributes – LAS X
http://www.leica-microsystems.com/products/microscope-software/details/product/leica-las-x-ls/
CellProfiler
http://cellprofiler.org/manuals/current/
Nuclei shape factor analysis
Choi, S., Wang, W., Ribeiro, A. J. S., Kalinowski, A., Gregg, S. Q., Opresko, P. L., … Dahl, K. N. (2011). Computational image analysis of nuclear morphology associated with various nuclear-specific aging disorders. Nucleus, 2(6), 570–579. http://doi.org/10.4161/nucl.2.6.17798
Nuclei shape factor analysis
Choi, S., Wang, W., Ribeiro, A. J. S., Kalinowski, A., Gregg, S. Q., Opresko, P. L., … Dahl, K. N. (2011). Computational image analysis of nuclear morphology associated with various nuclear-specific aging disorders. Nucleus, 2(6), 570–579. http://doi.org/10.4161/nucl.2.6.17798
Maize ear, cob and kernel shape attributes
Miller, N. D., Haase, N. J., Lee, J., Kaeppler, S. M., de Leon, N. and Spalding, E. P. (2017), A robust, high-throughput method for computing maize ear, cob, and kernel attributes automatically from images. Plant J, 89: 169–178. doi:10.1111/tpj.13320
Kernel size and shape analysis.(a) Example of raw image data with computed contour, kernel depth (major axis), and kernel width (minor axis) overlaid.(b) Features used in the statistical modelingapproach to finding the tip of each kernel, which produced the automated measurements of kernel area, length, and width.(c) Principal components analysis of the kernel contour data sets enabled their shapes to be described by three principal components PC1, PC2, and PC3. Changing PC1 causes the kernel to change size similarly in all directions. Changing PC2 changes eccentricity, or the ratio of the minor and major axes, while changing PC3 changes angularity of the contour.
Maize ear, cob and kernel shape attributes
Miller, N. D., Haase, N. J., Lee, J., Kaeppler, S. M., de Leon, N. and Spalding, E. P. (2017), A robust, high-throughput method for computing maize ear, cob, and kernel attributes automatically from images. Plant J, 89: 169–178. doi:10.1111/tpj.13320
Maize ear size and shape analysis.(a, b) Image processing steps convert the color image (a) to a value scale image (b).(c) After binarizing the image, a bounding box is fit to the ear object to obtain its length and maximum width.(d) Manually measured ear and cob lengths agreed almost perfectly with the values obtained automatically by the algorithm-based method.(e) Manually measured ear and cob widths agreed almost perfectly with the automated algorithm-based length measurements.(f) The ear contours determined from the binary mask (c) were subjected to principal components analysis to produce a set of three of basis vectors and the corresponding principal components PC1, PC2, and PC3. Sweeping PC1 primarily changed ear length. Sweeping PC2 primarily changed ear width, while sweeping PC3 controlled ear taper.
Identify, delineate and track stem cells
http://drexel.edu/now/archive/2015/September/CloneView/
Tracking moving / growing objects
http://www.albany.edu/celltracking/algorithms.html
https://youtu.be/WUIDQ1Ik7dw http://www.codesolorzano.com/software/CellFrost/
Root growth and traits - SmartRoot
SmartRoot: https://smartroot.github.io/
Multi-channel images
CellProfiler 2.1.0 manual: http://cellprofiler.org/getting_started/
ImageJ Sample images
Classification by supervised machine learning
http://jcs.biologists.org/content/126/24/5529
Pixel level
Cell objects
Whole images
Trainable segmentation – machine learning
http://forum.imagej.net/t/particle-segmentation/5110/4
No two workflows are the same…
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0148469
…its the results that matter
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0148469
Some examples of CIA being applied inFaculty of Science & Engineering
Examples of CIA applied in S&E projects
• Collagen fibre orientation and waviness analysis
– ImageJ / Fiji plugins and macro scripts:
• OrientationJ
• NeuronJ
• Quantification of mineral distribution in ore particles
– Avizo workflow and quantification
• Aquifer porosity analysis
– ImageJ plugins and macro scripts
Collagen fibre orientations
Wu, J.-P., Swift, B. J., Becker, T., Squelch, A., Wang, A., Zheng, Y.-C., Zhao, X., Xu, J., Xue, W., Zheng, M., Lloyd, D. and Kirk, T. B. (2017), High-resolution study of the 3D collagen fibrillary matrix of Achilles tendons without tissue labelling and dehydrating.Journal of Microscopy Volume 266, Issue 3, 2017. doi:10.1111/jmi.12537
Fibre orientations in cartilage
He, B., Wu, J. P., Chen, H. H., Kirk, T. B. and Xu, J. (2013), Elastin fibers display a versatile microfibril network in articular cartilage depending on the mechanical microenvironments. J. Orthop. Res., 31: 1345–1353. doi:10.1002/jor.22384
Fibre orientations in tendons
Wu, J.-P., Swift, B. J., Becker, T., Squelch, A., Wang, A., Zheng, Y.-C., Zhao, X., Xu, J., Xue, W., Zheng, M., Lloyd, D. and Kirk, T. B. (2017), High-resolution study of the 3D collagen fibrillary matrix of Achilles tendons without tissue labelling and dehydrating.Journal of Microscopy Volume 266, Issue 3, 2017. doi:10.1111/jmi.12537
Fibre orientations in tendons
Wu, J.-P., Swift, B. J., Becker, T., Squelch, A., Wang, A., Zheng, Y.-C., Zhao, X., Xu, J., Xue, W., Zheng, M., Lloyd, D. and Kirk, T. B. (2017), High-resolution study of the 3D collagen fibrillary matrix of Achilles tendons without tissue labelling and dehydrating.Journal of Microscopy Volume 266, Issue 3, 2017. doi:10.1111/jmi.12537
Collagen fibre waviness and global angle
Trace number Fibre length, Lf Straight line length, L0 Straightness parameter, Ps Global angle, a
N1 643.00 620.95 0.9657 29.20
N2 1029.44 987.53 0.9593 21.94
N3 1034.28 1000.69 0.9675 23.31
N4 1029.21 977.16 0.9494 26.43
N5 881.20 850.83 0.9655 14.43
N1N2N3 N4
N5
a(-) a(+)
Axial direction
Ps = L0 / Lf
Images courtesy of Mr Anas Almakhzoomi, PhD Candidate, Bioengineering Laboratory, Mechanical Engineering, Curtin University
Quantification of mineral distribution in ore particles
Rock pieces separated and labelledand size & volume fractions quantified
microCT image data
CT Image data courtesy of A/Prof Nimal Subasinghe, MEME, WASM, Curtin University. Labelled image courtesy of Yiyang Gao, Pawsey Internship final report.
Quantification of mineral distribution in ore particles
Histogram of mineralised ore particle
Histogram of waste rock particle
Mineral threshold
Total percentage of minerals vs mineral threshold
Perc
enta
ge o
f m
iner
als
in s
amp
le
Images courtesy of Yiyang Gao, Pawsey Internship final report, 2017.
Porosity estimation from microCT
Squelch, A., Harris, B. and AlMalki, M. Estimating porosity from CT scans of high permeability core plugs, ASEG Extended Abstracts 2012(1) 1 – 3, CSIRO Publishing. 2012.
Porosity estimation from microCT
ZoneWidth
(pixels)Height (pixels)
Area (pixels)
Porosity from image
analysis
Porosity from image histogram
1 670 123 82410 0.333 0.337
2 670 210 140700 0.389 0.388
3 670 158 105860 0.360 0.360
4 670 179 119930 0.394 0.393
All 670 670 448900 0.373 0.374
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 50 100
Po
rosi
ty
Grey-scale intensity
490 All
100 All
Zone 1
Zone 2
Zone 3
Zone 4
Squelch, A., Harris, B. and AlMalki, M. Estimating porosity from CT scans of high permeability core plugs, ASEG Extended Abstracts 2012(1) 1 – 3, CSIRO Publishing. 2012.
Porosity estimation from microCT
= -
Bringing images to life - 3D and lenticular prints
Pawsey compute resources
• Magnus• Athena
– Xeon Phi 7210 CPUs & Tesla P100 Pascal GPUs– Advanced Technology Cluster– Data analytics– Deep learning
• Nimbus– AMD CPUs– no GPUs (yet)– Cloud computing– Application clusters– Scientific data analytics and data workflow integration
THANK YOU!Questions / Discussion