Firma convenzione
Politecnico di Milano e Veneranda Fabbrica
del Duomo di Milano
Aula Magna – Rettorato
Mercoledì 27 maggio 2015
Neural Network approach for cathetersegmentation on Ultrasound images
Candidate: Maria Tirindelli
Supervisor: Elena De Momi
Cosupervisor: Christoph Hennersperger
Rudiger Goebl
Sara El Hadij
Nome Cognome, assoc.prof. ABC Dept.
Field of application
Brain Tumor
200000 deaths in the world per year (WHO, 2015)
Surgical
Resection
Maria Tirindelli
Low Grade High Grade
Pharmaceutical
treatmentsRadiotherapy
Nome Cognome, assoc.prof. ABC Dept.
Brain Tumor
Maria Tirindelli
Field of application
Pharmaceutical
treatmentsSurgical
ResectionRadiotherapy
Low Grade High Grade
200000 deaths in the world per year (WHO, 2015)
Nome Cognome, assoc.prof. ABC Dept.
Pharmaceutical treatments Oral or intravascular drug
administration
Limitation: Presence of the Blood-Brain barrier
Low drug concentration at the lesion
Lower effectiveness of the treatment
Maria Tirindelli
Field of application
Nome Cognome, assoc.prof. ABC Dept.
Automatic catheter insertion for
Local Drug Release
Higher drug concentration at
the lesion
Higher effectiveness of the
treatement
Catheter tracking on ultrasound volumes
Maria Tirindelli
Field of application
Nome Cognome, assoc.prof. ABC Dept.
Shadowing effects
Low Signal to Noise Ratio
Low resolution
Ultrasound data
3D in real time
Information about soft
tissues
No ionizing radiation
Maria Tirindelli
Field of application
Nome Cognome, assoc.prof. ABC Dept.
Shadowing effects
Low Signal to Noise Ratio
Low resolution
Ultrasound data
Maria Tirindelli
Field of application
3D in real time
Information about soft
tissues
No ionizing radiation
Nome Cognome, assoc.prof. ABC Dept.
Shadowing effects
Low Signal to Noise Ratio
Low resolution
Ultrasound data
Necessity of method for catheter
tracking on ultrasound volumes
Robust against noise and
artifacts
3D
Fast
Maria Tirindelli
Field of application
3D in real time
Information about soft
tissues
No ionizing radiation
Nome Cognome, assoc.prof. ABC Dept.
Biopsy needle tracking (Uhervicik et al., 2013)
Maria Tirindelli
State of the Art
Nome Cognome, assoc.prof. ABC Dept.
Biopsy needle tracking (Uhervicik et al., 2013)
Maria Tirindelli
State of the Art
Ablation catheter tracking
(Bauer et al., 2016)
Nome Cognome, assoc.prof. ABC Dept.
Biopsy needle tracking (Uhervicik et al., 2013)
(Milletari et al., 2016)
Prostate segmentation: VNet
Maria Tirindelli
State of the Art
Ablation catheter tracking
(Bauer et al., 2016)
Nome Cognome, assoc.prof. ABC Dept.
Goal of the thesis
Method for catheter tracking on
ultrasound volumes
Robust against noise and artifacts
3D
Fast
CONVOLUTIONAL NEURAL
NETWORKS
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Convolutional Neural Networks
CONVOLUTIONAL LAYER
DOWNSAMPLING ACTIVATIONDECONVOLUTIONAL
LAYER
CONVOLUTIONAL LAYER
ACTIVATION
DOWNSAMPLINGBRANCH
UPSAMPLINGBRANCH
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
*
CONVOLUTIONAL LAYER
Convolutional Neural Networks
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Convolutional Neural Networks
Maria Tirindelli
*
*
Nome Cognome, assoc.prof. ABC Dept.
Stride > 1
*
+ DOWNSAMPLING LAYER
Convolutional Neural Networks
Maria Tirindelli
*
Nome Cognome, assoc.prof. ABC Dept.
+
Convolutional Neural Networks
Maria Tirindelli
**
*
Nome Cognome, assoc.prof. ABC Dept.
+ACTIVATION
LAYER
Convolutional Neural Networks
Maria Tirindelli
**
Nome Cognome, assoc.prof. ABC Dept.
+
Convolutional Neural Networks
Maria Tirindelli
**
Nome Cognome, assoc.prof. ABC Dept.
+
Stride < 1
*
DECONVOLUTIONLAYER
Convolutional Neural Networks
Maria Tirindelli
**
Nome Cognome, assoc.prof. ABC Dept.
+
Convolutional Neural Networks
Maria Tirindelli
***
Stride < 1
*
Nome Cognome, assoc.prof. ABC Dept.
Convolutional Neural Networks
Maria Tirindelli
+ ***+*
Nome Cognome, assoc.prof. ABC Dept.
SKIP CONNECTION: concatenation
Convolutional Neural Networks
Maria Tirindelli
+ ***+*
Nome Cognome, assoc.prof. ABC Dept.
SKIP CONNECTION: concatenation
Convolutional Neural Networks
Maria Tirindelli
+ ***+*
Nome Cognome, assoc.prof. ABC Dept.
+* * * * +
SKIP CONNECTION: concatenation
CONVOLUTIONAL NEURAL NETWORK
Trainable parameters can be automatically tuned in orderto perform segmentation
TRAINABLE PARAMETERS filters’ coefficients
Convolutional Neural Networks
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
NEURAL NETWORK
Input Batch
SIGMOID LAYER LOSS FUNCTION
𝑻𝑷𝒊𝒕 = 𝑻𝑷𝒊
𝒕−𝟏 − 𝜶 ∙𝝏𝑳𝒐𝒔𝒔
𝝏𝑻𝑷𝒊
Learning rate
Convolutional Neural Networks - TRAINING
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
VNet
Maria Tirindelli
SIGMOID LAYER
Nome Cognome, assoc.prof. ABC Dept.
VNet
Maria Tirindelli
Convolutionallayer
SIGMOID LAYER
Nome Cognome, assoc.prof. ABC Dept.
VNet
Maria Tirindelli
Convolutionallayer
Short-cuts
SIGMOID LAYER
Nome Cognome, assoc.prof. ABC Dept.
VNet
Maria Tirindelli
Convolutionallayer
Downsamplinglayer
Short-cuts
SIGMOID LAYER
Nome Cognome, assoc.prof. ABC Dept.
VNet
Maria Tirindelli
Convolutionallayer
Downsamplinglayer
Short-cuts
Activationlayer
SIGMOID LAYER
Nome Cognome, assoc.prof. ABC Dept.
VNet
Maria Tirindelli
Convolutionallayer
Downsamplinglayer
Deconvolutionlayer
Short-cuts
Activationlayer
SIGMOID LAYER
Nome Cognome, assoc.prof. ABC Dept.
Workflow
Ultrasound
data
acquisition
Preprocessing
Vnet
Implementation
In tensorflow
Network training
and
fine tuning
Performance
evalutation
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Workflow
Ultrasound
data
acquisition
Preprocessing
Vnet
Implementation
In tensorflow
Network training
and
fine tuning
Performance
evalutation
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
1. Ultrasound volumes acquired with the EDEN catheter
inserted in an Agar phantom
2. Image cropping to 128 x 128 x 128
3. Mean subtraction and division by the standard deviation
• 3 dataset (Training, Validation, Test)
• 240 x 240 x 274 voxels
• Manually segmented
Data acquisition and preprocessing
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Workflow
Ultrasound
data
acquisition
Preprocessing
Vnet
Implementation
In tensorflow
Network training
and
fine tuning
Performance
evalutation
Optimized implementation
of the 3D convolution
Easier deployment of the
architecture
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Workflow
Ultrasound
data
acquisition
Preprocessing
Vnet
Implementation
In tensorflow
Network training
and
fine tuning
Performance
evalutation
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Network training and fine tuning
Batch sizeNEURAL NETWORK SIGMOID LAYER
LOSS FUNCTION
Input Batch
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Network training and fine tuning
Batch size
Activation Function
NEURAL NETWORK SIGMOID LAYERLOSS FUNCTION
Input Batch
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Network training and fine tuning
Batch size
Activation Function
Loss function
NEURAL NETWORK SIGMOID LAYERLOSS FUNCTION
Input Batch
𝑳𝒐𝒔𝒔 = 𝑷𝑾 ∙ 𝒍 ∙ 𝐥𝐨𝐠(𝒚) + 𝑵𝑾 ∙ (𝟏 − 𝒍) ∙ 𝐥𝐨𝐠(𝟏 − 𝒚)
Network outputGround Truth value
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Network training and fine tuning
Batch size
Activation Function
Loss function
Optimizer → Adam Optimizer
NEURAL NETWORK SIGMOID LAYERLOSS FUNCTION
Input Batch
𝑳𝒐𝒔𝒔 = 𝑷𝑾 ∙ 𝒍 ∙ 𝐥𝐨𝐠(𝒚) + 𝑵𝑾 ∙ (𝟏 − 𝒍) ∙ 𝐥𝐨𝐠(𝟏 − 𝒚)
Network outputGround Truth value
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Workflow
Ultrasound
data
acquisition
Preprocessing
Vnet
Implementation
In tensorflow
Network training
and
fine tuning
Performance
evalutation
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Metrics
A = segmented volumeB= Ground Truth
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Comparison with Frangi filter + SVM
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Results
Batch size Activation function
Maria Tirindelli
PreLu
tanh
Nome Cognome, assoc.prof. ABC Dept.
Results - Comparison with Frangi filter + SVM
Ground Truth FF + SVM CNN
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Results – Correlation with catheters’ depth
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Conclusion
Tensorflow implementation of VNet
Acquisition of an ultrasound dataset imaging the EDEN
catheter
Eveluation of the network with different
• Batch size
• Activation functions
Comparison with a method based on Frangi filtering + SVM
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Future work
Improve annotation quality → simultaneous CT acquisition
Train the network with the whole volume/automatic selection
of the ROI based on the previous frame and/or on the
planned trajectory
Exploit time information → multichannel CNN, RNN
(Recurrent Neural Network), Kalman filter
Network training on a more challenging dataset (animal
tissue, human brain)
Maria Tirindelli
Nome Cognome, assoc.prof. ABC Dept.
Acknowledgement
Maria Tirindelli