image analysiscourses.compute.dtu.dk/02502/presentations/02502 - week6.pdf · 2019-10-08 · – 20...
Post on 20-Mar-2020
1 Views
Preview:
TRANSCRIPT
-
DTU Compute
Image Analysis
Rasmus R. PaulsenTim B. DyrbyDTU Compute
rapa@dtu.dk
http://www.compute.dtu.dk/courses/02502
mailto:rrp@imm.dtu.dk
-
DTU Compute
2019Image Analysis2 DTU Compute, Technical University of Denmark
Lecture 6 – BLOB analysis and feature based classification
-
DTU Compute
2019Image Analysis3 DTU Compute, Technical University of Denmark
What can you do after today?
Calculate the connected components of a binary image. Both using 4-connected and 8-connected neighbours
Compute BLOB features including area, bounding box ratio, perimeter, center of mass, circularity, and compactness
Describe a feature space Compute blob feature distances in feature space Classify binary objects based on their blob features Estimate feature value ranges using annotated training data Compute a confusion matrix Compute rates from a confusion matrix including sensitivity,
specificity and accuracy Determine and discuss what is the importance of sensitivity and
specificity given an image analysis problem
-
DTU Compute
2019Image Analysis4 DTU Compute, Technical University of Denmark
Object recognition Recognise objects in images Put them into different classes
-
DTU Compute
2019Image Analysis6 DTU Compute, Technical University of Denmark
BLOB – what is it? BLOB = Binary Large
Object– Group of connected
pixels BLOB Analysis
– Connected component analysis
– Object labelling
.
.
.
-
DTU Compute
2019Image Analysis7 DTU Compute, Technical University of Denmark
Isolating a BLOB What we want:
– For each object in the image, a list with its pixels
How do we get that?– Connected component analysis
Connectivity– Who are my neighbors?– 4-connected– 8-connected
4-connected
8-connected Image
-
DTU Compute
2019Image Analysis8 DTU Compute, Technical University of Denmark
Connected component analysis Binary image Seed point: where do we
start? Grassfire concept
– Delete (burn) the pixels we visit
– Visit all connected (4 or 8) neighbors
34
12
4-connected
(1,1) x
Y
-
DTU Compute
2019Image Analysis9 DTU Compute, Technical University of Denmark
BLOBs 4- and 8-neighbours
A) 3 and 7B) 9 and 5C) 8 and 6D) 7 and 5E) 4 and 5
1 1
37
3 1
A B C D E
A BLOB analysis is performed using both 4- and 8-connectivity. How many BLOBS are found using the two different connectivities?
Chart1
A
B
C
D
E
1
1
37
3
1
Sheet1
A1
B1
C37
D3
E1
-
DTU Compute
2019Image Analysis11 DTU Compute, Technical University of Denmark
The result of connected component analysis An image where each BLOB
(component) is labelled Each blob now has a unique ID
number What do we do with these blobs?
1
2
3
…
-
DTU Compute
2019Image Analysis12 DTU Compute, Technical University of Denmark
Features Feature
– A prominent or distinctive aspect, quality, or characteristic
– This radio has many good features
Car (Ford-T) features– 4 wheels– 2 doors– 540 kg– 20 hp
http://upload.wikimedia.org/wikipedia/commons/1/15/Late_model_Ford_Model_T.jpg
-
DTU Compute
2019Image Analysis13 DTU Compute, Technical University of Denmark
Feature vector Feature vector
– Vector with all the features for one object Ford-T features
– 4 wheels– 2 doors– 540 kg– 20 hp
Ford Fiesta features– 4 wheels– 3 doors– 1100 kg– 90 hp
f=[4, 2, 540, 20]
f=[4, 3, 1100, 90]
http://upload.wikimedia.org/wikipedia/commons/1/15/Late_model_Ford_Model_T.jpghttp://en.wikipedia.org/wiki/File:Ford_Fiesta_2003_RF_14dec2006.jpg
-
DTU Compute
2019Image Analysis14 DTU Compute, Technical University of Denmark
Feature extractions Compute features for each BLOB that can be used to
identify it– Size– Shape– Position
From image operations to mathematical operations – Input: a list of pixel positions– Output: Feature vector
First step: remove invalid BLOBS – too small or big- using morphological
operations for example– border BLOBs
Feature vector = [2,1,…,3]
Feature vector = [4,7,…,0]
-
DTU Compute
2019Image Analysis15 DTU Compute, Technical University of Denmark
BLOB Features Area
– number of pixels in the BLOB– Can be used to remove noise (small
BLOBS)
One BLOB
-
DTU Compute
2019Image Analysis16 DTU Compute, Technical University of Denmark
BLOB Features Bounding box
– Minimum rectangle that contains the BLOB
– Height: 𝑦𝑦max − 𝑦𝑦min– Width: 𝑥𝑥max − 𝑥𝑥min
– Bounding box ratio:
– tells if the BLOB is elongatedOne BLOB
𝑦𝑦max − 𝑦𝑦min𝑥𝑥max − 𝑥𝑥min(𝑥𝑥max,𝑦𝑦max)
(𝑥𝑥min,𝑦𝑦min)
-
DTU Compute
2019Image Analysis17 DTU Compute, Technical University of Denmark
BLOB Features Bounding box
– Bounding box area:
– Compactness of BLOB
One BLOB
(𝑥𝑥max,𝑦𝑦max)
(𝑥𝑥min,𝑦𝑦min) (𝑦𝑦max − 𝑦𝑦min) ⋅ (𝑥𝑥max − 𝑥𝑥min)
Compactness = BLOB Area(𝑦𝑦max−𝑦𝑦min)⋅(𝑥𝑥max−𝑥𝑥min)
-
DTU Compute
2019Image Analysis18 DTU Compute, Technical University of Denmark
Compactness – which shape is most compact?
A) Shape 1:
B) Shape 2:
8
32
A B
Compactness = BLOB Area(𝑦𝑦max−𝑦𝑦min)⋅(𝑥𝑥max−𝑥𝑥min)
Chart1
A
B
8
32
Sheet1
A8
B32
-
DTU Compute
2019Image Analysis19 DTU Compute, Technical University of Denmark
BLOB Features Bounding box ratio
– Bounding box height divided by the width
One BLOB
(𝑥𝑥max,𝑦𝑦max)
(𝑥𝑥min,𝑦𝑦min)
-
DTU Compute
2019Image Analysis21 DTU Compute, Technical University of Denmark
BLOB Features Center of mass
(𝑥𝑥𝑐𝑐 ,𝑦𝑦𝑐𝑐)
(𝑥𝑥𝑖𝑖 ,𝑦𝑦𝑖𝑖)
(𝑥𝑥𝑐𝑐 ,𝑦𝑦𝑐𝑐)
𝑥𝑥𝑐𝑐 =1𝑁𝑁�𝑖𝑖=1
𝑁𝑁
𝑥𝑥𝑖𝑖
𝑦𝑦𝑐𝑐 =1𝑁𝑁�𝑖𝑖=1
𝑁𝑁
𝑦𝑦𝑖𝑖
-
DTU Compute
2019Image Analysis22 DTU Compute, Technical University of Denmark
Center of massA) (12, 1.5)B) (5, 8.5)C) (6.5, 3.5)D) (4.5, 0.5)E) (7, 4.5)
04
0 2
31
A B C D E
The smallest BLOB is found using 4-connectivity. What is the center of mass of this BLOB. The image has origin (0,0) and uses a (x,y) coordinatesystem.
Chart1
A
B
C
D
E
0
4
0
2
31
Sheet1
A0
B4
C0
D2
E31
-
DTU Compute
2019Image Analysis24 DTU Compute, Technical University of Denmark
BLOB Features Perimeter
– Length of perimeter– How can we compute that?
In practice (in Matlab) it is computed differently and more accurately
One BLOB
�( 𝑓𝑓 𝑥𝑥,𝑦𝑦 ⊕ SE − 𝑓𝑓 𝑥𝑥,𝑦𝑦 )
-
DTU Compute
2019Image Analysis27 DTU Compute, Technical University of Denmark
BLOB Features - circularity
How much does it look like a circle?
Circle– Area 𝐴𝐴 = 𝜋𝜋𝑟𝑟2– Perimeter 𝑃𝑃 = 2𝜋𝜋𝑟𝑟
New object assumed to be a circle– Measured perimeter 𝑃𝑃𝑚𝑚– Measured area 𝐴𝐴𝑚𝑚
Estimate perimeter from (measured) area– Estimated perimeter 𝑃𝑃𝑒𝑒 = 2 𝜋𝜋𝐴𝐴𝑚𝑚
Circle like
Not circle like
-
DTU Compute
2019Image Analysis28 DTU Compute, Technical University of Denmark
BLOB Features - circularity
Compare the perimeters– Measured perimeter 𝑃𝑃𝑚𝑚– Estimated perimeter 𝑃𝑃𝑒𝑒 = 2 𝜋𝜋𝐴𝐴𝑚𝑚
Circularity 1:
Circularity =𝑃𝑃𝑚𝑚𝑃𝑃𝑒𝑒
=𝑃𝑃𝑚𝑚
2 𝜋𝜋𝐴𝐴𝑚𝑚
Circle like
Not circle like
-
DTU Compute
2019Image Analysis29 DTU Compute, Technical University of Denmark
Circularity
A) 𝑃𝑃𝑚𝑚 < 𝑃𝑃𝑒𝑒B) 𝑃𝑃𝑚𝑚 = 𝑃𝑃𝑒𝑒C) 𝑃𝑃𝑚𝑚 > 𝑃𝑃𝑒𝑒
0
4
0
A B C
Circularity =𝑃𝑃𝑚𝑚𝑃𝑃𝑒𝑒
=𝑃𝑃𝑚𝑚
2 𝜋𝜋𝐴𝐴𝑚𝑚
Chart1
A
B
C
0
4
0
Sheet1
A0
B4
C0
-
DTU Compute
2019Image Analysis30 DTU Compute, Technical University of Denmark
Circularity 2
A) 𝑃𝑃𝑚𝑚 < 𝑃𝑃𝑒𝑒B) 𝑃𝑃𝑚𝑚 = 𝑃𝑃𝑒𝑒C) 𝑃𝑃𝑚𝑚 > 𝑃𝑃𝑒𝑒
14
1
25
A B C
Circularity =𝑃𝑃𝑚𝑚𝑃𝑃𝑒𝑒
=𝑃𝑃𝑚𝑚
2 𝜋𝜋𝐴𝐴𝑚𝑚
Chart1
A
B
C
14
1
25
Sheet1
A14
B1
C25
-
DTU Compute
2019Image Analysis31 DTU Compute, Technical University of Denmark
BLOB Features - circularity
Compare the perimeters– Measured perimeter 𝑃𝑃𝑚𝑚– Estimated perimeter 𝑃𝑃𝑒𝑒 = 2 𝜋𝜋𝐴𝐴𝑚𝑚
Circularity:
Circularity =𝑃𝑃𝑚𝑚𝑃𝑃𝑒𝑒
=𝑃𝑃𝑚𝑚
2 𝜋𝜋𝐴𝐴𝑚𝑚
This measure will normally be ≥1
Circle like
Not circle like
-
DTU Compute
2019Image Analysis32 DTU Compute, Technical University of Denmark
BLOB Features – circularity inverse
Compare the perimeters– Measured perimeter 𝑃𝑃𝑚𝑚– Estimated perimeter 𝑃𝑃𝑒𝑒 = 2 𝜋𝜋𝐴𝐴𝑚𝑚
Circularity (inverse):
Circularity inverse =𝑃𝑃𝑒𝑒𝑃𝑃𝑚𝑚
=2 𝜋𝜋A𝑚𝑚𝑃𝑃𝑚𝑚
This measure will normally be ≤1
Circle like
Not circle like
-
DTU Compute
2019Image Analysis33 DTU Compute, Technical University of Denmark
After feature extraction
Feature vector = [2,1,…,3]
Feature vector = [4,7,…,0]
Area, compactness, circularity etc calculated for all BLOB
One feature vector per blob
-
DTU Compute
2019Image Analysis34 DTU Compute, Technical University of Denmark
BLOB Classification Classification
– Put a BLOB into a class
Classes are normally pre-defined– Car– Bus– Motorcycle– Scooter
Object recognition
-
DTU Compute
2019Image Analysis35 DTU Compute, Technical University of Denmark
Object recognition: Circle example
Which objects are circles?
-
DTU Compute
2019Image Analysis36 DTU Compute, Technical University of Denmark
Circle classification
Two classes:– Circle– Not-circle
Lets make a model of a proto-type circle
-
DTU Compute
2019Image Analysis38 DTU Compute, Technical University of Denmark
Circle classification
Proto-type circle– Circularity : 1– Area: 6700
-
DTU Compute
2019Image Analysis39 DTU Compute, Technical University of Denmark
Feature SpaceProto-type circle
Objects in here are classified as circles
-
DTU Compute
2019Image Analysis40 DTU Compute, Technical University of Denmark
Feature space
Proto-type circle– Circularity : 1– Area: 6700
Some slack is added to allow non-perfect circles– Circularity: 1 +/- 0.15
-
DTU Compute
2019Image Analysis41 DTU Compute, Technical University of Denmark
Feature space - distances
How do we decide if an object is inside the circle?
Feature space distance Euclidean distance in
features space
D = (0.31 − 1)2+(6561 − 6700)2
Blob 1: circularity: 0.31, Area : 6561
Dominates all! – normalisation needed
-
DTU Compute
2019Image Analysis42 DTU Compute, Technical University of Denmark
BLOB Feature SelectionA) Compactness and
circularityB) Bounding box ratio and
circularityC) Area and compactnessD) Compactness and
bounding box ratioE) Area and bounding box
ratio
6
107
1
15
A B C D E
What set of BLOB features separate the below BLOBS best?
Chart1
A
B
C
D
E
6
10
7
1
15
Sheet1
A6
B10
C7
D1
E15
-
DTU Compute
2019Image Analysis43 DTU Compute, Technical University of Denmark
Cell classification
UV Microscopy Fluorescence Microscopy (DAPI)
Single Nuclei Multiple Nuclei
Images from ChemoMetec A/S
-
DTU Compute
2019Image Analysis44 DTU Compute, Technical University of Denmark
Nuclei classificationDAPI image Two classes
– Single nuclei– Noise
Multiple nuclei together Debris Other noise
-
DTU Compute
2019Image Analysis45 DTU Compute, Technical University of Denmark
Training and annotation Selection of true
single nuclei marked
Thresholding BLOB Analysis
– Circularity– Area
-
DTU Compute
2019Image Analysis46 DTU Compute, Technical University of Denmark
Training data - analysis
Probably outliers
Acceptance area
-
DTU Compute
2019Image Analysis47 DTU Compute, Technical University of Denmark
Feature ranges
Feature Min MaxArea 50 110Circularity 0.87 1.05
-
DTU Compute
2019Image Analysis48 DTU Compute, Technical University of Denmark
Using the classifier
DAPI input image
Threshold input image Morphological opening (SE 5x5) Morphological closing (SE 5x5) BLOBs found using 8-neighbours Border BLOBS removed Border features computed
– Area + circularity BLOBs with features inside the
acceptance range are single-nuclei
-
DTU Compute
2019Image Analysis49 DTU Compute, Technical University of Denmark
Using the classifier
DAPI input image Found single nuclei
Feature Min Max
Area 50 110
Circularity 0.87 1.05
-
DTU Compute
2019Image Analysis50 DTU Compute, Technical University of Denmark
How well does it work?We say we have a great
algorithm! Strangely the
doctor/biochemist do not trust this statement!– They need numbers!
How do we report the performance?
-
DTU Compute
2019Image Analysis51 DTU Compute, Technical University of Denmark
Creating ground truth – expert annotations
Found single nuclei Expert opinion on true single nucleiRed markings: Single nuclei
Not marked: Noise
-
DTU Compute
2019Image Analysis52 DTU Compute, Technical University of Denmark
Four cases True Positive (TP): A nuclei is classified as a nuclei True Negative (TN): A noise object is classified as noise object False Positive (FP): A noise object is classified as a nuclei False Negative (FN): A nuclei is classified as a noise object
Found single nuclei
-
DTU Compute
2019Image Analysis53 DTU Compute, Technical University of Denmark
Confusion matrix
Predicted as noise Predicted as single-nuclei
Actual noiseActual single-nuclei
-
DTU Compute
2019Image Analysis54 DTU Compute, Technical University of Denmark
Confusion matrix
Predicted as noise Predicted as single-nuclei
Actual noise TN=19Actual single-nuclei
-
DTU Compute
2019Image Analysis55 DTU Compute, Technical University of Denmark
Confusion matrix
Predicted as noise Predicted as single-nuclei
Actual noise TN=19Actual single-nuclei TP=51
-
DTU Compute
2019Image Analysis56 DTU Compute, Technical University of Denmark
Confusion matrix
Predicted as noise Predicted as single-nuclei
Actual noise TN=19 FP=2Actual single-nuclei TP=51
-
DTU Compute
2019Image Analysis57 DTU Compute, Technical University of Denmark
Confusion matrix
Predicted as noise Predicted as single-nuclei
Actual noise TN=19 FP=2Actual single-nuclei FN=5 TP=51
Something simpler?
-
DTU Compute
2019Image Analysis58 DTU Compute, Technical University of Denmark
Accuracy Tells how often the classifier is correct
N is the total number of annotated objects
Accuracy=𝑇𝑇𝑇𝑇+𝑇𝑇𝑁𝑁𝑁𝑁
𝑁𝑁 = 𝑇𝑇𝑁𝑁 + 𝑇𝑇𝑃𝑃 + 𝐹𝐹𝑃𝑃 + 𝐹𝐹𝑁𝑁
-
DTU Compute
2019Image Analysis59 DTU Compute, Technical University of Denmark
AccuracyA) 42%B) 65%C) 77%D) 91%E) 97%
0 0 1
39
1
A B C D E
Predicted as noise Predicted as single-nuclei
Actual noise TN=19 FP=2Actual single-nuclei FN=5 TP=51
Chart1
A
B
C
D
E
0
0
1
39
1
Sheet1
A0
B0
C1
D39
E1
-
DTU Compute
2019Image Analysis60 DTU Compute, Technical University of Denmark
True positive rate (sensivity) How often is a positive predicted when it actually is
positive
Sensivity= 𝑇𝑇𝑇𝑇𝐹𝐹𝑁𝑁+𝑇𝑇𝑇𝑇 All the experts true single-nuclei
-
DTU Compute
2019Image Analysis61 DTU Compute, Technical University of Denmark
SensitivityA) 62%B) 65%C) 71%D) 91%E) 93%
0 04
27
6
A B C D E
Predicted as noise Predicted as single-nuclei
Actual noise TN=19 FP=2Actual single-nuclei FN=5 TP=51
Chart1
A
B
C
D
E
0
0
4
27
6
Sheet1
A0
B0
C4
D27
E6
-
DTU Compute
2019Image Analysis62 DTU Compute, Technical University of Denmark
Specificity How often is a negative predicted when it actually is
negative
Specificity= 𝑇𝑇𝑁𝑁𝑇𝑇𝑁𝑁+𝐹𝐹𝑇𝑇 All the experts true noise objects
-
DTU Compute
2019Image Analysis63 DTU Compute, Technical University of Denmark
SpecificityA) 77%B) 81%C) 90%D) 92%E) 97%
1 2
33
1 1
A B C D E
Predicted as noise Predicted as single-nuclei
Actual noise TN=19 FP=2Actual single-nuclei FN=5 TP=51
Chart1
A
B
C
D
E
1
2
33
1
1
Sheet1
A1
B2
C33
D1
E1
-
DTU Compute
2019Image Analysis64 DTU Compute, Technical University of Denmark
Optimising the classification Changing the
classification limits The rates will be
changed:– Accuracy– Sensitivity– Specificity– …
Very dependent on the task what is optimal
-
DTU Compute
2019Image Analysis65 DTU Compute, Technical University of Denmark
Dependencies Increasing true positive rate
– Increased false positive rate– Decreased precision
-
DTU Compute
2019Image Analysis66 DTU Compute, Technical University of Denmark
Example – cell analysis We want only single-nuclei cells
– For further analysis
We do not want to do an analysis of a noise object
We are not interested in the true number of single nuclei
-
DTU Compute
2019Image Analysis67 DTU Compute, Technical University of Denmark
What measure is the most important?A) Low false positivesB) High true positivesC) High true negativesD) Low false negatives
30
4 2 4
A B C D We want only single-nuclei cells
– For further analysis We do not want to do an analysis of noise objects We are not interested in the true number of single
nuclei
Chart1
A
B
C
D
30
4
2
4
Sheet1
A30
B4
C2
D4
-
DTU Compute
2019Image Analysis68 DTU Compute, Technical University of Denmark
Advanced classification Fitting more advanced functions to the samples Multivariate Gaussians Mahalanobis distances
-
DTU Compute
2019Image Analysis69 DTU Compute, Technical University of Denmark
Feature Engineering vs. Deep learningUntil around 5-7
years ago feature engineering was the way to go
Now deep learning beats everything
However – feature engineering is still important
-
DTU Compute
2019Image Analysis70 DTU Compute, Technical University of Denmark
Feature engineering Given a classification problem
– Cars vs. Pedestrians Use background knowledge to
select relevant features– Area– Shape– Appearance– …
Use multivariate statistics to classify
Depending on the selected features
-
DTU Compute
2019Image Analysis71 DTU Compute, Technical University of Denmark
Deep learning You start with a dummy
classifier Feed it with lots and lots
of data with given labels The network learns the
optimal features Layer/network engineering
-
DTU Compute
2019Image Analysis72 DTU Compute, Technical University of Denmark
Feature Engineering vs. Deep learningDeep LearningWhen you have lot
of annotated dataWhere it is not clear
what features work
Manual featuresWhen you have
limited dataWhen it is rather
obvious what features can discriminate
-
DTU Compute
2019Image Analysis73 DTU Compute, Technical University of Denmark
The level of medical examplesA) Stop stop! Too much
medical stuffB) I would like some more
non-medical examplesC) Its okD) I like the medical
examplesE) Great with medical
examples1
1412
14
5
A B C D E
Chart1
A
B
C
D
E
1
14
12
14
5
Sheet1
A1
B14
C12
D14
E5
-
DTU Compute
2019Image Analysis74 DTU Compute, Technical University of Denmark
Next week Pixel Classification
Image AnalysisLecture 6 – BLOB analysis and feature based classificationWhat can you do after today?Object recognitionBLOB – what is it?Isolating a BLOBConnected component analysisBLOBs 4- and 8-neighboursThe result of connected component analysisFeaturesFeature vectorFeature extractionsBLOB FeaturesBLOB FeaturesBLOB FeaturesCompactness – which shape is most compact?BLOB FeaturesBLOB FeaturesCenter of massBLOB FeaturesBLOB Features - circularityBLOB Features - circularityCircularityCircularity 2BLOB Features - circularityBLOB Features – circularity inverseAfter feature extractionBLOB ClassificationObject recognition: Circle exampleCircle classificationCircle classificationFeature SpaceFeature spaceFeature space - distancesBLOB Feature SelectionCell classificationNuclei classificationTraining and annotationTraining data - analysisFeature rangesUsing the classifierUsing the classifierHow well does it work?Creating ground truth – expert annotationsFour casesConfusion matrixConfusion matrixConfusion matrixConfusion matrixConfusion matrixAccuracyAccuracyTrue positive rate (sensivity)SensitivitySpecificitySpecificityOptimising the classificationDependenciesExample – cell analysisWhat measure is the most important?Advanced classificationFeature Engineering vs. Deep learningFeature engineeringDeep learningFeature Engineering vs. Deep learningThe level of medical examplesNext week
top related