lecture 3: region based vision dr carole twining thursday 18th march 1:00pm – 1:50pm
TRANSCRIPT
![Page 1: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/1.jpg)
Lecture 3:Region Based Vision
Dr Carole Twining
Thursday 18th March
1:00pm – 1:50pm
![Page 2: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/2.jpg)
Slide 2
Assigning labels to pixels (cat, ball, floor)
● Point processing: colour or grayscale values, thresholding
● Neighbourhood Processing: Regions of similar colours or textures
● Edge information (next lecture)
● Prior information: (model-based vision) I know what I expect a cat to look like
Segmenting an Image
![Page 3: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/3.jpg)
Slide 3
Overview
● Automatic threshold detection Earlier, we did by inspection/guessing
● Multi-Spectral segmentation satellite and medical image data
● Split and Merge Hierarchical, region-based approach
● Relaxation labelling probabilistic, learning approach
● Segmentation as optimisation
![Page 4: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/4.jpg)
Automatic Threshold Selection
![Page 5: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/5.jpg)
Slide 5
Automatic Thresholding: Optimal
● Assume scene mixture of substances, each with normal/gaussian distribution of possible image values
● Minimum error in probabilistic terms
● But sum of gaussians not easy to find
● Doesn’t always fit actual distribution
ImageHistogram
Segmentation Rule
![Page 6: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/6.jpg)
Slide 6
● Extend to multiple classes
Automatic Thresholding: Otsu’s Method
T
3 4
![Page 7: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/7.jpg)
Slide 7
Automatic Thresholding: Max Entropy
● Makes two sub-populations as peaky as possible
T
![Page 8: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/8.jpg)
Slide 8
Automatic Thresholding: Example
cat floor ball
combine
![Page 9: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/9.jpg)
Slide 9
Automatic Thresholding: Summary● Geometric shape of histogram (bumps, curves etc)
Algorithm or just by inspection
● Statistics of sub-populations Otsu & variance
Entropy methods
● Model-based methods: Mixture of gaussians
● And so on. > 40 methods surveyed in literature
● Fundamental limit on effectiveness: Never give great result if distributions overlap
● Whatever method, need further processing
![Page 10: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/10.jpg)
Multi-Spectral Segmentation
![Page 11: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/11.jpg)
Slide 11
f2
f1
Multi-spectral Segmentation
● Multiple measurements at each pixel: Satellite remote imaging, various wavebands
MR imaging, various imaging sequences
Colour (RGB channels, HSB etc)
● Scattergram of pixels in vector space:
● Can’t separate using single measurement
● Can using multiple
![Page 12: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/12.jpg)
Slide 12
Multi-Spectral Segmentation:Example
Over-ripe Orange
Scratched Orange
Spectral Bands
Multispectral Image Segmentation by Energy Minimization for Fruit Quality Estimation: Martínez-Usó, Pla, and García-Sevilla, Pattern Recognition and Image Analysis , 2005
![Page 13: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/13.jpg)
Split and Merge
![Page 14: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/14.jpg)
Slide 14
● Obvious approaches to segmentation:
Start from small regions and stitch them together
Start from large regions and split them
● Start with large regions , split non-uniform regions
e.g. variance 2 > threshold
● Merge similar adjacent regions
e.g. combined variance 2 < threshold
● Region adjacency graph
housekeeping for adjacency as regions become irregular
regions are nodes, adjacency relations arcs
simple update rules during splitting and merging
A
B C
D
Split and Merge
Combine
A & B
![Page 15: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/15.jpg)
Slide 15
Split
Original Split
Split and Merge
Split
Merge
Merge
![Page 16: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/16.jpg)
Slide 16
Original
Split and Merge: Example
Splitting
Merge
![Page 17: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/17.jpg)
Relaxation Labelling
![Page 18: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/18.jpg)
Slide 18
● P(pet) = etc
● P( pet | mammal) =
● P( mammal | pet) =
● Bayes Theorem:
● P( pet | mammal )P(mammal) = P( mammal | pet )P(pet)
All Animal Species
Aside: Conditional Probabilityprobability of A given that B is the case
( + )
( + )
ALL
( + ) ( + )( + )
ALLx =
( + )
ALLx
( + )
Pets
fishdog
cat
Mammals
whale
![Page 19: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/19.jpg)
Slide 19
Relaxation Labelling: ● Image histogram, object/background
Values from object pixels
Values from background
Ove
rlap
: m
ista
kes
in l
abel
lin
g
threshold
Label assignments
Context:
Context to resolve
ambiguity
![Page 20: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/20.jpg)
Slide 20
Relaxation Labelling● Evidence for a label at a pixel:
Measurements at that pixel (e.g., pixel value) Context for that pixel (i.e., what neighbours are doing)
● Iterative approach, labelling evolves
● Soft-assignment of labels:
● Soft-assignment allows you to consider all possibilities
● Let context act to find stable solution
![Page 21: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/21.jpg)
Slide 21
Relaxation Labelling
● Compatibility:
● Contextual support for label at pixel i :
If not neighboursno effect
Neighbours & same labelsupport
Neighbours & different labeloppose
look at all other pixels
degree of compatibility
all possible labels & how
strong
![Page 22: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/22.jpg)
Slide 22
Relaxation Labelling:
● Update soft labelling given context:
● The more support, more likely the label
● Iterate
Noisy Image
After iterating
Threshold labelling
![Page 23: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/23.jpg)
Slide 23
Relaxation Labeling:● Value of alters final result
Fields
Trees
Initialisation
= 0.75
= 0.90
![Page 24: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/24.jpg)
Segmentation as Optimisation
![Page 25: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/25.jpg)
Slide 25
Segmentation as Optimisation
● Maximise probability of labelling given image:
● Re-write by taking logs, minimise cost function:
● How to find the appropriate form for the two terms.
● How to find the optimum.
label at i given value at i
label at i given labels in neighbourhood of i
label-data match label consistency
![Page 26: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/26.jpg)
Slide 26
Segmentation as Optimisation
label-data match
label consistency ● Exact form depends on type of data
● Histogram gives:
● Model of histogram(e.g., sum of gaussians, relaxation case)
Learning approach:
● Explicit training data (i.e., similar labelled images)
● Unsupervised, from image itself (e.g., histogram model):
•Given labels, construct model•Given model, update labels•Repeat
Image & Labelling
Model Parameters
E-step
M-step
Expectation/Maximization
![Page 27: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/27.jpg)
Slide 27
● General case:
● High-dimensional search space, local minima
● Analogy to statistical mechanics crystalline solid finding minimum energy state
stochastic optimisation
simulated annealing
● Search: Downhill
Allow slight uphill
Segmentation as Optimisation
label-data match term
labelconsistency
label values
co
st
![Page 28: Lecture 3: Region Based Vision Dr Carole Twining Thursday 18th March 1:00pm – 1:50pm](https://reader036.vdocument.in/reader036/viewer/2022062301/56649e935503460f94b98aa3/html5/thumbnails/28.jpg)
Slide 28
Segmentation as Optimisation
Fields
Trees
Original
= 0.90
Relaxation Optimisation