![Page 1: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/1.jpg)
Mestrado em Ciência de Computadores
Mestrado Integrado em Engenharia de Redes e
Sistemas Informáticos
VC 18/19 – TP8
Segmentation
Miguel Tavares Coimbra
![Page 2: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/2.jpg)
VC 18/19 - TP8 - Segmentation
Outline
• Thresholding
• Geometric structures
• Hough Transform
![Page 3: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/3.jpg)
VC 18/19 - TP8 - Segmentation
Topic: Thresholding
• Thresholding
• Geometric structures
• Hough Transform
![Page 4: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/4.jpg)
VC 18/19 - TP8 - Segmentation
Boundaries of Objects
Marked by many users
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/bench/html/images.html
![Page 5: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/5.jpg)
VC 18/19 - TP8 - Segmentation
Boundaries of Objects from
Edges
Brightness Gradient (Edge detection)
• Missing edge continuity, many spurious edges
![Page 6: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/6.jpg)
VC 18/19 - TP8 - Segmentation
Boundaries of Objects from
Edges
Multi-scale Brightness Gradient
• But, low strength edges may be very important
![Page 7: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/7.jpg)
VC 18/19 - TP8 - Segmentation
Image
Machine Edge Detection
Human Boundary Marking
![Page 8: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/8.jpg)
VC 18/19 - TP8 - Segmentation
Boundaries in Medical Imaging
Detection of cancerous regions.
[Foran, Comaniciu, Meer, Goodell, 00]
![Page 9: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/9.jpg)
VC 18/19 - TP8 - Segmentation
Boundaries in Ultrasound
Images
Hard to detect in the presence of large amount of speckle noise
![Page 10: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/10.jpg)
VC 18/19 - TP8 - SegmentationSometimes hard even for humans!
![Page 11: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/11.jpg)
VC 18/19 - TP8 - Segmentation
What is ‘Segmentation’?
• Separation of the
image in different
areas.
– Objects.
– Areas with similar
visual or semantic
characteristics.
Not trivial! It is the holy grail of
most computer vision problems!
![Page 12: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/12.jpg)
VC 18/19 - TP8 - Segmentation
Subjectivity
• A ‘correct’
segmentation result is
only valid for a
specific context.
– Subjectivity!
– Hard to implement.
– Hard to evaluate.
Person Suitcase
Face
What is the
correct
segmentation?
![Page 13: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/13.jpg)
VC 18/19 - TP8 - Segmentation
Core Technique: Thresholding
• Divide the image into two areas:
– 1, if f(x,y)>K
– 0, if f(x,y)<=K
• Not easy to find the ideal kmagic number.
• Core segmentation technique
– Simple
– Reasonably effective
Adequate
threshold
![Page 14: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/14.jpg)
VC 18/19 - TP8 - Segmentation
Finding the ‘magic number’
Wrong!
(k = 128)Correct
(k = 74)
![Page 15: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/15.jpg)
VC 18/19 - TP8 - Segmentation
http://www.cee.hw.ac.uk/hipr/html/adpthrsh.html
Global thresholds are not
always adequate...
![Page 16: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/16.jpg)
VC 18/19 - TP8 - Segmentation
Adaptive Thresholding
• Adapt the threshold
value for each pixel.
• Use characteristics of
nearby pixels.
• How?
– Mean
– Median
– Mean + K
– ...
Mean of 7x7 neighborhood
![Page 17: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/17.jpg)
VC 18/19 - TP8 - Segmentation7x7 window; K = 7 75x75 window; K = 10
![Page 18: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/18.jpg)
VC 18/19 - TP8 - Segmentation
Topic: Geometric structures
• Thresholding
• Geometric structures
• Hough Transform
![Page 19: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/19.jpg)
VC 18/19 - TP8 - Segmentation
Points
• What is a point?
– Pixel with a significant
illumination difference
to its neighbors.
– Group of pixels?
• Spatial Mask!
• Need to define a
threshold K.
-1 -1 -1
-1 8 -1
-1 -1 -1
point!
9
1
KR
zwRi
ii
![Page 20: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/20.jpg)
VC 18/19 - TP8 - Segmentation
Lines
• Spatial filter
– One per line direction
– Sensitive to line width
-1 -1 -1
2 2 2
-1 -1 -1
-1 2 -1
-1 2 -1
-1 2 -1
Horizontal Vertical
Diagonal?
![Page 21: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/21.jpg)
VC 18/19 - TP8 - Segmentation
Edges
• Edge:
– Spatial discontinuity of pixel amplitude.
– High spatial gradient• First derivative (peak)
• Second derivative (zero crossing)
![Page 22: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/22.jpg)
VC 18/19 - TP8 - Segmentation
Popular operators
• Edge detection
– Great utility for
several problems.
– Well studied
problem.
• A variety of
solutions exists.
![Page 23: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/23.jpg)
VC 18/19 - TP8 - Segmentation
![Page 24: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/24.jpg)
VC 18/19 - TP8 - Segmentation
Processing Edge Images
Image
Edge detection
and ThresholdingNoisy edge image
Incomplete boundaries
Edge Tracking
Thinning
![Page 25: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/25.jpg)
VC 18/19 - TP8 - Segmentation
Edge Tracking Methods
• Adjusting a priori Boundaries
Given: Approximate Location of Boundary
Task: Find Accurate Location of Boundary
• Search for STRONG EDGES along normals to approximate boundary.
• Fit curve (eg., polynomials) to strong edges.
![Page 26: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/26.jpg)
VC 18/19 - TP8 - Segmentation
Edge Tracking Methods
• Divide and ConquerGiven: Boundary lies between points A and B
Task: Find Boundary
• Connect A and B with Line
• Find strongest edge along line bisector
• Use edge point as break point
• Repeat
![Page 27: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/27.jpg)
VC 18/19 - TP8 - Segmentation
Fitting Lines to Edges (Least
Squares)cmxy
cmxy ii
),( ii yx
y
x
Given: Many pairs
Find: Parameters
Minimize: Average square distance:
Using:
Note:
),( ii yx),( cm
i
ii
N
cmxyE
2)(
0&0
c
E
m
Exmyc
i
i
i
ii
xx
yyxx
m2)(
))((
N
y
y i
i
N
x
x i
i
![Page 28: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/28.jpg)
VC 18/19 - TP8 - Segmentation
Topic: Hough Transform
• Thresholding
• Geometric structures
• Hough Transform
![Page 29: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/29.jpg)
VC 18/19 - TP8 - Segmentation
Hough Transform
• Elegant method for direct object
recognition
• Edges need not be connected
• Complete object need not be visible
• Key Idea: Edges VOTE for the possible
model
![Page 30: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/30.jpg)
VC 18/19 - TP8 - Segmentation
Image and Parameter Spaces
cmxy
),( ii yx
y
x
Equation of Line:
Find:
Consider point:
cmxy
),( cm
),( ii yx
iiii ymxcorcmxy m
c
),( cm
Image
Space
Parameter
Space
Parameter space also called Hough Space
![Page 31: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/31.jpg)
VC 18/19 - TP8 - Segmentation
Line Detection by Hough Transform
y
x
),( cm
Parameter Space
1 1
1 1
1 1
2
1 1
1 1
1 1
),( cmA
Algorithm:
• Quantize Parameter Space
• Create Accumulator Array
• Set
• For each image edge increment:
• If lies on the line:
• Find local maxima in
),( cm
),( cmA
cmcmA ,0),(
),( ii yx
1),(),( cmAcmA
),( cm
),( cmA
ii ymxc
![Page 32: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/32.jpg)
VC 18/19 - TP8 - Segmentation
Better Parameterization
NOTE:
Large Accumulator
More memory and computations
Improvement:
Line equation:
Here
Given points find
m
sincos yx
max0
20
(Finite Accumulator Array Size)
),( ii yx ),(
),( ii yx
y
x
Image
Space
Hough
Space
?
Hough Space Sinusoid
![Page 33: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/33.jpg)
VC 18/19 - TP8 - Segmentation
Image spaceVotes
Horizontal axis is θ,
vertical is rho.
![Page 34: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/34.jpg)
VC 18/19 - TP8 - Segmentation
![Page 35: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/35.jpg)
VC 18/19 - TP8 - Segmentation
![Page 36: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/36.jpg)
VC 18/19 - TP8 - Segmentation
Mechanics of the Hough Transform
• Difficulties– how big should the
cells be? (too big, and we merge quite different lines; too small, and noise causes lines to be missed)
• How many lines?– Count the peaks in the
Hough array
– Treat adjacent peaks as a single peak
• Which points belong to each line?– Search for points close
to the line
– Solve again for line and iterate
![Page 37: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/37.jpg)
VC 18/19 - TP8 - Segmentation
Fewer votes land in a
single bin when noise increases.
![Page 38: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/38.jpg)
VC 18/19 - TP8 - Segmentation
Adding more clutter increases
number of bins with false peaks.
![Page 39: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/39.jpg)
Real World Example
Original Edge Detection
Found Lines
Parameter Space
![Page 40: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/40.jpg)
VC 18/19 - TP8 - Segmentation
Other shapes
OriginalEdges when using circle
model
![Page 41: VC 18/19 TP8 Segmentation - DCCmcoimbra/lectures/VC_1819/VC_1819_TP… · VC 18/19 - TP8 - Segmentation Edge Tracking Methods • Divide and Conquer Given: Boundary lies between points](https://reader036.vdocument.in/reader036/viewer/2022071102/5fdba5256300ab717c0d19e4/html5/thumbnails/41.jpg)
VC 18/19 - TP8 - Segmentation
Resources
• Gonzalez & Woods – Chapter 7
• N. Otsu, “A threshold selection method
from gray-level histograms,” IEEE Trans.
Sys., Man., Cyber., vol. 9, pp. 62–66,
1979.