edge detection & image segmentation dr. md. altab hossain associate professor dept. of computer...
TRANSCRIPT
![Page 1: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/1.jpg)
Edge Detection &
Image SegmentationDr. Md. Altab Hossain
Associate ProfessorDept. of Computer Science & Engineering, RU
1
![Page 2: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/2.jpg)
2
Edge Detection
![Page 3: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/3.jpg)
PreprocessImage acquisition, restoration, and enhancement
Intermediate processImage segmentation and feature extraction
High level processImage interpretation and recognition
Element of Image Analysis
3
![Page 4: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/4.jpg)
1. Similarity properties of pixels inside the object are used to grouppixels into the same set.
2. Discontinuity of pixel properties at the boundary between objectand background is used to distinguish between pixels belonging to the object and those of background.
Discontinuity:Intensity change
at boundaryPoint, Line, Edge
Similarity:Internal
pixels sharethe same intensity
Image Attributes for Image Segmentation
4
![Page 5: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/5.jpg)
Point Detection We can use Laplacian masksfor point detection.
Laplacian masks have the largest coefficient at the center of the maskwhile neighbor pixels have anopposite sign.
This mask will give the high response to the object that has the similar shape as the mask such as isolated points.
Notice that sum of all coefficients of the mask is equal to zero. This is due to the need that the response of the filter must be zero inside a constant intensity area
-1 -1
-1
8
-1
-1
-1
-1
-1
-1 0
0
4
-1
-1
0
-1
0
5
![Page 6: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/6.jpg)
Point Detection
X-ray image of the turbine blade with
porosity
Laplacian image After thresholding
Location of porosity
Point detection can be done by applying the thresholding function:
otherwise 0
),( 1),(
Tyxfyxg
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
6
![Page 7: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/7.jpg)
Line Detection Similar to point detection, line detection can be performedusing the mask the has the shape look similar to a part of a line
There are several directions that the line in a digital image can be.
For a simple line detection, 4 directions that are mostly used areHorizontal, +45 degree, vertical and –45 degree.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Line detection masks 7
![Page 8: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/8.jpg)
Line Detection Example Binary wirebond mask
image
Absolute valueof result after
processing with-45 line detector
Result after thresholding
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Notice that –45 degreelines are most sensitive
8
![Page 9: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/9.jpg)
Definition of Edges
Edges are significant local changes of intensity in an image.
Edges typically occur on the boundary between two different regions in an image.
9
![Page 10: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/10.jpg)
What Causes Intensity Changes?
Geometric events surface orientation (boundary) discontinuities depth discontinuities color and texture discontinuities
Non-geometric events illumination changes specularities shadows inter-reflections
depth discontinuity
color discontinuity
illumination discontinuity
surface normal discontinuity
10
![Page 11: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/11.jpg)
Why is Edge Detection Useful?
Important features can be extracted from the edges of an image (e.g., corners, lines, curves).
These features are used by higher-level computer vision algorithms (e.g., recognition).
11
![Page 12: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/12.jpg)
Where are the edges?
12
![Page 13: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/13.jpg)
13
Edge normal: unit vector in the direction of maximum intensity change.
Edge direction: unit vector to perpendicular to the edge normal. Edge position or center: the image position at which the edge is
located. Edge strength: related to the local image contrast along the
normal.
Edge Descriptors
![Page 14: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/14.jpg)
Modeling Intensity Changes
Step edge: the image intensity abruptly changes from one value on one side of the discontinuity to a different value on the opposite side.
Ramp edge: a step edge where the intensity change is not instantaneous but occur over a finite distance.
14
![Page 15: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/15.jpg)
Modeling Intensity Changes (cont’d) Ridge edge: the image intensity abruptly changes value but
then returns to the starting value within some short distance (i.e., usually generated by lines).
Roof edge: a ridge edge where the intensity change is not instantaneous but occur over a finite distance (i.e., usually generated by the intersection of two surfaces).
15
![Page 16: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/16.jpg)
Main Steps in Edge Detection
(1) Smoothing: suppress as much noise as possible, without destroying true edges.
(2) Enhancement: apply differentiation to enhance the quality of edges (i.e., sharpening).
(3) Thresholding: determine which edge pixels should be discarded as noise and which should be retained (i.e., threshold edge magnitude).
(4) Localization: determine the exact edge location.
sub-pixel resolution might be required for some applications to estimate the location of an edge to better than the spacing between pixels.
16
![Page 17: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/17.jpg)
Edge Detection Using Derivatives
Often, points that lie on an edge
are detected by:
(1) Detecting the local maxima or
minima of the first derivative.
(2) Detecting the zero-crossings
of the second derivative.
2nd derivative
1st derivative
17
![Page 18: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/18.jpg)
Gray level profile
The 1st derivative
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5-0.20
0.20.40.60.8
11.2
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5-0.06-0.04-0.02
00.020.040.06
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5-5-4-3-2-1012345
x 10 -3
Edge Edge
Minimum point
Maximumpoint
+ +- -
Zero crossing
Inte
nsit
y
Smoothed Step Edge and Its Derivatives
The 2nd derivative
18
![Page 19: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/19.jpg)
Image Derivatives
How can we differentiate a digital image?
Option 1: reconstruct a continuous image, f(x,y), then compute the derivative.
Option 2: take discrete derivative (i.e., finite differences)
Consider this case first!
19
![Page 20: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/20.jpg)
Edge Detection Using First Derivative
(upward) step edge
(centered at x)
1D functions(not centered at x)
20
![Page 21: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/21.jpg)
Edge Detection Using Second Derivative
Approximate finding maxima/minima of gradient magnitude by finding places where:
Can’t always find discrete pixels where the second derivative is zero – look for zero-crossing instead.
21
![Page 22: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/22.jpg)
Replace x+1 with x (i.e., centered at x):
1D functions:
(centered at x+1)
Edge Detection Using Second Derivative (cont’d)
22
![Page 23: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/23.jpg)
Edge Detection Using Second Derivative (cont’d)
23
![Page 24: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/24.jpg)
Edge Detection Using Second Derivative (cont’d)
24
![Page 25: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/25.jpg)
Noise Effect on Images Edges
First column: images and gray-level profiles of a ramp edge corrupted by random Gaussian noise of mean 0 and = 0.0, 0.1, 1.0 and 10.0, respectively.
Second column: first-derivative images and gray-level profiles.
Third column : second-derivative images and gray-level profiles.
25
![Page 26: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/26.jpg)
Edge Detection Using First Derivative (Gradient)
The first derivate of an image can be computed using the gradient:
2D functions:
f
26
![Page 27: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/27.jpg)
The gradient is a vector which has magnitude and direction:
Magnitude: indicates edge strength.
Direction: indicates edge direction.i.e., perpendicular to edge direction
or
(approximation)
Gradient Representation
27
![Page 28: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/28.jpg)
Approximate Gradient
Approximate gradient using finite differences:
28
![Page 29: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/29.jpg)
Approximate Gradient (cont’d)
Cartesian vs pixel-coordinates:
- j corresponds to x direction
- i to -y direction
29
![Page 30: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/30.jpg)
Approximating Gradient (cont’d)
We can implement and using the following masks:
(x+1/2,y)
(x,y+1/2)*
*
good approximationat (x+1/2,y)
good approximationat (x,y+1/2)
30
![Page 31: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/31.jpg)
Approximating Gradient (cont’d)
A different approximation of the gradient:
3 x 3 neighborhood:
31
![Page 32: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/32.jpg)
Approximating Gradient (cont’d)
and can be implemented using the following masks:
32
![Page 33: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/33.jpg)
Another Approximation
Consider the arrangement of pixels about the pixel (i, j):
The partial derivatives can be computed by:
The constant c implies the emphasis given to pixels closer to the center of the mask.
3 x 3 neighborhood:
33
![Page 34: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/34.jpg)
Prewitt Operator
Setting c = 1, we get the Prewitt operator:
34
![Page 35: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/35.jpg)
Sobel Operator
Setting c = 2, we get the Sobel operator:
35
![Page 36: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/36.jpg)
Edge Detection Steps Using Gradient
36
![Page 37: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/37.jpg)
An Example
Idx
d
Idy
d
37
![Page 38: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/38.jpg)
An Example (cont’d)
22d d
I Idx dy
100Threshold
38
![Page 39: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/39.jpg)
Comparison of Gradient based Operator
39
![Page 40: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/40.jpg)
40
The Canny Edge Detector
![Page 41: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/41.jpg)
41
The Canny Edge Detector Canny – smoothing and derivatives:
xf
yf
f
![Page 42: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/42.jpg)
42
The Canny Edge Detector Canny – gradient magnitude:
image gradient magnitude
![Page 43: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/43.jpg)
Practical Issues Choice of threshold.
gradient magnitude
low threshold high threshold
43
![Page 44: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/44.jpg)
Practical Issues (cont’d)Edge thinning and linking.
44
![Page 45: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/45.jpg)
Criteria for Optimal Edge Detection
(1) Good detection Minimize the probability of false positives (i.e., spurious edges). Minimize the probability of false negatives (i.e., missing real edges).
(2) Good localization Detected edges must be as close as possible to the true edges.
(3) Single response Minimize the number of local maxima around the true edge.
45
![Page 46: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/46.jpg)
46
Edge Contour Extraction
Two main categories of methods: local methods (extend edges by seeking the most "compatible" candidate
edge in a neighborhood). global methods (more computationally expensive - domain knowledge can
be incorporated in their cost function).
Edge detectors typically produce short, disjoint edge segments.
These segments are generally of little use until they are aggregated into extended edges.
We assume that edge thinning has already be done (e.g., non-maxima suppression).
![Page 47: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/47.jpg)
47
Edge Contour Extraction
![Page 48: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/48.jpg)
48
Local Processing Methods
Edge Linking using neighbor
![Page 49: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/49.jpg)
49
Local Processing Methods
Contour extraction using heuristic search
A more comprehensive approach to contour extraction is based on graph searching.
Graph representation of edge points: (1) Edge points at position pi correspond to graph nodes.
(2) The nodes are connected to each other if local edge linking rules are satisfied.
![Page 50: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/50.jpg)
50
Local Processing Methods Contour extraction using
heuristic search – cont.
The generation of a contour (if any) from pixel pA to pixel pB the generation of a minimum-cost path in the directed graph.
A cost function for a path connecting nodes p1 = pA to pN = pB could be defined as follows:
Finding a minimum-cost path is not trivial in terms of computation.
![Page 51: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/51.jpg)
51
Local Processing Methods Contour extraction using dynamic programming
Dynamic programming
It is an optimization method that searches for optima of functions in which not all the variables are simultaneously interrelated.
It subdivides a problem recursively into smaller sub-problems that may need to be solved in the future, solving each sub-problem – proceeding from the smaller ones to the larger ones – and storing the solutions in a table that can be looked up when need arises.
Principle of optimality (applied to the case of graph searching): the optimal path between two nodes pA and pB can be split into two optimal sub-paths pApi and pipB for any pi lying on the optimal path pApB.
![Page 52: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/52.jpg)
52
Global Processing Methods If the gaps between pixels are very large, local processing
methods are not effective. Global methods are more effective in this case !!
Hough Transform can be used to determine whether points lie on a curve of a specified shape (model-based method).
Deformable Models (Snakes) can be used to extract the boundaries of objects having arbitrary shapes.
Grouping can be used to decide which groups of features are likely to be part of the same object.
![Page 53: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/53.jpg)
53
Image segmentation
![Page 54: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/54.jpg)
Definition of Segmentation
Segmentation: Image segmentation refers to the partition of an image into a set of regions that have a strong correlation with objects or areas of the real world. Complete segmentation (High-level): Region(s) correspond(s)
uniquely with image object(s); not an easy task; requires domain specific knowledge; context is known and a
priori knowledge is available and used for segmentation. Partial segmentation (Low-level): Regions do not always
correspond directly with image objects. Image is divided into separate regions that are homogenous with respect to a chosen property such as brightness, color, texture, motion, etc.
54
![Page 55: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/55.jpg)
Example-Complete Segmentation
The image is segmented into two regions: face and background. We know that the human face is of some certain color. Several other heuristics using the knowledge of human face can
be used. Small holes and isolated regions are eliminated.
55
![Page 56: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/56.jpg)
Example-Partial Segmentation
Regions do not directly correspond to objects, but to regionsthat are similar in color Needs further processing
56
![Page 57: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/57.jpg)
57
Segmentation Criteria
Segmentation of an image I into a set of regions S should satisfy:1. Si = S Partition covers the whole
image.
2. Si Sj = , i j No regions intersect.
3. Si, P(Si) = true Homogeneity predicate is satisfied by each
region.
4. P(Si Sj) = false, Union of adjacent regionsi j, Si adjacent Sj does not satisfy it.
![Page 58: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/58.jpg)
58
Image segmentation
So, all we have to do is to define and implement the similarity predicate. But, what do we want to be similar in each region? Is there any property that will cause the regions to
be meaningful objects?
Example approaches: Histogram-based Clustering-based Region growing Split-and-merge Graph-based
![Page 59: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/59.jpg)
Histogram-based segmentation
Global Thresholding = Choose threshold T that separates object from background.
59
![Page 60: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/60.jpg)
Simple thresholding is not always possible: Many objects at different gray
levels. Variations in background gray
level. Noise in image.
Histogram-based segmentation
60
![Page 61: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/61.jpg)
Local Thresholding-4 Thresholds Divide image in to regions. Perform thresholding independently
in each region.
61
![Page 62: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/62.jpg)
Adaptive Thresholding
Every pixel in image is thresholded according to the histogram of the pixel neighborhood.
62
![Page 63: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/63.jpg)
Any feature vector that can be associated with a pixel can be used to group pixels into clusters. Once pixels have been grouped into clusters, it is easy to find connected regions using connected components labeling.
Clustering-based segmentation
K-means algorithm can be used for clustering 63
![Page 64: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/64.jpg)
64
Clustering-based segmentation
K-means clustering of color.
![Page 65: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/65.jpg)
65
Clustering-based segmentation Clustering can also be used with other features (e.g., texture)
in addition to color.
Original Images Color Regions Texture Regions
![Page 66: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/66.jpg)
Region Growing
66
![Page 67: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/67.jpg)
67
Region growing
Usually a statistical test is used to decide which pixels can be added to a region. Region is a population with similar statistics. Use statistical test to see if neighbor on border fits
into the region population.
Let R be the N pixel region so far and p be a neighboring pixel with gray level.
Define the mean X and scatter S2 (sample variance) by
Rc)(r,
c)I(r,N
1X 2
Rc)(r,
2 X-c)I(r,N1
S
![Page 68: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/68.jpg)
68
Region growing
The T for a pixel p statistic is defined by
It has a TN-1 distribution if all the pixels in R and the test pixel p are independent and identically distributed Gaussians (i.i.d. assumption).
1/2
22 /S)X(p1)(N
1)N(NT
![Page 69: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/69.jpg)
69
Region growing
For the T distribution, statistical tables give us the probability Pr(T ≤ t) for a given degrees of freedom and a confidence level. From this, pick a suitable threshold t.
If the computed T ≤ t for desired confidence level, add p to region R and update the mean and scatter using p.
If T is too high, the value p is not likely to have arisen from the population of pixels in R. Start a new region.
Many other statistical and distance-based methods have also been proposed for region growing.
![Page 70: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/70.jpg)
70
Region growing
image
segmentation
![Page 71: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/71.jpg)
Region growing
71
![Page 72: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/72.jpg)
Split and Merge Segmentation
2 Stage Algorithm:
Stage 1: Split Split image into regions using a Quad Tree representation.
Stage 2: Merge Merge "leaves" of the Quad Tree which are neighboring
and "similar".
72
![Page 73: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/73.jpg)
Quad Tree Representation
73
![Page 74: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/74.jpg)
Split and Merge Example
74
![Page 75: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/75.jpg)
Graph Based Segmentation Segmentation can be viewed as a graph partitioning problem
75
![Page 76: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/76.jpg)
Forming Graph
Each pixel is treated as a node in a graph.
Each pixel has an edge to its neighbours. (e.g. 8 adjacent neighbours in 2D).
Edge weight w(i,j) is a function of the similarity between nodes i and j.
high weight vertices are probably part of the same element.
Low weight nodes are probably part of different element.
76
![Page 77: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/77.jpg)
Images as graphs
Nodes for every pixel
Edge between every pair of pixel (or every pair of “sufficiently close” pixels)
Each edge is weighted by the affinity or similarity of the two nodes
wij
i
j
77
![Page 78: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/78.jpg)
Constructing a Graph from an Image
Two kinds of vertices
Two kinds of edges
Cut-Segmentation
78
![Page 79: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/79.jpg)
What is a “cut”?
A graph G = (V,E) can be partitioned into two disjoint sets,
by simply removing edges connecting the two parts.
The degree of dissimilarity between these two pieces can be computed as total weight of the edges that have been removed. In graph theoretic language it is called the cut:
BvAu
vuwBAcut,
,,
79
![Page 80: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/80.jpg)
Example cut
80
![Page 81: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/81.jpg)
Graph Cut
Each cut corresponds to some cost (cut ): sum of the weights for the edges that have been removed.
BvAu
vuwBAcut,
,,
81
![Page 82: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/82.jpg)
Graph Based Algorithms
Graph Cut-Wu and Leahy 1993
Segmentation by normalized-cut
82
![Page 83: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/83.jpg)
Interactive Image Segmentation
Magic Wand (198?)
Intelligent Scissors [Mortensen and Barrett, 1995]
GrabCut[Rother et al., 2004]
Graph Cuts [Boykov and Jolly, 2001]
LazySnapping[Li et al., 2004]
83
![Page 84: Edge Detection & Image Segmentation Dr. Md. Altab Hossain Associate Professor Dept. of Computer Science & Engineering, RU 1](https://reader038.vdocument.in/reader038/viewer/2022110404/56649e935503460f94b98c93/html5/thumbnails/84.jpg)
Thank You
84