chapter 4 image segmentation. outline introduction to segmentation point,line and edge detection ...
TRANSCRIPT
Chapter 4 Image Segmentation
Outline
Introduction to segmentationPoint,line and edge detectionThresholding techniquesWatershed segmentationRegion based segmentationApplications
2
3
Preview
• Segmentation is to subdivide an image into its component regions or objects.
• Segmentation should stop when the objects of interest in an application have been isolated.
4
Principal approaches
• Segmentation algorithms generally are based on one of 2 basis properties of intensity values– discontinuity : to partition an image based on
sharp changes in intensity (such as edges)– similarity : to partition an image into regions
that are similar according to a set of predefined criteria.
5
Detection of Discontinuities
• detect the three basic types of gray-level discontinuities– points , lines , edges
• the common way is to run a mask through the image
6
Point Detection
• a point has been detected at the location on which the mark is centered if
|R| T
• where – T is a nonnegative threshold – R is the sum of products of the coefficients with
the gray levels contained in the region encompassed by the mark.
7
Point Detection
• Note that the mark is the same as the mask of Laplacian Operation
• The only differences that are considered of interest are those large enough (as determined by T) to be considered isolated points.
|R| T
Example
8
9
Line Detection
• Horizontal mask will result with max response when a line passed through the middle row of the mask with a constant background.
• the similar idea is used with other masks.• note: the preferred direction of each mask is weighted with
a larger coefficient (i.e.,2) than other possible directions.
10
Line Detection
• Apply every masks on the image• let R1, R2, R3, R4 denotes the response of
the horizontal, +45 degree, vertical and -45 degree masks, respectively.
• if, at a certain point in the image
|Ri| > |Rj|,• for all ji, that point is said to be more likely
associated with a line in the direction of mask i.
11
Line Detection
• Alternatively, if we are interested in detecting all lines in an image in the direction defined by a given mask, we simply run the mask through the image and threshold the absolute value of the result.
• The points that are left are the strongest responses, which, for lines one pixel thick, correspond closest to the direction defined by the mask.
12
Example
13
Edge Detection
• we discussed approaches for implementing– first-order derivative (Gradient operator)– second-order derivative (Laplacian operator)
14
Ideal and Ramp Edges
because of optics, sampling, image acquisition imperfection
15
Thick edge• The slope of the ramp is inversely proportional to the
degree of blurring in the edge.• We no longer have a thin (one pixel thick) path.• Instead, an edge point now is any point contained in the
ramp, and an edge would then be a set of such points that are connected.
• The thickness is determined by the length of the ramp.• The length is determined by the slope, which is in turn
determined by the degree of blurring.• Blurred edges tend to be thick and sharp edges tend Blurred edges tend to be thick and sharp edges tend
to be thinto be thin
16
First and Second derivatives
the signs of the derivatives would be reversed for an edge that transitions from light to dark
17
Second derivatives
• produces 2 values for every edge in an image (an undesirable feature)
• an imaginary straight line joining the extreme positive and negative values of the second derivative would cross zero near the midpoint of the edge. (zero-crossing zero-crossing propertyproperty)
18
Noise in Images
• First column: images and gray-level profiles of a ramp edge corrupted by random Gaussian noise of mean 0 and = 0.0, 0.1, 1.0 and 10.0, respectively.
• Second column: first-derivative images and gray-level profiles.
• Third column : second-derivative images and gray-level profiles.
19
Noise in images(contd)
• fairly little noise can have such a significant impact on the two key derivatives used for edge detection in images
• image smoothing should be serious consideration prior to the use of derivatives in applications where noise is likely to be present.
20
Edge point
• to determine a point as an edge point– the transition in grey level associated with the
point has to be significantly stronger than the background at that point.
– use threshold to determine whether a value is “significant” or not.
– the point’s two-dimensional first-order derivative must be greater than a specified threshold.
21
Gradient Operator
• first derivatives are implemented using the magnitude of the gradientmagnitude of the gradient.
y
fx
f
G
G
y
xf
21
22
2122 ][)f(
y
f
x
f
GGmagf yx
the magnitude becomes nonlinearyx GGf
commonly approx.
22
Gradient Masks
23
Diagonal edges with Prewitt and Sobel masks
24
Example
25
Example
26
Example
27
Laplacian
2
2
2
22 ),(),(
y
yxf
x
yxff
(linear operator)
Laplacian operator
)],(4)1,()1,(
),1(),1([2
yxfyxfyxf
yxfyxff
28
Laplacian of Gaussian
• Laplacian combined with smoothing to find edges via zero-crossing.
2
2
2)( r
erh
where r2 = x2+y2, and is the standard deviation
2
2
24
222 )(
r
er
rh
29
Mexican hat
the coefficient must be sum to zero
positive central term surrounded by an adjacent negative region (a function of distance)zero outer region
30
Linear Operation
• second derivation is a linear operation
• thus, 2f is the same as convolving the image with Gaussian smoothing function first and then computing the Laplacian of the result
31
Example
a). Original imageb). Sobel Gradientc). Spatial Gaussian smoothing functiond). Laplacian maske). LoGf). Threshold LoGg). Zero crossing
32
Zero crossing & LoG
• Approximate the zero crossing from LoG image
• to threshold the LoG image by setting all its positive values to white and all negative values to black.
• the zero crossing occur between positive and negative values of the thresholded LoG.
33
Thresholdingimage with dark background and
a light object
image with dark background and two light objects
34
Multilevel thresholding
• a point (x,y) belongs to – to an object class if T1 < f(x,y) T2
– to another object class if f(x,y) > T2
– to background if f(x,y) T1
• T depends on – only f(x,y) : only on gray-level values Global threshold– both f(x,y) and p(x,y) : on gray-level values and its
neighbors Local threshold
35
The Role of Illumination
f(x,y) = i(x,y) r(x,y)
a). computer generated reflectance functionb). histogram of reflectance functionc). computer generated illumination function (poor)d). product of a). and c).e). histogram of product image
easily use global thresholdingobject and background are separated
difficult to segment
36
Basic Global Thresholding
generate binary image
use T midway between the max and min gray levels
37
Basic Global Thresholding• based on visual inspection of histogram1. Select an initial estimate for T.2. Segment the image using T. This will produce two
groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values T
3. Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2
4. Compute a new threshold value5. T = 0.5 (1 + 2)6. Repeat steps 2 through 4 until the difference in T in
successive iterations is smaller than a predefined parameter To.
38
Example: Heuristic methodnote: the clear valley of the histogram and the effective of the segmentation between object and background
T0 = 03 iterations with result T = 125
39
Basic Adaptive Thresholding
• subdivide original image into small areas.
• utilize a different threshold to segment each subimages.
• since the threshold used for each pixel depends on the location of the pixel in terms of the subimages, this type of thresholding is adaptive.
40
Example : Adaptive Thresholding
41
Further subdivisiona). Properly and improperly segmented subimages from previous example b)-c). corresponding histogramsd). further subdivision of the improperly segmented subimage.e). histogram of small subimage at topf). result of adaptively segmenting d).
Segmentation by Morphological Watersheds
Introduction• Based on visualizing an image in 3D
020
4060
80100
0
50
1000
5
10
15
20
25
imshow(I,[ ]) mesh(I)
• Instead of working on an image itself, this technique is often applied on its gradient image.– In this case, each object is
distinguished from the background by its up-lifted edges
020
4060
80100
0
20
40
60
80
1000
2
4
6
8
Basic Definitions
• I: 2D gray level image• DI: Domain of I• Path P of length l between p and q in I
– A (l +1)-tuple of pixels (p0=p,p1,…,pl=q) such that pi,pi+1 are adjacent (4 adjacent, 8 adjacent, or m adjacent, see Section 2.5)
• l(P): The length of a given path P• Minimum
– A minimum M of I is a connected plateau of pixels from which it is impossible to reach a point of lower altitude without having to climb
Basic Definitions• Instead of working on an image itself, this technique is often applied
on its gradient image.• Three types of points
– Points belonging to a regional minimum– Catchment basin / watershed of a regional minimum
• Points at which a drop of water will certainly fall to a single minimum– Divide lines / Watershed lines
• Points at which a drop of water will be equally likely to fall to more than one minimum
• Crest lines on the topographic surface• This technique is to identify all the third type of points for
segmentation
Basic Steps1. Piercing holes in
each regional minimum of I
2. The 3D topography is flooded from below gradually
3. When the rising water in distinct catchment basins is about to merge, a dam is built to prevent the merging
The dam boundaries correspond to the watershed lines to be extracted by a watershed segmentation algorithm
- Eventually only constructed dams can be seen from above
Dam Construction
• Based on binary morphological dilation• At each step of the algorithm, the binary
image in obtained in the following manner1. Initially, the set of pixels with minimum gray
level are 1, others 0.2. In each subsequent step, we flood the 3D
topography from below and the pixels covered by the rising water are 1s and others 0s. (See previous slides)
Notations• M1, M2:
– Sets of coordinates of points in the two regional minima
• Cn-1(M1), Cn-1(M2)– Sets of coordinates of points
in the catchment basins associated with M1 M2 at stage n-1 of flooding (catchment basins up to the flooding level)
• C[n-1]– Union of Cn-1(M1), Cn-1(M2)
Dam Construction• At flooding step n-1, there are
two connected components. At flooding step n, there is only one connected component
– This indicates that the water between the two catchment basins has merged at flooding step n
– Use “q” to denote the single connected component
• Steps– Repeatedly dilate Cn-1(M1), Cn-1(M2)
by the 3×3 structuring element shown, subject to the following condition
• Constrained to q (center of the structuring element can not go beyond q during dilation
Dam Construction• The dam is constructed by
the points on which the dilation would cause the sets being dilated to merge.
– Resulting one-pixel thick connected path
• Setting the gray level at each point in the resultant path to a value greater than the maximum gray value of the image. Usually max+1
Watershed Transform• Denote M1, M2, …, MR as the sets of the coordinates of the points in
the regional minima of an (gradient) image g(x,y)
• Denote C(Mi) as the coordinates of the points in the catchment basin associated with regional minimum Mi.
• Denote the minimum and maximum gray levels of g(x,y) as min and max
• Denote T[n] as the set of coordinates (s,t) for which g(s,t) < n
• Flood the topography in integer flood increments from n=min+1 to n=max+1
• At each flooding, the topography is viewed as a binary image
Watershed Transform• Denote Cn(Mi) as the set of
coordinates of points in the catchment basin associated with minimum Mi at flooding stage n.– Cn(Mi)= C(Mi) T[n]
– Cn(Mi)=T[n]
• Denote C[n] as the union of the flooded catchment basin portions at stage n:
• Initialization– Let C[min+1]=T[min+1]
• At each step n, assume C[n-1] has been constructed. The goal is to obtain C[n] from C[n-1]
C(n)R
ii
R
iin MCCMCnC
11
)(]1[max and )(][
Watershed Transform• Denote Q[n] as the set of connected components in T[n]. • For each qQ[n], there are three possibilities
1. q C[n-1] is empty (q1)
• A new minimum is encountered• q is incorporated into C[n-1] to form C[n]
2. q C[n-1] contains one connected component of C[n-1] (q2)
• q is incorporated into C[n-1] to form C[n]
3. q C[n-1] contains more than one connected components of C[n-1] (q3)
• A ridge separating two or more catchment basins has been encountered
• A dam has to be built within q to prevent overflow between the catchment basins
4. Repeat the procedure until n=max+1
Examplea: Original imageb: Gradient image of image ac: Watershed lines obtained
from image b (oversegmentation)
Each connected region contains one local minimum in the corresponding gradient image
d: Watershed lines obtained from smoothed image b
The Use of Markers• Internal markers are used to limit the number of regions by
specifying the objects of interest
– Like seeds in region growing method
– Can be assigned manually or automatically– Regions without markers are allowed to be merged (no
dam is to be built)
• External markers those pixels we are confident to belong to the background
– Watershed lines are typical external markers and they belong the same (background) region
Watershed Based Image Segmentation
1. Use internal markers to obtain watershed lines of the gradient of the image to be segmented.
2. Use the obtained watershed lines as external markers3. Each region defined by the external markers contains a single
internal marker and part of the background4. The problem is reduced to partitioning each region into two
parts: object (containing internal markers) and a single background (containing external markers)
– Global thresholding, region growing, region splitting and merging, or watershed transform
59
Region-Based Segmentation - Region Growing
• start with a set of “seed” points
• growing by appending to each seed those neighbors that have similar properties such as specific ranges of gray level
60
Region Growing
select all seed points with gray level 255
criteria:1. the absolute gray-
level difference between any pixel and the seed has to be less than 65
2. the pixel has to be 8-connected to at least one pixel in that region (if more, the regions are merged)
• Region growing: Groups pixels or sub-region into larger regions.– step1:
• Start with a set of “seed” points and from these grow regions by appending to each seed those neighboring pixels that have properties similar to the seed.
– step2:• Region splitting and merging
61
Region-based segmentation
• Advantage:– With good connectivity
• Disadvantage:– Initial seed-points:
• different sets of initial seed-point cause different segmented result
– Time-consuming problem
62
Region-based segmentation
• Unseeded region growing:– no explicit seed selection is necessary, the
seeds can be generated by the segmentation procedure automatically.
– It is similar to SRG except the choice of seed point
63
Region-based segmentationUnseeded region growing
• Advantage:– easy to use– can readily incorporate high level knowledge of
the image composition through region threshold
• Disadvantage:– slow speed
64
Region-based segmentationUSRG
Region-based segmentationfast scanning
• Fast scanning Algorithm: – The fast scanning
algorithm somewhat resembles unseeded region growing
– the number of clusters of both two algorithm would not be decided before image passing through them.
65
66
Region-based segmentationfast scanning
Region-based segmentationfast scanning
• Last step:
– merge small region to big region
67
• Advantage:– The speed is very fast– The result of segmentation will be intact with good
connectivity
• Disadvantage:– The matching of physical object is not good
• It can be improved by morphology and geometric mathematic
68
Region-based segmentationfast scanning