Institute of InformaticsEotvos Lorand University

Budapest, Hungary

Basic Algorithms for Digital Image Analysis:

a course

Dmitrij Csetverikov

with help of Attila Lerch, Judit Verestoy, Zoltan Megyesi, Zsolt Jankoand Levente Hajder

http://visual.ipan.sztaki.hu

Lecture 11: Image segmentation

• Discussion of grey-level thresholding

◦ Examples of thresholding◦ Imroving the histogram for better peak separation◦ Thresholding versus edge detection◦ Limits of thresholding

• Region-oriented segmentation

◦ Definition◦ Region growing and merging◦ Quadtrees◦ Split-and-merge algorithms

• Other segmentation methods

2

Examples of thresholding

images Otsu results Gaussian results

• Here, both methods give acceptable results.

• The Gaussian algorithm sets lower thresholds in both cases.

⇒ Fits object contours better than Otsu.

3

fingerprint image image histogram Otsu T = 158 Gauss T = 199

• Here, both methods still give satisfactory results.

• The Otsu algorithm sets threshold in valley of histogram.

⇒ Fingerprint lines are well-separated.

• The Gaussian algorithm sets slightly high threshold.

⇒ Some fingerprint lines touch.

4

low-quality image image histogram Otsu T = 159 Gauss T = 201

• Here, only Otsu gives a satisfactory result.

• The Otsu algorithm finds the small class of pixels (dark discs).

• The Gaussian algorithm tries to separate two high peaks formed by thebackground. Noisy valley is selected because the true class is

◦ too small◦ too far

5

image histogram Otsu result

• Here, only Otsu algorithm produces results.

• Gaussian algorithm gives no results at all.

◦ Upper row: unimodal histogram, no approximation obtained◦ Lower row: an approximation obtained, but the threshold equation has no

real root.

6

Imroving the histogram for better peak separation

Use of gradient to improve histogram: Combine intensity and gradient informationfor better separation of objects and background.

• Pixels close to edges have high gradients and medium intensities.

• Pixels of object and background have low gradient and low or high intensities.

• To better separate objects from background, discard high gradient pixels whencomputing the histogram.

P(i)

ibackground

object original

improved

T

T

edge

Principle of histogram peak separation.

7

Thresholding versus edge detection

• Thresholding with a constant threshold is a global operation.

◦ Advantage: Closed contours guaranteed◦ Drawback: Not applicable to images with uneven illumination

• Edge detection is a local operation

◦ Advantage: Applicable to images with uneven illumination◦ Drawback: Closed contours not guaranteed

f (x)x

f(x)thresholds

Signal with varying level that cannot be thresholded. Edges can be detected.

8

Limits of thresholding

stone crack Otsu method Another method

Merit (quality) of thresholding is task-dependent.

• The merit may include geometric properties.

• Image histogram does not account for geometry.

◦ The crack is detected as set of bright pixels independently of the crack shape.

9

stone Otsu method Another method

In this example, the merit of thresholding is uncertain.

Limits of thresholding:

• No geometric informaton is taken into account.

⇒ Compact and connected regions are not guaranteed.

◦ Select threshold, then arbitrarily interchange pixels in image, select again ⇒same threshold

• Solution: Combine intensity and geometry using region-oriented methods.

10

Region-oriented segmentation

Goal: Obtain homogeneous connected regions.

Basic formulation of segmentation: Partition the image R into n subregionsR1, R2, . . . , Rn so that

(a)n⋃i=1

Ri = R

(b) Ri is a connected region, i = 1, 2, . . . , n

(c) Ri ∩Rj = ∅ for all i and j, i 6= j

(d) P (Ri) = TRUE for i = 1, 2, . . . , n

(e) P (Ri ∪Rj) = FALSE for i 6= j

where P (Ri) is a logical homogeneity predicate over the points in region Ri:P (Ri) = TRUE means that all pixels in Ri have similar properties, that is, Ri ishomogeneous.

11

Examples of homogeneity predicates for a region:

• Difference between max and min greyvalues is small.

• Difference between any pixel and mean greyvalue is small.

Meaning of conditions (a–e):

(a) Completeness: Each pixel is assigned to a region.

(b) Connectedness: Points in each region are connected.

(c) Regions are disjoint.

(d) Each region is homogeneous.

(e) Any union of regions is inhomogeneous: Minimise number of regions.

12

Region growing and region merging

Region growing: Group pixels or subregions into larger regions while thehomogeneity criterion is satisfied.

A simple form of region growing is pixel aggregation:

• Select seed points and a homogeneity criterion.

• Grow regions around each seed point by appending those neighboring pixels thathave similar properties.

Pixel aggregation is usually followed by region merging.

• Region merging is a form of region growing.

• It can be applied as a separate operation, or be incorporated into an iterativesegmentation algorithm.

13

Algorithm 1: Region growing by pixel aggregation

1. Initialisation: Select N seed pixels Si, i = 1, 2, . . . , N , and a threshold T .

Initialise N regions R(0)i as the seed pixels Si. Initialise N region means M

(0)i

as the greyvalues of Si.

2. At each iteration k, find border pixels of each region R(k)i .

3. Search the 8-neighbourhood of each border pixel of each region. Assign a pixel

p(x, y) to R(k)i if

• p(x, y) is a 8-neighbour of a border pixel of R(k)i and

• p(x, y) has not been assigned yet and

• greyvalue of p is sufficiently close to region mean: |f(x, y)−M (k)i | ≤ T

4. Stop if no region growing was possible. Otherwise, update M(k)i and iterate by

going to step 2.

14

Different homogeneity criteria can be used. In the example below:

• the criterion is the maximum allowed absolute difference T within region;

• region merging is applied after growing.

0 0 5 6 71 1 5 8 70 1 6 7 72 0 7 6 60 1 5 6 5

a a b b ba a b b ba a b b ba a b b ba a b b b

a a a a aa a a a aa a a a aa a a a aa a a a a

(a) (b) (c)

Example of pixel aggregation followed by region merging: (a) Original imagematrix with two seeds indicated. (b) Result for T = 4. (c) Result for T = 8.

Major problem with pixel aggregation: How to select seeds?

• Result depends heavily on choice and order of seeds.

15

Region merging

Region merging is a form of region growing. This operation is applied whenadjacent regions have similar properties.

• For example, this happens in pixel aggregation when several seeds are given forone region.

Different merging criteria: Merge two adjacent regions Ri and Rj if

• Pagg(Ri ∪Rj) = TRUE, where Pagg is the criterion used for aggregation, or

• P (Ri ∪Rj) = TRUE, where P is another homogeneity predicate, or

• border between Ri and Rj has no strong edges (gradient > Thigh), or

• border between Ri and Rj has many weak locations (gradient < Tlow),

16

Region growing by merging

Merging and growing can be combined with image division (splitting). This can bedone in different ways. The goal is to avoid interactive specification of seeds.

Basic idea of region growing by merging:

• Divide image into atomic regions of constant greyevel, or other local propery.

• Merge similar adjacent regions sequentially until the adjacent regions becomesufficiently different.

R3

R55R

R

weak boundary

3

R1

R4

R2

mergeRm=R11 UR2UR4

Region growing by merging.

17

Summary of region growing

Advantages of region growing:

• Connected regions guaranteed.

• Possibility to incorporate priori knowledge about regions: shape, size, texture,spatial relations, etc.

⇒ Regions obtained have desired properties.

Drawbacks:

• Prior information may be needed: seeds, starting points.

• Heuristics needed: for example, merging rules (‘weak borders’, etc.)

• Intrinsically sequential: no parallel implementation possible.

• Order-dependence: Result depends on the order in which pixels are examined.

Possible solutions: Look-ahead, back-tracking.

18

Split-and-merge method and quadtrees

The split-and-merge algorithm uses a homogeneity predicate P and has two stages:

• Top-down: Split image into homogeneous quadrant regions

• Bottom-up: Merge similar adjacent regions

Algorithm 2: Split-and-merge algorithm

1. Top-down: Successively subdivide image and regions into smaller quadrantregions until P (Ri) = TRUE for each Ri. Obtain a quadtree structure.

2. Bottom-up: At each level, merge any adjacent regions Ri and Rj for whichP (Ri ∪Rj) = TRUE.

3. Repeat steps 1 and 2 until no further splitting/merging is possible.

19

A R

R4

D C

2

A

A

D C

B

D C

B 1B

3

21

Split Merge

stages of segmentation

4

A B D C A B C D A B C D

1 2 31 (A,B,D)2 (A,B,C)3 (B,C,D)

1C,2D,3A

4

1R R2

quadtree split and merge segmented regions

Segmentation by split-and-merge approach.

20

Summary of split-and-merge

Advantages of split-and-merge:

• Connected regions guaranteed

• Some (limited) possibility to incorporate geometric knowledge

Drawbacks:

• Result may depend on position and orientation of image with respect to raster

• Imposes rectangular structure on segmented image

• Relatively large computational load and storage requirements

• Simple quadtree only supports vertical dataflow

◦ Adjacent regions to be merged can be ‘far away’ in quadtree

Quadtrees are widely used in GIS (Geographic Information Systems).

21

Comparison of a growing and a merging algorithm

image with seeds grow T = 20 grow T = 30 grow T = 40

• When T = 20, face regions with large variation cannot grow.

⇒ Dark hair region with small variation spreads and occupies large area

• When T = 30, face regions can grow and occupy larger areas.

⇒ Dark hair region shrinks

• Result depends on order and positions of seeds.

22

merge T = 20 merge T = 30 merge T = 40

merge T = 20 merge T = 40 grow T = 20 grow T = 40

• As T increases, number of regions decreases, merging segmentation becomescoarser. Dark hair region extends.

◦ In region growing, the effect is just the opposite: dark region shinks.

23

Comparison of split-and-merge and region merging

cell image SaM T = 40 SaM T = 50 SaM T = 60

merge T = 40 merge T = 50 merge T = 60

• Rectangular structure of splitting is visible in split-and-merge images.

• Merging segmentation looks finer.

24

Other segmentation methods

• Model-based segmentation: Partition image into segments of known shape andother properties.

• Edge-based segmentation: Find object contours by edge tracing.

• Texture-based segmentation: Apply region-based techniques to texturalproperties of regions.

• Motion-based segmentation: In an image sequence, locate objects based on theirmotion.

• Rule-based segmentation: Expert system with general-purpose knowledge aboutimages and grouping criteria.

25