Manuchehr AminianAdvisor: Prof. Andrew Knyazev

University of Colorado Denver

Algorithms for Generalized Image Segmentation

What is image segmentation?

Given a digital image, we want to separate/partition it into a small number of segments

Guidelines: Pixels in a given segment should be similar in some

respect (for example, color or intensity) Pixels in adjacent segments should be dissimilar Segments should be contiguous regions

Some good/bad examples

Let’s look at some good and bad segmentations with this image.

Source:

http://www.reggie.net/photos/ireland/sligo/carrowkeel/4737304_black_and_white_sheep-600.jpg

Good example

Each segment has similar colors, different segments are dissimilar, and each segment is contiguous.

Bad example

The first segment is okay, but the second one has many pixels which are dissimilar.

What we use:Spectral image segmentation

Using so-called “spectral” image segmentation satisfies all our requirements.

Trust me. A mathematician said it works.

What is the algorithm?

A physical interpretation

In physics, a classic problem is to attach masses to each other by springs and examine how they oscillate

Masses connected by a stiffer spring have a tendency to oscillate together

A physical interpretation (cont. 1)

On our image, connect a pixel to all its surrounding neighbors with “springs”. We connect pixels which are alike (in color or intensity) with stiff springs

For example, if we take this image...

A physical interpretation (cont. 2) Doing this for all the pixels in the image, we

give it a shake. The result?

All the reds vibrate together, as do the blues. So, put all the reds/oranges into one

segment, and all the blues/greens into the other.

Generalize to three dimensions!

We follow the same basic procedure in three dimensions. The only thing that changes is the grid.

Examples of 3D images

The third dimension is either time or spatial Some examples:

– 3D images created from a 3D scanner or an MRI machine

– Any animations or videos

Interesting things can happen

Compare the following two segmentations:

Why are they so different?

The first image was taken from the 3D algorithm run over the entire animation, whereas the second one was taken from the 2D algorithm run on the single frame.

Conclusions

Spectral segmentation: Satisfies all our guidelines Has a (relatively) easy-to-understand physical

interpretation Using freely available math software packages

HYPRE and BLOPEX, this can be scaled up to parallel computers

Thanks to:

UC Denver UROP, providing financial support for the research

The University of Colorado Denver