boundary extraction in natural images using ultrametric contour

Boundary Extraction in Natural Images Using Ultrametric Contour Maps

Pablo Arbeláez Université Paris Dauphine

Presented byDerek Hoiem

What is segmentation?

What is segmentation?

• Segmentation is a result

What is segmentation?

• Segmentation is a result

• Segmentation is a process

Woman

Face

What is segmentation?

• Segmentation is a result

• Segmentation is a process

• Segmentation is a guide

Segmentation as a Guide

• Multiple Segmentations

Segmentation as a Guide

• Multiple Segmentations

• Hierarchy of Segmentations

Key Concepts/Contributions

• Hierarchical segmentation by iterative merging

• Ultrametric dissimilarities

• Thorough evaluation on BSDS

Hierarchical Segmentation

λ

3 Region Image Dendrogram

Contour Image

Ultrametric Contour Map

• Ultrametric– Definition: D(x,y) <= max{ D(x,z), D(z,y) }

The union R12 of two regions R1 and R2 must have >= distance to adjacent region R3 than either R1 or R2

λ

Ultrametric Contour Map

Region Dissimilarity

1. Dc(R1, R2): mean boundary contrast– contrast(x) = max L*a*b* diff within radius of x

2. Dg(R1, R2): mean boundary gradient– gradient(x) = Pb(x)

3. Da(R1): Area + α3 Scatter (in color space)

D(R1, R2) = [Dc(R1, R2) + α1 Dg(R1, R2)] · min{ Da(R1) , Da(R2) }α2

Learned Parameters: xi = 4.5 α1 = 5 α2 = 0.2 α3 = 0

Examples

Contrast

Contrast + Gradient

Contrast + Gradient + Region

Algorithm Summary

• Create Initial Contours:– Extrema in gray channel form regions– Assign pixels to regions based on above

ultrametric

• Iteratively merge regions– Keep adjacency/distance matrix

Comparison

• Martin et al. (Pb)• Canny edge detector• Hierarchical watersheds (using MFM for gradient)

[Najman and Schmitt 1996]• Variational (global energy minimization)

Pb

No Boundary

Boundary

[Martin Fowlkes Malik 2004]

Oriented Edges

Brightness Gradient

Color Gradient

Texture Gradient

Pb

Variational Method

[Koepfler Lopez Morel 1994]

Originally Wavelet-based Textons

Comparison• MFM: Martin et al. (Pb)• Canny: Canny edge detector• WS: Hierarchical watersheds (using MFM for gradient) [Najman and Schmitt 1996]• MS: Variational (global energy minimization)

Edge-Based Region-Based

Comparison

Results

Results

Best Results

http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

Best Results

http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

Best Results

http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

Best Results

http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

Median Results

Median Results

Median Results

Median Results

Worst Results

Worst Results

Worst Results

Worst Results

Hierarchies vs. Multiple Segmentations

Revising Segmentation