ieee international conference on image processing, icip ... · cvsp group, national technical...

Computer Vision, Speech Communication and Signal Processing Group

National Technical University of Athens, GreeceSchool of Electrical and Computer Engineering

URL: http://cvsp.cs.ntua.gr

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Iasonas Kokkinos, Giorgos Evangelopoulos and Petros Maragos

Modulation-Feature based Textured Image Segmentation

Using Curve Evolution

CVSP Group, National Technical University of Athens


Presentation Overview and MotivationExtracting texture features for segmentation:

Intensity does not suffice.

High dimensional features, common for texture description (e.g. Gabor filterbanks, orthonormal bases etc.) suboptimal segmentation

In this work, we propose a low dimensional texture descriptor, based on the AM-FM image model.

Unsupervised segmentation using state of the art techniques:

Curve Evolution & Level-Set methods combine efficiency, elegance and mathematical tractability.

Region Based Terms guarantee robustness, which is essential to texture segmentation.



Image AM-FM Modulation Model

Estimate amplitude & frequency coefficients via• Multiband Gabor filtering Narrowband image components

• Demodulation using 2D Energy Operator

and the ESA

( ) ( ) ( )1

, , cos , ,K

k kk

I x y a x y x yf=

È ˘= ◊ Î ˚Â

Modulation Features for Texture Analysis (I)

( ) ( ) ( ), , , ,k kI x y I x y h x y= *

( )( ) ( )

( ),kk

k k

a x yI

I x I yY

Y Yª

+∂ ∂ ∂ ∂

( ) 1( ) ( , ) ,k k kI x I x yY ∂ ∂ Y ª W ( ) 2( ) ( , )k k kI y I x yY ∂ ∂ Y ª W

Refs: P. Maragos & A.C. Bovik, JOSA 95, J.E. Daugman, JOSA 85, Havlicek, Harding & Bovik, 1996, 2000.

( ) ( ), ,kk x y x yf— = W

2 2( )I I I IY — - —



Modulation Features for Texture Analysis (II)

Dominant Components Analysis (DCA) proposes using at each pixel only the most prominent channel, j

Use maximization criterion for choosing among channels:

Amplitude-Based DCA Teager Energy-Based DCA

Refine results with texture vs. non-texture mask at various scales using multiple statistical hypothesis testing.

Using a single channel amounts to locally modeling the texture with a Gabor-like ‘texton’ whose features are described by the DCA components.

Refs: Havlicek, Harding & Bovik, T-Image 2000. Kokkinos, Evangelopoulos & Maragos, ICIP 04.

( ) ( )( ), ,k kx y I h x yÈ ˘G = Y *Î ˚( )( )

,)| (

,max |

k

kk x y H

a x y

W WG =

( ) ( ), , ,ja x y a x y= | ( , ) | | ( , ) |jx y x yW = W

1

arg max kk K

j£ £

= GG K



Modulation Features Extraction Examples

( ),a x y ( )1 ,x yWReal Modulation Parameters

DCA Estimated Parameters( )2 ,x yWSynthetic AM-FM

ADCA EDCA

Amplitude

Frequency Magnitude



Modulation Features Extraction Examples

Texture vs. Non-Texture

EDCA Estimates: Amplitude and Spatial Frequencies



Region Based Segmentation by Active Contours

Functional expressing segmentation cost(Region Competition Algorithm):

Model multivariate features within each region using a Gaussian pdf :

( )( ) ( )1

[ , ] log , 2

N

i i i

ii

vE C p p dx Len CR

Y=

= - +ÂÚÚ

1R2R

3R

3p2p

1p

( )( )

( ) ( )1

/ 2 1/ 2

1

2

1; , exp2 | |

i i ii i i di

Y Yp I µ µµπ

−Τ− − Σ − Σ = Σ

1 ,..., NC C C=

Refs: Zhu & Yuille, PAMI 96 Paragios & Deriche, JVCIR 02

Rousson, Brox & Deriche, CVPR 03

1[ ,..., ]dY Y Y T=



Euler-Lagrange equations lead to:

Level Set Implementation: view front as zero set of embedding function:

Functional Minimization Algorithm

( )log( )

i ii i

d p Yvdt p Yc

kÊ ˆF

= + —FÁ ˜Ë ¯

( )log( )

i ii i

c

C p Yv Nt p Y

kÊ ˆ∂

= +Á ˜∂ Ë ¯

iF

Refs: Osher & Sethian, 1988.



Unsupervised Segmentation by Active Contours

Unsupervised segmentation pdfs not known a priori

Fronts are initiated so that the union of their interiors occupies the whole of the image

Iterate: Estimate the parameters of the Gaussian p.d.f. for each region, using the front’s current position:

Evolve fronts in the direction dictated by region competition (statistics force + geometrical information)

1| |

i

ii R

ii

YR

µ∈

= ∑ ( )( )1| |

i

i i i ii R

ii

Y YR

µ µΤ

∈

Σ = − −∑

Refs: Zhu & Yuille, PAMI 1996, Rousson et. al., CVPR 2003



Modulation Features for Texture Segmentation

Dominant Component Features provide a low-dimensional and rich texture descriptor, that contains local information about

• Oscillation Amplitude• Frequency (Scale)• Orientation (Variation)

Features for segmentation: Amplitude Horizontal & Vertical Frequency ( or ) Frequency Magnitude & OrientationIntensity

1 2[ , , , ]Y a I Τ= Ω Ω [ ,| |, , ]Y a I Τ= Ω ∠Ω



Segmentation Examples Using Modulation Features

Segmentation RegionsSynthetic Textures

DCA Estimated Texture Features

a

a 1W 2W

W –W



1W 2Wa


Segmentation Regions




1W 2Wa




a W◊



Texture representation by simple, information rich, low-dimensional feature vector.

Important texture characteristics are captured with good localization.

Unsupervised segmentation scheme that combines the merits of region competition and modulation/DCA.

Efficient segmentation of a wide variety of natural textured images.

Summary & Conclusions



Evolution of related ideas

Zhu & Yuille, 1996: Region Competition (No level set implementation, fixed filterbank) Zray, Havlicek, Acton & Pattichis, 2001: Modulation features & GAC (no region based term, curve evolution used for post-processing)Paragios & Deriche, 2002: Textured Image Segmentation Using Level Set Methods (Supervised)Sagiv et al. , 2002. Vese et al, 2002 : Unsupervised textured Image Segmentation using active contours (Mostly heuristic methods fordimensionality reduction)Rousson, Brox & Deriche, 2003: Unsupervised Image Segmentation using structure tensor features. No scale information, anisotropic diffusion used for feature pre-processing.

Kokkinos, Evangelopoulos, Maragos, ICIP 2004: Modulation featureextraction and texture vs. non-texture decision as segmentation cues.



Appendix: 2D Energy Operator

Continuous

Discrete

2 2( )( , ) ( , ) ( , ) ( , )c f x y f x y f x y f x yF — - —22 2 2

2 2

f f f ff fx x y y

Ê ˆ Ê ˆÊ ˆ∂ ∂ ∂ ∂Ê ˆ= - + -Á ˜ Á ˜Á ˜ Á ˜Ë ¯∂ ∂ Ë ∂ ¯ ∂Ë ¯ Ë ¯

( )( , )d f i jF =22 ( , ) ( 1, ) ( 1, ) ( , 1) ( , 1)f i j f i j f i j f i j f i j- - + - - +



Appendix: 2D AM-FM Energy Tracking

Spatial AM-FM signal:

Energy tracking

( ) ( ) ( ), , cos ,f x y a x y x yjÈ ˘= Î ˚

( )1 , ,x yxj∂W∂ ( )2 ,x y

yf∂W∂

( )1 2,W = W W

[ ] 22cos( )c a ajF ª W22 2

1[ ]cf ax∂F ª W W∂

: Instantaneous Frequency vector



Appendix: 2D Continuous Energy Separation Algorithm

2D CESA

2D cosine:

CESA exact estimates:

1 1ˆ ( ) ( , )c c

f f x yx∂Ê ˆW = F F ª WÁ ˜Ë ¯∂ 2 2

ˆ ( ) ( , )c cf f x yy

Ê ˆ∂W = F F ª WÁ ˜Ë ∂ ¯

( ) ( )( )ˆ ( , )c

c c

fa a x yf x f y

F= ªF ∂ ∂ +F ∂ ∂

1 2( , ) cos( )c c of x y A x y j= W +W +

1 1( , ) cx yW = W

2 2( , ) cx yW = W( , )a x y = A

Æ



Appendix: 2D Gabor filterbanks and DCA

2 2 2( , ) exp[ ( ) ( ) 2 ] cos[2 ( )]h x y ax by Ux Vys p= - + ◊ +

2D Gabor function

DCA Block diagram, (by Havlicek et. al. 1996)



Appendix: Curve Evolution

is evolving curve (front) for is a simple smooth closed curve

Position vector =Normal vector =Speed: Curvature =

curve evolution PDE (flow)

( )tΓ 0t ≥

( , ) ( , )C p t VN p tt

∂ =∂

(0)Γ

( , )C p t( , )N p t

( , )K p t( )V F K=



Appendix: Level Set Method for Curve Evolution

Embed curve as zero-level curve of function :

Function evolution PDE:

Advantages of level set method:- remains a function even if topology of changes- Compute curvature & normal of from :

- Efficient Numerics: entropy-satisfying finite differences- Extends easily to 3D

( )tΓ ( , , )x y tΦ( ) ( , ) : ( , , ) 0t x y x y tΓ = Φ =

0( , ) ( , , 0) signed distance tranform from (0)x y x yΦ = Φ = Γ

Vt

∂Φ = ∇Φ∂

Φ ( )tΓ( )tΓ Φ

/N = −∇Φ ∇Φ( )div / Nκ = − ∇Φ ∇Φ = ∇ ⋅

Ref: Osher & Sethian 1988



Appendix: Region Competition

1R 2RiR

QR( )( )

1

[ , ] log 2

N

i i ii

i

vE C p p I dx CR=

= - +ÂÚÚ

Forces acting on the contours, (by Zhu and Yuille, 1996)



Minimize Image functional (generalized ‘energy’)

Euler PDE:

Gradient Descent:

Solution reached at steady state:

Example 1:

Example 2:

,[ ] ( , , , )x yD

E u F x y u u u dxdy= ∫∫

Euler derivative [ ]

0

u

u u ux y

F

u F F Ft x y

∂ ∂ ∂= − − =∂ ∂ ∂

[ ]uu Ft∂ = −∂

/ 0u t∂ ∂ =2 2

tF u u u= ∇ ⇒ =∇( ) ( ) /tF u u curv u u u= ∇ ⇒ = ∇ ∇

Appendix: Functional Minimization



Appendix: Energy Tracking in AM-FM Signals

Cont.-Time AM-FM

Cont.-Time TK Energy Operator:

Energy Tracking

Discrete-Time AM-FM

Discrete-Time TK Energy Operator:

Energy Tracking

[ ] )()()()( 2 txtxtxtxc −=Ψ [ ] 2[ ] [ ] [ 1] [ 1]d x n x n x n x nΨ = − + −

( )0( ) ( )cos ( )

tx t a t dω τ τ= ∫ ( )0

[ ] [ ]cos ( )n

x n a n m dm= Ω∫

[ ] 2 2( ) ( ) ( )c x t t tα ωΨ ≅ [ ] ( )2 2[ ] [ ]sin [ ]x n n nδ αΨ ≅ Ω



[ ][ ]

2 [ ][ ]

[ 1] [ 1]

x na n

x n x n

Ψ≅

Ψ + − −[ ]

[ ][ 1] [ 1]1 arcsin [ ]

2 4 [ ]x n x n

f nx nπ

Ψ + − −≅

Ψ

Appendix: 1D Continuous & Discrete-Time ESA

[ ( )] ( )[ ( )]x t a tx t

Ψ≅

Ψ1 [ ( )] ( )

2 [ ( )]x t f tx tπ

Ψ≅

Ψ

Assumption: x(t) is a narrowband AM-FM Signal

ieee international conference on image processing, icip ... · cvsp group, national technical...

Documents