ieee international conference on image processing, icip ... · cvsp group, national technical...

Computer Vision, Speech Communication and Signal Processing Group

National Technical University of Athens, GreeceSchool of Electrical and Computer Engineering

URL: http://cvsp.cs.ntua.gr

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Iasonas Kokkinos, Giorgos Evangelopoulos and Petros Maragos

Modulation-Feature based Textured Image Segmentation

Using Curve Evolution

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Presentation Overview and MotivationExtracting texture features for segmentation:

Intensity does not suffice.

High dimensional features, common for texture description (e.g. Gabor filterbanks, orthonormal bases etc.) suboptimal segmentation

In this work, we propose a low dimensional texture descriptor, based on the AM-FM image model.

Unsupervised segmentation using state of the art techniques:

Curve Evolution & Level-Set methods combine efficiency, elegance and mathematical tractability.

Region Based Terms guarantee robustness, which is essential to texture segmentation.

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Image AM-FM Modulation Model

Estimate amplitude & frequency coefficients via• Multiband Gabor filtering Narrowband image components

• Demodulation using 2D Energy Operator

and the ESA

( ) ( ) ( )1

, , cos , ,K

k kk

I x y a x y x yf=

È ˘= ◊ Î ˚Â

Modulation Features for Texture Analysis (I)

( ) ( ) ( ), , , ,k kI x y I x y h x y= *

( )( ) ( )

( ),kk

k k

a x yI

I x I yY

Y Yª

+∂ ∂ ∂ ∂

( ) 1( ) ( , ) ,k k kI x I x yY ∂ ∂ Y ª W ( ) 2( ) ( , )k k kI y I x yY ∂ ∂ Y ª W

Refs: P. Maragos & A.C. Bovik, JOSA 95, J.E. Daugman, JOSA 85, Havlicek, Harding & Bovik, 1996, 2000.

( ) ( ), ,kk x y x yf— = W

2 2( )I I I IY — - —

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Modulation Features for Texture Analysis (II)

Dominant Components Analysis (DCA) proposes using at each pixel only the most prominent channel, j

Use maximization criterion for choosing among channels:

Amplitude-Based DCA Teager Energy-Based DCA

Refine results with texture vs. non-texture mask at various scales using multiple statistical hypothesis testing.

Using a single channel amounts to locally modeling the texture with a Gabor-like ‘texton’ whose features are described by the DCA components.

Refs: Havlicek, Harding & Bovik, T-Image 2000. Kokkinos, Evangelopoulos & Maragos, ICIP 04.

( ) ( )( ), ,k kx y I h x yÈ ˘G = Y *Î ˚( )( )

,)| (

,max |

k

kk x y H

a x y

W WG =

( ) ( ), , ,ja x y a x y= | ( , ) | | ( , ) |jx y x yW = W

1

arg max kk K

j£ £

= GG K

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Modulation Features Extraction Examples

( ),a x y ( )1 ,x yWReal Modulation Parameters

DCA Estimated Parameters( )2 ,x yWSynthetic AM-FM

ADCA EDCA

Amplitude

Frequency Magnitude

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Modulation Features Extraction Examples

Texture vs. Non-Texture

EDCA Estimates: Amplitude and Spatial Frequencies

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Region Based Segmentation by Active Contours

Functional expressing segmentation cost(Region Competition Algorithm):

Model multivariate features within each region using a Gaussian pdf :

( )( ) ( )1

[ , ] log , 2

N

i i i

ii

vE C p p dx Len CR

Y=

= - +ÂÚÚ

1R2R

3R

3p2p

1p

( )( )

( ) ( )1

/ 2 1/ 2

1

2

1; , exp2 | |

i i ii i i di

Y Yp I µ µµπ

−Τ− − Σ − Σ = Σ

1 ,..., NC C C=

Refs: Zhu & Yuille, PAMI 96 Paragios & Deriche, JVCIR 02

Rousson, Brox & Deriche, CVPR 03

1[ ,..., ]dY Y Y T=

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Euler-Lagrange equations lead to:

Level Set Implementation: view front as zero set of embedding function:

Functional Minimization Algorithm

( )log( )

i ii i

d p Yvdt p Yc

kÊ ˆF

= + —FÁ ˜Ë ¯

( )log( )

i ii i

c

C p Yv Nt p Y

kÊ ˆ∂

= +Á ˜∂ Ë ¯

iF

Refs: Osher & Sethian, 1988.

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Unsupervised Segmentation by Active Contours

Unsupervised segmentation pdfs not known a priori

Fronts are initiated so that the union of their interiors occupies the whole of the image

Iterate: Estimate the parameters of the Gaussian p.d.f. for each region, using the front’s current position:

Evolve fronts in the direction dictated by region competition (statistics force + geometrical information)

1| |

i

ii R

ii

YR

µ∈

= ∑ ( )( )1| |

i

i i i ii R

ii

Y YR

µ µΤ

∈

Σ = − −∑

Refs: Zhu & Yuille, PAMI 1996, Rousson et. al., CVPR 2003

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Modulation Features for Texture Segmentation

Dominant Component Features provide a low-dimensional and rich texture descriptor, that contains local information about

• Oscillation Amplitude• Frequency (Scale)• Orientation (Variation)

Features for segmentation: Amplitude Horizontal & Vertical Frequency ( or ) Frequency Magnitude & OrientationIntensity

1 2[ , , , ]Y a I Τ= Ω Ω [ ,| |, , ]Y a I Τ= Ω ∠Ω

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Segmentation Examples Using Modulation Features

Segmentation RegionsSynthetic Textures

DCA Estimated Texture Features

a

a 1W 2W

W –W

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

1W 2Wa

Segmentation Examples Using Modulation Features

Segmentation Regions

DCA Estimated Texture Features

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

1W 2Wa

Segmentation Examples Using Modulation Features

Segmentation Regions

DCA Estimated Texture Features

a W◊

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Segmentation Examples Using Modulation Features

Segmentation Regions

Segmentation Regions

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Texture representation by simple, information rich, low-dimensional feature vector.

Important texture characteristics are captured with good localization.

Unsupervised segmentation scheme that combines the merits of region competition and modulation/DCA.

Efficient segmentation of a wide variety of natural textured images.

Summary & Conclusions

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Evolution of related ideas

Zhu & Yuille, 1996: Region Competition (No level set implementation, fixed filterbank) Zray, Havlicek, Acton & Pattichis, 2001: Modulation features & GAC (no region based term, curve evolution used for post-processing)Paragios & Deriche, 2002: Textured Image Segmentation Using Level Set Methods (Supervised)Sagiv et al. , 2002. Vese et al, 2002 : Unsupervised textured Image Segmentation using active contours (Mostly heuristic methods fordimensionality reduction)Rousson, Brox & Deriche, 2003: Unsupervised Image Segmentation using structure tensor features. No scale information, anisotropic diffusion used for feature pre-processing.

Kokkinos, Evangelopoulos, Maragos, ICIP 2004: Modulation featureextraction and texture vs. non-texture decision as segmentation cues.

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: 2D Energy Operator

Continuous

Discrete

2 2( )( , ) ( , ) ( , ) ( , )c f x y f x y f x y f x yF — - —22 2 2

2 2

f f f ff fx x y y

Ê ˆ Ê ˆÊ ˆ∂ ∂ ∂ ∂Ê ˆ= - + -Á ˜ Á ˜Á ˜ Á ˜Ë ¯∂ ∂ Ë ∂ ¯ ∂Ë ¯ Ë ¯

( )( , )d f i jF =22 ( , ) ( 1, ) ( 1, ) ( , 1) ( , 1)f i j f i j f i j f i j f i j- - + - - +

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: 2D AM-FM Energy Tracking

Spatial AM-FM signal:

Energy tracking

( ) ( ) ( ), , cos ,f x y a x y x yjÈ ˘= Î ˚

( )1 , ,x yxj∂W∂ ( )2 ,x y

yf∂W∂

( )1 2,W = W W

[ ] 22cos( )c a ajF ª W22 2

1[ ]cf ax∂F ª W W∂

: Instantaneous Frequency vector

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: 2D Continuous Energy Separation Algorithm

2D CESA

2D cosine:

CESA exact estimates:

1 1ˆ ( ) ( , )c c

f f x yx∂Ê ˆW = F F ª WÁ ˜Ë ¯∂ 2 2

ˆ ( ) ( , )c cf f x yy

Ê ˆ∂W = F F ª WÁ ˜Ë ∂ ¯

( ) ( )( )ˆ ( , )c

c c

fa a x yf x f y

F= ªF ∂ ∂ +F ∂ ∂

1 2( , ) cos( )c c of x y A x y j= W +W +

1 1( , ) cx yW = W

2 2( , ) cx yW = W( , )a x y = A

Æ

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: 2D Gabor filterbanks and DCA

2 2 2( , ) exp[ ( ) ( ) 2 ] cos[2 ( )]h x y ax by Ux Vys p= - + ◊ +

2D Gabor function

DCA Block diagram, (by Havlicek et. al. 1996)

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: Curve Evolution

is evolving curve (front) for is a simple smooth closed curve

Position vector =Normal vector =Speed: Curvature =

curve evolution PDE (flow)

( )tΓ 0t ≥

( , ) ( , )C p t VN p tt

∂ =∂

(0)Γ

( , )C p t( , )N p t

( , )K p t( )V F K=

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: Level Set Method for Curve Evolution

Embed curve as zero-level curve of function :

Function evolution PDE:

Advantages of level set method:- remains a function even if topology of changes- Compute curvature & normal of from :

- Efficient Numerics: entropy-satisfying finite differences- Extends easily to 3D

( )tΓ ( , , )x y tΦ( ) ( , ) : ( , , ) 0t x y x y tΓ = Φ =

0( , ) ( , , 0) signed distance tranform from (0)x y x yΦ = Φ = Γ

Vt

∂Φ = ∇Φ∂

Φ ( )tΓ( )tΓ Φ

/N = −∇Φ ∇Φ( )div / Nκ = − ∇Φ ∇Φ = ∇ ⋅

Ref: Osher & Sethian 1988

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: Region Competition

1R 2RiR

QR( )( )

1

[ , ] log 2

N

i i ii

i

vE C p p I dx CR=

= - +ÂÚÚ

Forces acting on the contours, (by Zhu and Yuille, 1996)

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Minimize Image functional (generalized ‘energy’)

Euler PDE:

Gradient Descent:

Solution reached at steady state:

Example 1:

Example 2:

,[ ] ( , , , )x yD

E u F x y u u u dxdy= ∫∫

Euler derivative [ ]

0

u

u u ux y

F

u F F Ft x y

∂ ∂ ∂= − − =∂ ∂ ∂

[ ]uu Ft∂ = −∂

/ 0u t∂ ∂ =2 2

tF u u u= ∇ ⇒ =∇( ) ( ) /tF u u curv u u u= ∇ ⇒ = ∇ ∇

Appendix: Functional Minimization

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

Appendix: Energy Tracking in AM-FM Signals

Cont.-Time AM-FM

Cont.-Time TK Energy Operator:

Energy Tracking

Discrete-Time AM-FM

Discrete-Time TK Energy Operator:

Energy Tracking

[ ] )()()()( 2 txtxtxtxc −=Ψ [ ] 2[ ] [ ] [ 1] [ 1]d x n x n x n x nΨ = − + −

( )0( ) ( )cos ( )

tx t a t dω τ τ= ∫ ( )0

[ ] [ ]cos ( )n

x n a n m dm= Ω∫

[ ] 2 2( ) ( ) ( )c x t t tα ωΨ ≅ [ ] ( )2 2[ ] [ ]sin [ ]x n n nδ αΨ ≅ Ω

CVSP Group, National Technical University of Athens

IEEE International Conference on Image Processing, ICIP 2004, Singapore, 24 – 27 Oct.

[ ][ ]

2 [ ][ ]

[ 1] [ 1]

x na n

x n x n

Ψ≅

Ψ + − −[ ]

[ ][ 1] [ 1]1 arcsin [ ]

2 4 [ ]x n x n

f nx nπ

Ψ + − −≅

Ψ

Appendix: 1D Continuous & Discrete-Time ESA

[ ( )] ( )[ ( )]x t a tx t

Ψ≅

Ψ1 [ ( )] ( )

2 [ ( )]x t f tx tπ

Ψ≅

Ψ

Assumption: x(t) is a narrowband AM-FM Signal