poster 1-13-paper id 207

Mul$criteria Energy Minimiza$on with Boundedness, Edge density and Rarity, for Object Saliency in Natural Images Abstract Sudeshna Roy and Sukhendu Das Visualiza$on and Percep$on Lab. Dept. of Computer Science and Engineering, IIT Madras, India Algorithm Overview Results and Evalua$ons on PASCAL VOC 2012 Dataset Image Edge Map Superpixels and Edges Proposed Output Edge Density (ed i ) Saliency (s i ) Boundedness (b i ) Conclusion • Mo#vated by saliency and category independent object segmenta#on methods, we predict a single object proposal which captures all the salient objects in an image, within O(number of superpixels) 3 . • The CRF parameters are learned using a marginrescaled structSVM approach with (1 IoU) loss func#on. • Fast but effec#ve objectness criteria, viz. boundedness and edgedensity. Submodular CRF is minimized using graphcut. Results are shown on the challenging PASCAL VOC 2012 dataset. • An efficient method based on saliency in conjunc#on with objectness cues to predict a categoryindependent object segmenta#on, using a CRF based formula#on. • Can be a preprocessing step for many highlevel vision task. References Method CPMC [1] OP [2] Proposed IoU Score 0.3319 0.3266 0.4097 [1] [5] [4] [3] [2] 1. Carreira and Sminchisescu, “Constrained parametric mincuts for automa#c object segmenta#on,” CVPR, 2010. 2. Endres and Hoiem, “Category independent object proposals,” ECCV, 2010. 3. Roy and Das, “Saliency detec#on in images using graphbased rarity, spa#al compactness and background prior,” VISAPP, 2014. 4. Yang, et. al., “Saliency detec#on via graphbased manifold ranking,” CVPR, 2013. 5. Perazzi et. al., Saliency filters: Contrast based filtering for salient region detec#on. CVPR. 2012. Input Objectness Image Plane i j Unary Poten$al (Saliency, Objectness) Pairwise Poten$al (based on color) Submodular CRF Inference: Graphcut * * * Considering top 10 proposal masks φ i,j (2) (y i, y j, x i, x j |w 2 ) φ i (1) (y i ,x i |w 1 ) Salient Object Likelihood: φ i (1) (y i ,x i (1) |w 1 )=w s ( (1 y i ) (1s i )+y i s i )+w b ( (1 y i ) (1b i )+y i b i ) +w ed ( (1 y i ) ed i +y i (1ed i )) φ i,j (2) (y i, y j, x i (2) , x j (2) ) = |y i y j | exp( k c ||c i –c j ||) Edge Cost: c i denotes color of ith superpixel in lab space and k c indicates the sensi#vity of color similarity. CRF parameter Learning: • Convex in w, given L • Itera#vely op#mized, by finding y n using graphcut, and thus L • M is the number of training instances. • w min 1 2 w 2 + 1 M ξ n n = 1 M ∑ subject to, w T v=1 w T φ n − w T ˆ φ n ≤ L ( y n , ˆ y n ) + ξ n ξ n ≥ 0, w 2 > 0 L = (1 IoU) E( y , x, w) = E ( y, x, w ) = φ i (1) i ∈V ∑ + λ φ i , j (2) (i , j )∈ E ∑ Energy Func#on over superpixels: w = [ w 1 w 2 ] = [ w s w b w ed λ ] T

Upload: sudeshna-roy

Post on 28-Jan-2018

133 views

Category:

Documents

1 download

Report

Download

Embed Size (px):

TRANSCRIPT

Mul$-‐criteria Energy Minimiza$on with Boundedness, Edge-‐density and Rarity, for Object Saliency in Natural Images

Abstract

Sudeshna Roy and Sukhendu Das Visualiza$on and Percep$on Lab.

Dept. of Computer Science and Engineering, IIT Madras, India

Algorithm Overview

Results and Evalua$ons on PASCAL VOC 2012 Dataset

Image

Edge Map

Superpixels and Edges

Proposed Output Edge Density (edi)

Saliency (si)

Boundedness (bi)

Conclusion

•  Mo#vated by saliency and category independent object segmenta#on methods, we predict a single object proposal which captures all the salient objects in an image, within O(number of superpixels)3.

•  The CRF parameters are learned using a margin-‐rescaled struct-‐SVM approach with (1-‐ IoU) loss func#on. •  Fast but effec#ve objectness criteria, viz. boundedness and edge-‐density. Submodular CRF is minimized using graph-‐cut.

Results are shown on the challenging PASCAL VOC 2012 dataset.

•  An efficient method based on saliency in conjunc#on with objectness cues to predict a category-‐independent object segmenta#on, using a CRF based formula#on.

•  Can be a preprocessing step for many high-‐level vision task.

References

Method CPMC [1] OP [2] Proposed IoU Score 0.3319 0.3266 0.4097

[1] [5] [4] [3] [2]

1.  Carreira and Sminchisescu, “Constrained parametric min-‐cuts for automa#c object segmenta#on,” CVPR, 2010. 2.  Endres and Hoiem, “Category independent object proposals,” ECCV, 2010. 3.  Roy and Das, “Saliency detec#on in images using graph-‐based rarity, spa#al compactness and background

prior,” VISAPP, 2014. 4.  Yang, et. al., “Saliency detec#on via graph-‐based manifold ranking,” CVPR, 2013. 5.  Perazzi et. al., Saliency filters: Contrast based filtering for salient region detec#on. CVPR. 2012.

Input

Objectness

Image Plane

Unary Poten$al (Saliency, Objectness)

Pairwise Poten$al (based on color)

Submodular CRF Inference: Graph-‐cut

* *

* Considering top 10 proposal masks

φi,j(2) (yi, yj, xi, xj|w2)

φi(1)(yi,xi|w1)

Salient Object Likelihood: φi(1)(yi, xi(1)|w1) = ws ( (1-‐ yi) (1-‐si) + yisi ) + wb ( (1-‐ yi) (1-‐bi) + yibi )

+ wed ( (1-‐ yi) edi + yi (1-‐edi) )

φi,j(2) (yi, yj, xi(2), xj(2)) = |yi -‐ yj| exp(-‐ kc||ci – cj||) Edge Cost:

ci denotes color of ith superpixel in lab space and kc indicates the sensi#vity of color similarity.

CRF parameter Learning: •  Convex in w, given L •  Itera#vely op#mized, by finding yn using graph-‐cut, and thus L • M is the number of training instances. • 

wmin

12w 2 +

ξnn=1

M∑

subject to, wTv =1

wTφn −wTφn ≤ L (yn,yn )+ξn

ξn ≥ 0,w2 > 0L = (1-‐ IoU)

E(y, x,w) =

E(y, x,w) = φi(1)

i∈V∑ +λ φi, j

(2)

(i, j)∈E∑Energy Func#on over superpixels: