1

Region-Based Saliency Detection and Its Application in Object Recognition

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24

NO. 5, MAY 2014

Zhixiang Ren, Shenghua Gao, liang-Tien Chia, and Ivor Wai-Hung Tsang

2

Overview

• Introduction

• Related Work

• Proposed Method of Saliency

• Experiments For Saliency Detection

• Conclusion

3

Introduction

4

Introduction

• Visual Saliency

Measure to what extent a region attracts human attention.

• Potential Application

Adaptive compression, image retargeting, object detection

5

Introduction (cont.)

• Many saliency detection algorithms (pixel-grid) have been proposed

[36]-[38], [57], [68]

• Drawbacks

• Perform poorly in the images with large salient regions

• Suffer from the messy background, e.g. natural scenes.

6

Introduction (cont.)

• [17], [18] suggest that early feature like color, contrast, and orientation indirectly affect human attention

Human is attracted by objects not by individual pixels

• It is natural to work with those perceptually meaningful image regions in saliency detection

Concept of superpixel [54]

7

Introduction (cont.)

• Proposed work applies two existing techniques to improve saliency detection

Superpixel representation – Used to represent the input image

PageRank algorithm – Applied to propagate saliency among similar clusters and refine saliency map

8

Related Work

9

Related Work

• Saliency detection methods can be divided into two categories

Top-down method : Task-dependent and based on prior knowledge about the object and their interrelations

Bottom-up method : Hypothesis for saliency is that salient stimulus is distinct from its surrounding stimuli. (contrast)

• For bottom-up method, research usually focus on identifying those regions with high contrast.

10

Related Work (cont.)

• [38] proposed to determine the contrast by DoG

• [57] measured the likeness of a pixel to its surroundings by the local regression kernels

• [1] measured the saliency of each pixel by the difference between the feature of each pixel and mean of the whole image.

• [68] measured the global contrast with all the other pixels.

• [30] model both local and global contrast by taking the positional distance into account

• Most of approach represent the input image in pixel-grid manner, and these method may failed to detect the homogeneous and quite large salient objects

11

Related Work (cont.)

12

Proposed Method

13

Proposed Method

• Superpixel Extraction and Clustering

• Salient Region Detection

• Saliency Refinement With Propagation

14

Superpixel Extraction

• Given an input image, mean shift algorithm[13] will be performed in color space to extract superpixels.

• In mean shift algorithm, , , and are needed.

Spatial Radius

Range Radius

Minimum Point Density

Maximum range radius

Average color variance

Color variance

15

Superpixel Extraction (cont.)

• Mean shift

Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function.

Scale parameter

16

Superpixel Extraction (cont.)

m(x)

17

Superpixel Clustering

• After mean shift, every superpixel will obtain a unique RGB color

• GMM is introduced to cluster superpixels in RGB color space

• The RGB value of this superpixel will be set as a 3-D vector to represent the superpixel during GMMR 139

G 160

B 127

18

Superpixel Clustering (cont.)

• K-means is used to initialize the GMM

• Expectation maximization algorithm is used to train GMM parameter [5]

• The probability of the th superpixel belong to the th cluster

19

Superpixel Clustering (cont.)

20

Superpixel Clustering (cont.)

• Mixture density

Assume the set of N training samples is drawn from a mixture of models

21

Superpixel Clustering (cont.)

• How to estimate and

EM algorithm

22

Superpixel Clustering (cont.)

23

Superpixel Clustering (cont.)

24

Superpixel Clustering (cont.)

25

Superpixel Clustering (cont.)

26

Salient Region Detection

• Idea : background has larger spread in spatial domain

• i.e., the more compact the clusters are spread, the more salient they will be

• [32] proposed compactness metric to evaluate the spread of cluster

• Inter-cluster distances defined as

Cluster Spatial Center

27

Refinement With Propagation

• In some situation, the perceptually meaningful regions are less than the cluster number.

• That is, some regions, which should belong to one cluster, will be grouped into several clusters.

R 139

G 160

B 127

28

Refinement With Propagation (cont.)

• If one cluster is over-segmented into several subclusters, the compactness may be highly distorted.

• Thus PageRank algorithm is proposed to propagate saliency between similar clusters.

• Original PageRank algorithm

• Question : How the original PageRank come from?

29

Refinement With Propagation (cont.)

• Idea : A page linked by many pages with high PageRank receives high rank as well.

• Modified algorithm

30

Refinement With Propagation (cont.)

31

Experiment Result

32

Experimental DataSet and Compared Method

• Dataset

EPFL dataset [1], CMU dataset [4], MSRA dataset [46] ,Itti’s method (ITTI) [38]

• Method

Spectral residual method (SR) [37]

Graph-based saliency method (GB) [36]

Frequency-tuned method (FT) [37]

Method based on color and orientation distributions (COD) [32]

Region contrast method (RC) [12] (Region-based)

Context-based method (CB) [39] (Region-based)

33

Experiment Result

34

Experiment Result (cont.)

• Linear Correlation Coefficient

35

Experiment Result (cont.)

36

Conclusion

37

Conclusion

• The paper proposes a promising saliency detection approach, which can generate accurate saliency maps with well-defined object boundary.

• Mean shift, GMM are used to extract meaningful superpixel.

• Saliency value is refined as well with a modified PageRank algorithm.