scene labeling using beam search under mutex constraints anirban roy and sinisa todorovic

1
IEEE 2014 Conference on Computer Vision and Pattern Recognition Beam Search for Solving QP Beam Search for Solving QP Results Results Acknowledgment Acknowledgment Problem and Motivation Problem and Motivation Approach Approach 1. 1. Extracting superpixels Extracting superpixels 2. 2. Incorporating mutex in the standard CRF Incorporating mutex in the standard CRF formulation formulation 3. 3. Formulating CRF inference as QP Formulating CRF inference as QP 4. 4. Beam search for solving QP Beam search for solving QP 5. 5. Learning – piecewise Learning – piecewise How to Specify CRF Energy? How to Specify CRF Energy? CRF Inference as QP CRF Inference as QP Specifying Mutex Constraints Specifying Mutex Constraints Scene Labeling Using Beam Search Under Mutex Constraints Anirban Roy and Sinisa Todorovic Input Image Semantic segmentation with Mutex Semantic segmentation without Mutex Mutex violati ons Appearance features of the superpixels Smoothness and Contex t Pixelwise accuracy(%) can be arbitrary Assignment vector Matrix of CRF potentials NSF RI 1302700 MUTual EXclusion = (object, object, relationship) State: label assignment Heuristic function: Score: Superpixel Class label Method MSRC Test time Galleguillos et al. CVPR 10 70.4 N/A Gould et al. ICCV 09 76.4 N/A Payet et al. PAMI 12 82.9 30-32s Krahenbuhl et al. NIPS 12 86.0 0.2s Yao et al. CVPR 12 86.5 N/A Zhang et al. CVPR 12 87.0 N/A Ours 91.5 0.8s Method Stanford Backgrou nd Gould et al. ICCV 09 76.4 Munoz et al. ECCV 10 76.9 Singh et al. CVPR 13 74.1 Ours 81.1 Matrix of mutex constraints maximum score next state previous state must be

Upload: mignon

Post on 21-Mar-2016

28 views

Category:

Documents


2 download

DESCRIPTION

Scene Labeling Using Beam Search Under Mutex Constraints Anirban Roy and Sinisa Todorovic. Beam Search for Solving QP Results Acknowledgment. Problem and Motivation Approach Extracting superpixels Incorporating mutex in the standard CRF formulation Formulating CRF inference as QP - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Scene Labeling Using Beam Search Under Mutex Constraints  Anirban Roy and Sinisa Todorovic

IEEE 2014 Conference on Computer Vision and Pattern

Recognition

Beam Search for Solving QPBeam Search for Solving QP

Results Results

Acknowledgment Acknowledgment

Problem and MotivationProblem and Motivation

ApproachApproach1.1. Extracting superpixels Extracting superpixels 2.2. Incorporating mutex in the standard CRF formulationIncorporating mutex in the standard CRF formulation3.3. Formulating CRF inference as QPFormulating CRF inference as QP4.4. Beam search for solving QPBeam search for solving QP5.5. Learning – piecewiseLearning – piecewise

How to Specify CRF Energy?How to Specify CRF Energy?

CRF Inference as QPCRF Inference as QP

Specifying Mutex ConstraintsSpecifying Mutex Constraints

Scene Labeling Using Beam Search Under Mutex Constraints Anirban Roy and Sinisa Todorovic

Input Image Semantic segmentation with Mutex

Semantic segmentation without Mutex

Mutex violations

Appearance features of the superpixels

Smoothnessand Context

Pixelwise accuracy(%)

can be arbitrary

Assignment vector

Matrix of CRF potentials

NSF RI 1302700

MUTual EXclusion = (object, object, relationship)

State: label assignmentHeuristic function:

Score:Superpixel Class label

Method MSRC Test timeGalleguillos et al. CVPR 10 70.4 N/A

Gould et al. ICCV 09 76.4 N/APayet et al. PAMI 12 82.9 30-32s

Krahenbuhl et al. NIPS 12 86.0 0.2sYao et al. CVPR 12 86.5 N/A

Zhang et al. CVPR 12 87.0 N/AOurs 91.5 0.8s

Method StanfordBackground

Gould et al. ICCV 09 76.4Munoz et al. ECCV 10 76.9Singh et al. CVPR 13 74.1

Ours 81.1

Matrix of mutex constraints

maximum score

next state previous state

must be