object segmentation - eecs at uc berkeley
TRANSCRIPT
Object Segmentation
Stella X. Yu
Department of Computer Science
University of California at Berkeley
Yu: Bay Area Vision Meeting 2003 – p.1/20
Object Segmentation
Yu: Bay Area Vision Meeting 2003 – p.2/20
Direct Feature Classification for Object Detection
image
recognition
Yu: Bay Area Vision Meeting 2003 – p.3/20
Why Perceptual Organization
15
9
Mahamud multi-object detector
Yu: Bay Area Vision Meeting 2003 – p.4/20
Traditional Use of Perceptual Organization
image
segmentation
figure-ground
recognitionperceptual organization
sequential processing (Marr, Lowe, Witkin, Tenenbaum, ...)
Yu: Bay Area Vision Meeting 2003 – p.5/20
Our Approach to Object Segmentation
Yu: Bay Area Vision Meeting 2003 – p.6/20
Our Approach to Object Segmentation
Yu: Bay Area Vision Meeting 2003 – p.6/20
Our Approach to Object Segmentation
Yu: Bay Area Vision Meeting 2003 – p.6/20
Our Approach to Object Segmentation
Yu: Bay Area Vision Meeting 2003 – p.6/20
Our Approach to Object Segmentation
Yu: Bay Area Vision Meeting 2003 – p.6/20
Our Approach to Object Segmentation
Yu: Bay Area Vision Meeting 2003 – p.6/20
Grouping in a Graph-Theoretic Framework
�
�
�
��
�
�
�
�
�
��
Yu: Bay Area Vision Meeting 2003 – p.7/20
Grouping in a Graph-Theoretic Framework
�
�
�
��
�
�
�
�
�
��
Representation: G = {V, E, W} = { nodes, edges, weights }
Yu: Bay Area Vision Meeting 2003 – p.7/20
Grouping in a Graph-Theoretic Framework
�
�
�
��
�
�
�
�
�
��
Representation: G = {V, E, W} = { nodes, edges, weights }
Clustering: ΓKV = {V1, . . . , VK} = K-way node partitioning
(Shi & Malik, Zabih, Boykov, Veksler, Kolmogorov,...)
Yu: Bay Area Vision Meeting 2003 – p.7/20
Links in Graph Cuts
�
�
�
��
�
�
�
�
�
��
Yu: Bay Area Vision Meeting 2003 – p.8/20
Links in Graph Cuts
�
�
�
��
�
�
�
�
�
��
�
�
�
��
�
�
�
�
�
��
P Q
Yu: Bay Area Vision Meeting 2003 – p.8/20
Links in Graph Cuts
�
�
�
��
�
�
�
�
�
��
P Q
links(P, Q) =∑
p∈P, q∈Q
W (p, q)
Yu: Bay Area Vision Meeting 2003 – p.8/20
Degree in Graph Cuts
�
�
�
��
�
�
�
�
�
��
P Q
degree(P) = links(P, V)
Yu: Bay Area Vision Meeting 2003 – p.9/20
Linkratio in Graph Cuts
�
�
�
��
�
�
�
�
�
��
P Q
linkratio(P, Q) =links(P, Q)
degree(P)
Yu: Bay Area Vision Meeting 2003 – p.10/20
Goodness of Grouping in Graph Cuts
�
�
�
��
�
�
�
�
�
��
P
V \ P
Maximize within-group connections: linkratio(P, P)
Minimize between-group connections: linkratio(P, V \ P)
Equivalent: linkratio(P, P) + linkratio(P, V \ P) = 1
Yu: Bay Area Vision Meeting 2003 – p.11/20
K-Way Normalized Cuts
�
�
�
��
�
�
�
�
�
��
V1 V2
V3
max knassoc(ΓKV ) =
1
K
K∑
l=1
linkratio(Vl, Vl)
min kncuts(ΓKV ) =
1
K
K∑
l=1
linkratio(Vl, V \ Vl)
Yu: Bay Area Vision Meeting 2003 – p.12/20
Joint Pixel-Patch Grouping: Criterion
A
B
C
ε̄(ΓKV , ΓK
U ; A, B) =1
K
K∑
l=1
linkratio(Ul, Ul; B) · degree(Ul; B)
degree(Vl; A) + degree(Ul; B)
+1
K
K∑
l=1
linkratio(Vl, Vl; A) · degree(Vl; A)
degree(Vl; A) + degree(Ul; B)
Yu: Bay Area Vision Meeting 2003 – p.13/20
Joint Pixel-Patch Grouping: Consistency
A
B
C
ΓKU = {U1, . . . , UK}, ΓK
V = {V1, . . . , VK}
Bias linking patches with their pixels
Yu: Bay Area Vision Meeting 2003 – p.14/20
Computing Constrained Normalized Cuts
• Constrained eigenvalue problem
• Efficient solution using a projector onto the feasible space
• Generalize Rayleigh-Ritz theorem to projected matrices
Yu: Bay Area Vision Meeting 2003 – p.15/20
How Object Knowledge Helps Segmentation
pixel only pixel w/ ROI pixel-patch
541s 150s 110s
Yu: Bay Area Vision Meeting 2003 – p.16/20
How Segmentation Helps Object Detection
image patch density segmentation
Yu: Bay Area Vision Meeting 2003 – p.17/20
When Does Our Method Fail
image patch density segmentation
Yu: Bay Area Vision Meeting 2003 – p.18/20
Equally Applicable to Multiple Objects
Yu: Bay Area Vision Meeting 2003 – p.19/20
Acknowledgements
• Shyjan MahamudJianbo Shi
• CMU HumanID LabCMU Hebert Lab
• ONR N00014-00-1-0915NSF IRI-9817496
Yu: Bay Area Vision Meeting 2003 – p.20/20