data-driven shape analysis ---...
TRANSCRIPT
Data-Driven Shape Analysis--- Recap
Qixing HuangStanford University
Shape Descriptors
What Is A Shape Descriptor
Shape Distributions (D2)
Spin images
Lightfield Descriptor
Shape matching
Registration
Align two shapes/scans
given initial guess for
relative transform
ICP [Besl and Mckay’92]
Shape matching
• Rigid matching --- how to generate the initial guess
Applications
Surface reconstruction Fragment assembly
Protein dockingObject completion
Scan
Template
Reconstruction
Approaches --- point-based
Spectral matchingRANSAC Voting
1 3 5 2 4
1 1 1 1 0 0
3 1 1 1 0 0
5 1 1 1 0 0
2 0 0 0 1 1
4 0 0 0 1 1
Partial similarity Stable
Non-rigid registration
Applications
Dynamic geometry reconstruction[Li et al. 13]
Tracking[Li et al. 09]
Interpolation[Kilian et al. 08]
Shape completion[Pauly et al. 05]
Application --- distance learning
Fine-Grained Semi-Supervised Labeling of Large Shape Collections, Q. Huang, H. Su, L. Guibas, SIGGRAPH ASIA’ 13
Input Rigid Non-rigid
• Compute closest point pairs
• Deform the source shape P
Non-Rigid ICP
Q
P = fpig
Q
P = fpig
Distance term Deformation term
Heat kernel map
Conformal mapping
Mobius voting
Blended intrinsic maps
Functional maps
Data-driven matching
Piece assembly
22
Ambiguous matches
23
Additional data helps
Additional data helpsBlended intrinsic maps[Kim et al. 11]
Composite
Intermediateobject
Cycle-consistency
Consistent
• Maps are consistent along cycles
Cycle-consistency
Inconsistent
• Maps are consistent along cycles
Cycle-consistencyBlended intrinsic maps[Kim et al. 11]
Composite
Inconsistent
Cycle-consistency
Direct
Blended intrinsic maps[Kim et al. 11]
Composite
Consistent
Joint matching formulation
• Input:
– Shapes
– Pair-wise maps
(existing algorithms)
Joint matching formulation
• Input:
– Shapes
– Pair-wise maps
(existing algorithms)
• Output:
– Cycle-consistent
– “Close” to the input maps
NP-complete [Huber 2002]
Point-maps
X º 0
(Positive) semidefiniteness
Xij = XTj1Xi1 X=
264
Im...
XTn1
375hIm ¢ ¢ ¢ Xn1
i
Convex program
Xii = Im; 1 · i · nsubject to
minimize
P(i;j)2E
kXinputij ¡Xijk1
Xij1= 1;XTij1= 1; 1 · i < j · n
X º 0
X ¸ 0
ADMM [Boyd et al.11]
Deterministic guarantee
• Exact recovery condition:
#incorrect corres. per point< algebraic-connectivity(G)/4
Constrained optimization framework
minimize
Subject to
Constraints on X
Symmetricmatrices
minimize
Subject to Constraints on X
Asymmetricmatrices
Affordance
Fine-grained analysis
Segmentation
Segmentation methods
• Extraneous geometric clues
Structural similarity of segmentations
Joint shape segmentation
Single shape segmentation[Chen et al. 09]
Joint shape segmentation[Huang et al. 11]
Joint shape segmentation[Huang et al. 11]
Structural similarity of segmentations
• Low saliency
Joint shape segmentation
Single shape segmentation[Chen et al. 09]
• Articulated structures
Joint shape segmentation
Joint shape segmentation[Huang et al. 11]
(Rigid) invariance of segments
Single shape segmentation[Chen et al. 09]
Shape classification
Shape classification tasks
Category level Fine-grained
loungerocking
folding rex
Category level
Dense labels
Relativelyclean labels
Similar shape voting
Chair
Chair
Chair
Stool
Fine-grained --- challenges
Sparse and noisy labels Features
System overview
Graph-Based Classification
with-arms side windsor rex
Data-driven shape modeling
Shape grammar for a building
Shape grammar for a building
Understand variations
Discrete probabilistic part relations
Shape synthesis
Data-Driven Reconstruction
Combine data + priors (from existing shapes)
Data-driven scene analysis
Sketch-based scene synthesis
Future Direction
Big data
Current status
1012
1010
108
106
104
Images 3D ModelsVideos(Per minute)
2007
2014
10x 10x 1000x
Management
Data qualityHuman factor
Visualization
Similarity/Variability
Intra IntraInter
Data management/visualization
Variability
Bas
is
Bigdata-driven modeling
Can we learn shape grammar big shape data?
Big data
High-levelunderstanding
Similar symmetries
Tevs, Huang, Wand, Seidel, Guibas.Relating Shapes via Geometric Symmetries and Regularities, SIGGRAPH’14
Similar styles
Chinese furniture
Similar styles
Gothic buildings
Human object interaction
The data-driven perspective
Big data
High-levelunderstanding
Cross-domain
Image world Shape world
Very big: Trillions Big: Tens of millions
Rich labels Sparse labels
2D 3D
Documents
Images/Shapes
Videos/Trajectories
Big data
High-levelunderstanding
Cross-domain
Big data
High-levelunderstanding
Cross-domain