3d shape matching and retrieval - university of...
TRANSCRIPT
![Page 1: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/1.jpg)
3D Shape Matching and Retrieval
Computer Animation and Visualisation
Lecture 17
Taku Komura
![Page 2: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/2.jpg)
Overview
● Introduction– Motivation, applications
– Challenges, key issues
● Global approaches– Image-based Methods
– Graph-based Methods
● Local approaches– Curvature
● Bag of visual words– Heat Kernel Signature
![Page 3: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/3.jpg)
Many 3D objects in the world today to shape match
● Medical, CAD/CAM design, 3D scans, animation,
![Page 4: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/4.jpg)
CAD/CAM
Manufacturing and production (You and Tsai, 2010)
![Page 5: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/5.jpg)
Synthesizing new objects using existing objects [Kalogerakis et al.]
![Page 6: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/6.jpg)
Recognizing and registering scanned data
● Recognizing objects
● Searching in the database for similar data and registering
![Page 7: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/7.jpg)
Biometrics
![Page 8: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/8.jpg)
3D video sequences
● Characterize a motion (Huang et al., 2010)
![Page 9: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/9.jpg)
3D retrieval for navigation (Goodall et al. 2004)
![Page 10: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/10.jpg)
Archeology
● Matching for reconstruction (Huang et al. 2006)
![Page 11: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/11.jpg)
Challenges● The geometry can be very different
– Rigid objects
– Articulated objects
– Deformable objects
● The topology can be very different ● The data can contain a lot of noise
– Topological / geometric noise
![Page 12: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/12.jpg)
Geometric and Topological Noise
● Noise can be added when scanning or processing the data
geometric noise topological noise
![Page 13: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/13.jpg)
Key issues ● Input format
– Point clouds, meshes, volume data
● What type of features/descriptors are used?● Are the features/descriptors translation/rotation/scale
invariant? ● Abstraction
– Representation
– Whether the method can handle objects with different geometry and topology
![Page 14: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/14.jpg)
Matching problems
● Non-rigid matching
● Partial matching
![Page 15: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/15.jpg)
● Transform a 3D object into a numeric/symbolic
representation– Feature vectors
– Graphs
● Compare two objects through their representationsThe global approachThe global approachThe global approachThe global approach
The Global Approach
![Page 16: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/16.jpg)
Depth-buffer descriptor
● Image-based descriptor (Vranic 2004)– Pose normalization
– Depth-buffer construction
– Fourier transformations
– Selection of coefficients
![Page 17: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/17.jpg)
Depth-buffer descriptor
● Pose normalization - Typical procedure– Translate the center of mass to the origin of the
coordinate system
● Rotate according to the largest spread● Scale to common size
![Page 18: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/18.jpg)
Depth-buffer descriptor
● Pose normalization– Done using principal component
analysis
![Page 19: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/19.jpg)
Depth-buffer descriptor
● Construction– Project the object into the faces of a bounding
rectangle
![Page 20: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/20.jpg)
Fourier Transform
Transform into the Frequency Domain
![Page 21: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/21.jpg)
Spherical Harmonics
● The Fourier transform for a spherical function
![Page 22: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/22.jpg)
Other approaches
● Ray-based feature vector (Vranic 2004)
![Page 23: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/23.jpg)
Issues: Requires Alignment
● Difficult to align objects – whose points are equally distributed
– symmetric around an axis
● A rotation invariant feature is preferred
![Page 24: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/24.jpg)
Rotation Invariant Spherical Harmonic Representation
Kazhdan et al SGP03
● Given a spherical depth map, convert to spherical harmonics
● Generate a vector of the power of each frequency
![Page 25: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/25.jpg)
Issues with image-based methods
● Not very suitable for articulated / deformable objects well – The depth information is not consistent when the
pose changes
![Page 26: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/26.jpg)
Graph-based Approaches
● Computing a graph structure out of the geometry and compare objects by graph matching
● Two types of graphs introduced here – Medial axis
– Reeb graph
![Page 27: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/27.jpg)
Medial Axis
● A set of points that have two or more closest points on the surface
● The trajectory of the maximal circles/spheres
![Page 28: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/28.jpg)
Computing the Medial Axis[Amenta et al.], Powercrust
● Approximate Method: – Using the Voronoi diagram
– Sample points on the surface and then compute the Voronoi diagram
– A Voronoi vertex is a point with three (2D)/four(3D) or more closest sample points.
– The corresponding Voronoi ball is the unique ball centred at the vertex that contains the closest sample points in its boundary.
– For every sample point there is an inner pole, namely the farthest Voronoi vertex in the intersection of the shape and the Voronoi cell of the sample point.
– Connect the inner poles according to the adjacency of the samples to approximate the medial axis
![Page 29: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/29.jpg)
Medial Axis: Noise filtering
● The medial axis is sensitive geometric noise (bumps at the surface)
● We need filter the noise to produce a medial axis that represents the shape well
![Page 30: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/30.jpg)
Scale Axis Transform
● Removing the inner poles whose maximal spheres are absorbed by nearby larger maximal spheres when their sizes are scaled
https://www.youtube.com/watch?v=-VsFTXfFXHQ#t=82
![Page 31: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/31.jpg)
Reeb Graph
● Named after Georges Reeb (A French mathematician)
● A graph structure that represents the object geometry
● Used a lot in computer graphics and pattern recognition
![Page 32: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/32.jpg)
Reeb Graph
● Given a function that we call Morse function, that defines a scalar value on the surface of a manifold, the Reeb graph is a graph whose nodes represent the isolines that have the same values, and the edges represent the connection of the isolines between different values
![Page 33: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/33.jpg)
Geodesic Distance
● The shortest distance over the surface
● An approximation can be computed using graph distance
● Geodesic distance can be used as the Morse function
![Page 34: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/34.jpg)
Geodesic Integral[Hilaga et al. 2001]
● We can use the integral of the geodesic distance to resolve the ambiguity of setting the start point
● µ(v) = ∫p g(v,p) dS● Normalize
– µn(v) = (µ(v) – min(µ)) / min(µ)
![Page 35: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/35.jpg)
Multiresolution Reeb Graph
● Graph matching can be costly so can use the multiresolution Reeb graph
● Binary discretization preserve parent-child relationships
● Exploit them for matching
[Hilaga et al. 2001]
![Page 36: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/36.jpg)
Matching process
● Calculate similarity● Match nodes
– Find pairs with maximal similarity
– Preserve multires hierarchy topology
● Sum up similarity
[Hilaga et al. 2001]
![Page 37: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/37.jpg)
Content-based Retrieval[Hilaga et al. 2001]
https://www.youtube.com/watch?v=xA0S_oEWD1Y
![Page 38: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/38.jpg)
Summary for Graph-based Approaches
● Good for computing the similarity based on the articulated structures
● Rotation / scale invariant (if normalized) ● Medial axis is sensitive to geometric noise but
can do robust matching after filtering● Not robust to topological noise
geometric noise topological noise
![Page 39: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/39.jpg)
Overview
● Introduction– Motivation, applications
– Challenges, key issues
● Global approaches– Image-based Methods
– Graph-based Methods
● Local approaches– Curvature
● Bag of visual words– Heat Kernel Signature
![Page 40: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/40.jpg)
Local Features and Partial Matching
● Features based on local geometry ● Good for finding saliency ● Useful for partial shape matching● Use curvature
![Page 41: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/41.jpg)
Curvature
● Amount by which a geometric object deviates from being flat, or straight
● The curvature is the magnitude of the rate of change of the tangent vector T:
● Or can be computed by
![Page 42: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/42.jpg)
Segmenting and Matching based on Curvature
● Curvature has been used for segmenting 2D contours and matching
● Need multiscale analysis for robustness
(conduct smoothing, and compute the curvature at different levels)
![Page 43: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/43.jpg)
Curvature on Surfaces : Principal Curvature (1)
● At a point p on the surface, we can define a normal plane that contains the normal vector
● It also contains a unique direction tangent to the surface and cut the surface in a plane curve
![Page 44: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/44.jpg)
Curvature on Surfaces: Principal Curvature (2)
● The plane curve have different curvature for different normal planes
● The principal curvatures at p, denoted and , are the maximum and minimum values of this curvature.
● Special Curvature – Mean curvature :
– Gaussian curvature
![Page 45: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/45.jpg)
Discrete Laplace-Beltrami [Meyer et al.]
![Page 46: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/46.jpg)
Discrete Curvatures
![Page 47: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/47.jpg)
Voronoi Region Area
● Sum the A below for all the triangles of 1-ring(v i)
If the triangle T is non-obtuse● A + ●
● If obtuse and vi is the obtuse angle,
A += area(T) / 2
– Else A += area (T) / 4
![Page 48: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/48.jpg)
Extracting Saliency using Curvature
● The salient points can be extracted using the curvature
● Can use the curvature as an attribute at the surface for matching
![Page 49: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/49.jpg)
Overview
● Introduction– Motivation, applications
– Challenges, key issues
● Global approaches– Image-based Methods
– Graph-based Methods
● Local approaches– Curvature
● Bag of visual words– Heat Kernel Signature
![Page 50: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/50.jpg)
Heat Kernel Signature● A vector that represents the temperature at a
heated point as time passes
[t1, t2, t3, ...., tn]● The temperature after a short time describes
the local geometry● The temperature after a long time describes the
global geometry
![Page 51: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/51.jpg)
Local Geometric Feature by HKS
● The heat in short time has a similar distribution to the curvature
![Page 52: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/52.jpg)
Comparison of the shapes
● Using the heat kernel signature vector, conduct a K-means clustering and produce a geometric vocabulary P = {p1 , . . . , pV } of size V.
● For each point x, we can compute the feature distribution of the vocabulary
θ (x) = (θ1(x), . . . , θV(x)) where
![Page 53: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/53.jpg)
Bag of Features
● Integrating the feature distribution over the entire shape X yields a V × 1 vector
● For comparing two shapes, we can compute the distance between bag of features
![Page 54: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/54.jpg)
Example of Bag of Features
![Page 55: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/55.jpg)
Summary: Heat Kernel Signature
● Can obtain both the local and global features● Robust against topological noise
– The heat does not diffuse much when there is a connection with less volume
● The bag of features provide a quick scheme to compare the objects
![Page 56: 3D Shape Matching and Retrieval - University of Edinburghhomepages.inf.ed.ac.uk/tkomura/cav/presentation17_2015.pdf · 3D Shape Matching and Retrieval Computer Animation and Visualisation](https://reader033.vdocument.in/reader033/viewer/2022041420/5e1e9622b77c053c3044b90c/html5/thumbnails/56.jpg)
References● Kazhdan et al. “Rotation Invariant Spherical Harmonic Representation of 3D Shape
Descriptors”, SGP03
● Ivan Sipiran and Benjamin Bustos, “Shape Matching for 3D Retrieval and Recognition”, SIBGRAPI 2013 Tutorial , Arequipa - Perú, August 5, 2013
● ALEXANDER M. BRONSTEIN et al., Shape Google: Geometric Words and Expressions for Invariant Shape Retrieval, Transactions on Graphics 2011, 30(1)
● Mark Meyer et al. Discrete Differential-Geometry Operators for Triangulated 2-Manifolds
● Amenta et al. “The Power Curst”
● Joachim Giesen, Balint Miklos, Mark Pauly, Camille Wormser: The Scale Axis Transform, ACM Symposium on Computational Geometry 2009
●