deformation invariant image matching
DESCRIPTION
Deformation Invariant Image Matching. Haibin Ling and David W. Jacobs Center for Automation Research Computer Science Department University of Maryland, College Park Oct, 20, 2005, ICCV. Outline. Introduction Deformation Invariant Framework Experiments Conclusion and Future Work. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/1.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Deformation Invariant Image Matching
Haibin Ling and David W. Jacobs
Center for Automation ResearchComputer Science Department
University of Maryland, College Park
Oct, 20, 2005, ICCV
![Page 2: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/2.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Outline
Introduction
Deformation Invariant Framework
Experiments
Conclusion and Future Work
![Page 3: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/3.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
General Deformation
• One-to-one, continuous mapping.• Intensity values are deformation invariant.
– (their positions may change)
![Page 4: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/4.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Our Solution
• A deformation invariant framework
– Embed images as surfaces in 3D
– Geodesic distance is made deformation invariant by adjusting an embedding parameter
– Build deformation invariant descriptors using geodesic distances
![Page 5: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/5.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Related Work• Embedding and geodesics
– Beltrami framework [Sochen&etal98]– Bending invariant [Elad&Kimmel03]– Articulation invariant [Ling&Jacobs05]
• Histogram-based descriptors– Shape context [Belongie&etal02]– SIFT [Lowe04]– Spin Image [Lazebnik&etal05, Johnson&Hebert99]
• Invariant descriptors– Scale invariant descriptors [Lindeberg98, Lowe04]– Affine invariant [Mikolajczyk&Schmid04, Kadir04,
Petrou&Kadyrov04]– MSER [Matas&etal02]
![Page 6: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/6.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Outline
Introduction
Deformation Invariant Framework Intuition through 1D images2D images
Experiments
Conclusion and Future Work
![Page 7: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/7.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
1D Image Embedding
1D Image I(x)
EMBEDDINGI(x) ( (1-α)x, αI )αI(1-α)x
Aspect weight α : measures the importance of the intensity
![Page 8: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/8.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Geodesic Distance
αI
(1-α)x
p qg(p,q)
• Length of the shortest path along surface
![Page 9: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/9.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Geodesic Distance and α
I1 I2
Geodesic distance becomes deformation invariant
for α close to 1
embed embed
![Page 10: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/10.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Image Embedding & Curve Lengths
]1,0[:),( 2 RyxI
dtIyxl ttt 222222 )1()1(
))('),('),('()( tztytxt
Depends only on intensity I Deformation Invariant
IzyyxxI ',)1(',)1('),(
dtI t
21
Image I
Embedded Surface
Curve on
Length of
Take limit
![Page 11: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/11.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Geodesic Distance for 2D Images
• Computation– Geodesic level curves – Fast marching [Sethian96]
is the marching speed 2/122222)1(
yx IIF
• Geodesic distance– Shortest path– Deformation invariant
F
T is the geodesic distance
T=1T=2
T=3
T=4
p
q1|| FT
![Page 12: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/12.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Deformation Invariant Sampling
Geodesic Sampling1. Fast marching: get
geodesic level curves with sampling interval Δ
2. Sampling along level curves with Δ
p
sparsedense
Δ
ΔΔ
Δ
Δ
![Page 13: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/13.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Deformation Invariant Descriptor
p qp q
Geodesic-Intensity Histogram (GIH)
geodesic distance
inte
nsity
geodesic distance
inte
nsity
![Page 14: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/14.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Real Example
pq
![Page 15: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/15.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Deformation Invariant Framework
Image Embedding ( close to 1)
Deformation Invariant SamplingGeodesic Sampling
Build Deformation Invariant Descriptors(GIH)
),(),( IyxI
![Page 16: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/16.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Practical Issues
• Lighting change– Affine lighting model– Normalize the intensity
• Interest-Point– No special interest-point is required– Extreme point (LoG, MSER etc.) is more
reliable and effective
![Page 17: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/17.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Invariant vs. Descriminative
0
10
1
![Page 18: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/18.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Outline
Introduction
Deformation Invariance for Images
ExperimentsInterest-point matching
Conclusion and Future Work
![Page 19: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/19.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Data Sets
Synthetic Deformation & Lighting Change (8 pairs) Real Deformation (3 pairs)
![Page 20: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/20.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Interest-Points
* Courtesy of Mikolajczyk, http://www.robots.ox.ac.uk/~vgg/research/affine/
Interest-point Matching
• Harris-affine points [Mikolajczyk&Schmid04] *
• Affine invariant support regions• Not required by GIH• 200 points per image
• Ground-truth labeling• Automatically for synthetic image pairs• Manually for real image pairs
![Page 21: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/21.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Descriptors and Performance Evaluation
Descriptors• We compared GIH with following descriptors:
Steerable filter [Freeman&Adelson91], SIFT [Lowe04], moments [VanGool&etal96], complex filter [Schaffalitzky&Zisserman02], spin image [Lazebnik&etal05] *
•
Performance Evaluation• ROC curve: detection rate among top N matches. • Detection rate
matches possible#
matchescorrect #r
* Courtesy of Mikolajczyk, http://www.robots.ox.ac.uk/~vgg/research/affine/
98.0
![Page 22: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/22.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Synthetic Image Pairs
![Page 23: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/23.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Real Image Pairs
![Page 24: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/24.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Outline
Introduction
Deformation Invariance for Images
Experiments
Conclusion and Future Work
![Page 25: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/25.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Conclusion and Future Work
Conclusion A new deformation invariant framework Deformation invariant descriptor (GIH)
Future Work Understanding how to effectively vary α Noise & Occlusion Fast algorithm Real application …
![Page 26: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/26.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Acknowledgement
• Krystian Mikolajczyk and Cordelia Schmid for the feature extraction code.
• Paolo Favaro and Kevin S. Zhou for discussion.
• NSF (ITR- 03258670325867).
• The Horvitz Assistantship.
![Page 27: Deformation Invariant Image Matching](https://reader033.vdocument.in/reader033/viewer/2022051402/56815966550346895dc6a32e/html5/thumbnails/27.jpg)
Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005
Thank You!