coherency sensitive hashing (csh) simon korman and shai avidan dept. of electrical engineering tel...
Post on 19-Dec-2015
217 views
TRANSCRIPT
![Page 1: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/1.jpg)
Coherency Sensitive Hashing (CSH)Simon Korman and Shai AvidanDept. of Electrical Engineering Tel Aviv UniversityICCV2011 | 13th International Conference on Computer Vision
![Page 2: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/2.jpg)
Outline• Introduction• Locality Sensitive Hashing for Finding Nearest Neighbors• Coherency Sensitive Hashing • Experiments• Conclusions
![Page 3: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/3.jpg)
Introduction(1/2)Patch : k*k block
1920
1080
Find the closest patch
linear search(search every patch one by one)
query
![Page 4: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/4.jpg)
Introduction(2/2)• (Streaming) Massive Data Sets => High Dimensional Vectors
• E.g. 8*8 patch => v = [ v1, v2, ..., vi , …, vN ] , dimension N = 64• Linear search = find nearest neighbor
• For very large databases of high-dimensional items• Time-consuming• Needs to find Approximate Nearest Neighbors (ANN) for each
patch in real time.• Curse of dimensionality
• Existing ANN methods include trees and hashes.• KD-trees• Locality-Sensitive Hashing(LSH)
![Page 5: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/5.jpg)
• A collision occurs when two points hash to the same value
Hash tablebucket
Look up table
Projection = hash function
Repeat L times=>the closest point will collide most times
Random line
Locality Sensitive Hashing for Finding Nearest Neighbors(1/2)
![Page 6: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/6.jpg)
Locality Sensitive Hashing for Finding Nearest Neighbors(2/2)
• Hash function:
• a is d-dimensional random vector• r is predefine integer
• Constant width of each quantization
bin• b is random value from range [0, r]
• To balance quantization error• v is the original vector
1. indexing
2. search
![Page 7: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/7.jpg)
Coherency Sensitive Hashing (CSH)
s=1, 4x4 kernel
White:1, Black:-1
![Page 8: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/8.jpg)
Coherency Sensitive Hashing (CSH)
![Page 9: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/9.jpg)
4.2.1 Candidate Creation• Patch a, a1,a2 of image A; Patch b, b1,b2 of image B :
Each entry can store k patches from each image=>total k+2*(k+1)+k=4k+2 candidates
![Page 10: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/10.jpg)
4.2.2 Candidate Ranking• Given the candidate set (of size 4k + 2), to find the
nearest one.• Main overall time consumer• Approximate the process, have a little impact, greatly reduce time.
• Use Walsh Hadamard (WH) projections• Already computed in the indexing stage.• Accumulating the projections of the differences of patches on the
WH kernels.
![Page 11: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/11.jpg)
Experiments• We collected 133 pairs of images, taken from 1080p HD
![Page 12: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/12.jpg)
Experiments• We computed the exact nearest neighbor match to serve as a ground
truth.• A novel algorithm PatchMatch [4] to compare :
• Not as accurate as LSH or KD-trees.• So fast. The key to this speedup is spatial coherent.
[4] C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman.PatchMatch: A randomized correspondence algorithm for structural image editing. In SIGGRAPH, 28(3), 2009.
Error : not the same match with ground truth
![Page 13: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/13.jpg)
Image Reconstruction• Reconstruct image A, use image B• Such reconstructions are very common in many
applications.• image editing (e.g. retargetting, inpainting), image denoising and
super-resolution…
• It simply replaces each pixel with the average of the corresponding pixels.
Image A Image B
![Page 14: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/14.jpg)
![Page 15: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/15.jpg)
![Page 16: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/16.jpg)
![Page 17: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/17.jpg)
![Page 18: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/18.jpg)
![Page 19: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/19.jpg)
More results on web : http://www.eng.tau.ac.il/~simonk/CSH/index.html
![Page 20: Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference](https://reader036.vdocument.in/reader036/viewer/2022062320/56649d2d5503460f94a03b7d/html5/thumbnails/20.jpg)
Conclusions• We proposed an algorithm for computing ANN fields
termed Coherency Sensitivity Hashing.• Follows LSH search scheme • But combines image coherency cues
• It was shown to be faster and more accurate than PatchMatch.