low-rank based algorithms for rectification, repetition ...€¦ · 2d image rectification façade...
TRANSCRIPT
![Page 1: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/1.jpg)
Low-Rank Based Algorithms for Rectification, Repetition Detection and De-Noising in Urban Images
A dissertation proposal by
Juan Liu
![Page 2: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/2.jpg)
Committee
Committee:
Professor Ioannis Stamos, Mentor, Hunter College
Professor Yingli Tian, City College
Professor Zhigang Zhu, City College
Outside Member:
Professor Emmanouil Z. Psarakis, University of Patras, Greece
![Page 3: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/3.jpg)
Current Work
2D image rectification
Façade texture selection
Efficient Kronecker Product model
Automatic repeated patterns detection
Reconstruction & photorealistic rendering of urban environments, symmetry detection, hole filling, etc.
![Page 4: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/4.jpg)
Façade Image Rectification
Vanishing points (VPs) detection
TILT
Vanishing points Draw m hypothetical lines at angular intervals for each vanishing point
Output
![Page 5: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/5.jpg)
Façade Texture Selection
Let C be the non-negative m×m matrix, with each element representing the number of Harris corners inside a block
Slide a window of size r×c along C
Compute the sample mean deviation of the sample median of matrix C, μC
![Page 6: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/6.jpg)
Façade Texture Selection
![Page 7: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/7.jpg)
Kronecker Product Model
![Page 8: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/8.jpg)
A Toy Example
Input facade Repeated patterns
![Page 9: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/9.jpg)
Repeated Patterns Detection via Kronecker Product Model
• Define the cost function:
• The minimization problem:
![Page 10: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/10.jpg)
Solution to The Minimization Problem
Theorem 1: Let be the Singular Value Decomposition of the rearranged counterpart of matrix . Then partition matrices , patterns and weighting factors should satisfy the following:
![Page 11: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/11.jpg)
Solution to The Minimization Problem
Lemma 1: Assuming that matrices are known, then the optimal and are related as follows:
![Page 12: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/12.jpg)
Spatial Periods Estimation via Cross Correlation
• Consider that are known, then we can estimate the periods for facade partition
3×10 partition blocks Fi,j
Column-wise Cross-Correlation sequences: distance between the
adjacent peaks provides the period information.
Partition
![Page 13: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/13.jpg)
Block Vectorization
• Vectorization of partition blocks
• The equivalent cost function:
Vectorization
rank = 30
![Page 14: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/14.jpg)
Estimating K by Clustering
Reshape
rank = 4
![Page 15: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/15.jpg)
Estimation of Mκ, κ =1, 2, …, K★
• Reshape the K★ indicators vectors from Algorithm 2
![Page 16: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/16.jpg)
Computing Patterns and Weighting Factors
• Using Lemma 1 and Algorithm 2
Group 1
Group 2
Group 3
Group 4
Classification
Classification
Classification
Classification
Refinement
Refinement
Refinement
Refinement
1-0 patterns re-construction via Kronecker Product Model
![Page 17: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/17.jpg)
Experiments and Evaluation
• Experiment 1: image rectification and texture selection
• Experiment 2: repeated patterns detection 89 façade images with ground-truth
Success rate 96%
Pixel-wise comparison
Success threshold: 91% match with ground-truth
![Page 18: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/18.jpg)
Results
![Page 19: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/19.jpg)
Proposed Work
Improve the estimation of K★
Extend the method to model nested patterns
Apply the model to 3D point clouds
![Page 20: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/20.jpg)
Improve the estimation of K★
• Limitation: the accuracy and efficiency is drawn back by
– K-means clustering algorithm
![Page 21: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/21.jpg)
Period Computation for Nested Patterns
• The current model may cut a bigger pattern into pieces due to that:– The most frequently appeared pattern dominates the final period
![Page 22: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/22.jpg)
Apply the model to 3D point clouds
![Page 23: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/23.jpg)
Timeline
• My work plan to complete the dissertation is arranged as follows:– Jan. 9 2015 - Jan. 20 2015:
• Finalize the implementation of new methods.
• Run experiments to test the new methods.
• Analyze the performance based upon experiment results.
– Jan. 21 2015 - Feb. 21 2015: Complete the rest part of my thesis and prepare for defense.
– Finally: Defend in March 2015.
![Page 24: Low-Rank Based Algorithms for Rectification, Repetition ...€¦ · 2D image rectification Façade texture selection Efficient Kronecker Product model Automatic repeated patterns](https://reader033.vdocument.in/reader033/viewer/2022050210/5f5ca1fc3767802f5f2d706c/html5/thumbnails/24.jpg)
Related Publications
• One paper “Automatic Kronecker product model based detection of repeated patterns in 2D urban images” that is related to this proposal has been published in IEEE International Conference on Computer Vision (ICCV) 2013 (accepatance rate 28%, 1600 submissions).
• Another related paper has been submitted to the IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) recently.