qiaochu li, qikun guo, saboya yang and jiaying liu* institute of computer science and technology...
TRANSCRIPT
Qiaochu Li, Qikun Guo, Saboya Yang and Jiaying Liu*
Institute of Computer Science and TechnologyPeking University
Scale-Compensated Nonlocal MeanSuper Resolution
2013
2
Outline
Introduction Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
3
Outline
Introduction Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
4
Multi-Frame SR
Converge low resolution images into a high resolution image Direct motion estimation
INVALID in complex situation
5
Nonlocal Means SR
Image content repeats in neighborhoods In temporal and spatial domains Probabilistic motion estimation Weighted average
NLM weight distribution. The weights go from 1 (white) to 0 (black).
6
Problem
Scale may be varied in frames by zooming. Camera motion Object motion
Scale changing effects in adjacent frames. (a) Two adjacent frames, (b) some critical areas of the frames.
7
Outline
Introduction Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
8
Scale-Detector
Using SIFT descriptor to compute scales
Partial matched keypoints and the corresponding scale values.
9
Verification
Verification of scale-detector
Always appears region
The performances of scale-detector in different standard scales and different resolutions,(a) average error by frame scale, (b) average error by frame resolution.
(a) (b)
10
Scale-Compensated NLM
SC NLM finds more similar patches
Comparison of unmodified and modified patch-extractor in patch matching.
11
Procedures
Overview of SC NLM
Scale-detector
Patch extraction
&modification
NLM SR
12
Experimental Results
Downsample Blurred using 3×3 uniform mask Decimated by 3× factor Additive noise with standard deviation 2
Objective measurement Subjective measurement
13
Experimental Results
3×, Objective measurement (PSNR)
Sequence NLM ARI-SWR SC-NLM
Foreman 31.15 30.96 31.27
Tempete 22.85 22.74 23.00
Text 29.23 30.06 30.11
Man 27.14 27.02 27.29
14
Experimental Results
3×, Subjective measurement (SSIM)
Sequence NLM ARI-SWR SC-NLM
Foreman 0.8109 0.8001 0.8151
Tempete 0.6927 0.6737 0.7013
Text 0.8592 0.8512 0.8633
Man 0.7780 0.7617 0.7831
15
Experimental Results
a) Result of whole frame. b) High resolution frame. c) NLM SR. d) SC NLM.
16
Experimental Results
a) Result of whole frame. b) High resolution frame. c) NLM SR. d) SC NLM
17
Outline
Introduction Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
18
Conclusion
When patches are convert into
SAME SCALE, we can find more
SIMILAR PATCHES, we can use more
COMPLEMENTARY INFORMATION to reconstruct a
HIGH RESOLUTION & QUANLITY IMAGE.
19
Future Work
More accurate scale-detector Segmentation based scale-detector
Combination of rotation and translation-invariant algorithm Rotation-invariant measurement Translation-invariant measurement
20