Qiaochu Li, Qikun Guo, Saboya Yang and Jiaying Liu*

Institute of Computer Science and TechnologyPeking University

Scale-Compensated Nonlocal MeanSuper Resolution

2013

2

Outline Introduction

Multi-frame SR Nonlocal means SR (NLM SR)

Our Algorithm Scale-detector Scale-Compensated NLM Experimental results

Conclusion & Future work

3

Outline Introduction

Multi-frame SR Nonlocal means SR (NLM SR)

Our Algorithm Scale-detector Scale-Compensated NLM Experimental results

Conclusion & Future work

4

Multi-Frame SR Converge low resolution images into a high

resolution image Direct motion estimation

INVALID in complex situation

5

Nonlocal Means SR Image content repeats in neighborhoods

In temporal and spatial domains Probabilistic motion estimation Weighted average

NLM weight distribution. The weights go from 1 (white) to 0 (black).

6

Problem Scale may be varied in frames by zooming.

Camera motion Object motion

Scale changing effects in adjacent frames. (a) Two adjacent frames, (b) some critical areas of the frames.

7

Outline Introduction

Multi-frame SR Nonlocal means SR (NLM SR)

Our Algorithm Scale-detector Scale-Compensated NLM Experimental results

Conclusion & Future work

8

Scale-Detector Using SIFT descriptor to compute scales

Partial matched keypoints and the corresponding scale values.

9

Verification Verification of scale-detector

Always appears region

The performances of scale-detector in different standard scales and different resolutions,(a) average error by frame scale, (b) average error by frame resolution.

(a) (b)

10

Scale-Compensated NLM SC NLM finds more similar patches

Comparison of unmodified and modified patch-extractor in patch matching.

11

Procedures Overview of SC NLM

Scale-detector

Patch extraction

&modification

NLM SR

12

Experimental Results Downsample

Blurred using 3×3 uniform mask Decimated by 3× factor Additive noise with standard deviation 2

Objective measurement Subjective measurement

13

Experimental Results 3×, Objective measurement (PSNR)

Sequence NLM ARI-SWR SC-NLMForeman 31.15 30.96 31.27Tempete 22.85 22.74 23.00Text 29.23 30.06 30.11Man 27.14 27.02 27.29

14

Experimental Results 3×, Subjective measurement (SSIM)

Sequence NLM ARI-SWR SC-NLMForeman 0.8109 0.8001 0.8151Tempete 0.6927 0.6737 0.7013Text 0.8592 0.8512 0.8633Man 0.7780 0.7617 0.7831

15

Experimental Results

a) Result of whole frame. b) High resolution frame. c) NLM SR. d) SC NLM.

16

Experimental Results

a) Result of whole frame. b) High resolution frame. c) NLM SR. d) SC NLM

17

Outline Introduction

Multi-frame SR Nonlocal means SR (NLM SR)

Our Algorithm Scale-detector Scale-Compensated NLM Experimental results

Conclusion & Future work

18

Conclusion When patches are convert into

SAME SCALE, we can find more

SIMILAR PATCHES, we can use more

COMPLEMENTARY INFORMATION to reconstruct a

HIGH RESOLUTION & QUANLITY IMAGE.

19

Future Work More accurate scale-detector

Segmentation based scale-detector

Combination of rotation and translation-invariant algorithm Rotation-invariant measurement Translation-invariant measurement

20