We estimate {𝜎𝑖} conditioning on {𝜌𝑖𝑗∗ }

𝜎𝑖2 =argmin 𝑤𝑖𝑗 𝜎𝑖

2 − 𝜎𝑗2 − 𝜌𝑖𝑗

∗ 2𝑖≠𝑗

𝑤𝑖𝑗: similarity between two faces

Solving a linear system The system is under-determined, up to adding

a constant number. - option 1: assign some images to be zero noise - option 2: assuming the collection contains clean images, assign the least noisy one to be zero. We use this one for evaluations

Joint Noise Level Estimation from Personal Photo Collections

YiChang Shih1*,2 Vivek Kwatra1 Troy Chinen1 Hui Fang1 Sergey Ioffe1 1Google Research 2MIT CSAIL *Internship work at Google

Goal

Pair-wise Relative Noise 𝜌𝑖𝑗 Estimation

Contributions Overview

Absolute Noise Level Estimation with Global Optimization

Results Ground Truth Experiment and Comparison

Given a set of face images from the same person, taken under different lighting and cameras, estimate the noise levels in each image

𝑰𝑛 = 𝑰𝑜𝑟𝑖𝑔 + 𝒏, i.i.d, zero mean. 𝜎 = noise level ≜ 𝑠𝑡𝑑[𝒏]

This is difficult because we cannot decouple 𝒏 from 𝑰𝑛

Key observation: given two noisy images, the noise levels are correlated if they share the same underlying

image content, since 𝜎12 − 𝜎2

2 = 𝑣𝑎𝑟[𝑰𝒏,𝟏] − 𝑣𝑎𝑟[𝑰𝒏,𝟐]

We formulate the estimation as maximizing the joint

probability distribution between all images’ noise levels

The joint distribution is conditioned on the pair-wise

relative noise levels {𝜌𝑖𝑗|𝜌𝑖𝑗 ≜ 𝜎𝑖2 − 𝜎𝑗

2}. We use a two-

stage optimization that first estimates {𝜌𝑖𝑗}, then {𝜎𝑖}

Starting from a face image collection: Preprocess: geometrically and photometrically align the

images with affine transform and color match

Two-stage optimization: Estimating {𝜌𝑖𝑗}: We take a patch-based method. We

first find the patch correspondence between 𝑰𝒊 and 𝑰𝒋,

then find the best estimated relative noise {𝜌𝑖𝑗∗ } from

the patch pairs.

With {𝜌𝑖𝑗∗ }, estimate {𝜎𝑖} by constraining 𝜎𝑖

2 − 𝜎𝑗2 = 𝜌𝑖𝑗

∗

p q q

𝜎𝑖2

𝜎𝑗2

𝜌𝑖𝑗∗

Acknowledgements

33.00

34.00

35.00

36.00

37.00

38.00

39.00

40.00

41.00

0.00 5.00 10.00 15.00 20.00

Me

an P

SNR

Mean Noise Sigma

Mean PSNR (Metric-Q)Mean PSNR (Our Method)Mean PSNR(Best BM3D)

2

6

10

14

18

22

26

30

2 7 12 17

Esti

mat

ed

No

ise

Sig

ma

True Noise Sigma

y=x Line (Ground Truth)Liu et. al. (Mean)Metric-Q (Mean)Our Method (Mean)

Input BM3D Input BM3D

σ=14.5

σ=23.9

σ=23.2

σ=38.2

Input image Our method Metric-Q

σ=35

σ=27

σ=46.3

σ=9.1

User Study

The two faces are not perfectly aligned

We break down the image into patches, and estimate the patch-wise relative noise levels 𝜁𝑝𝑞 by 𝜁𝑝𝑞 ≜ 𝑣𝑎𝑟[𝒑1𝑝] − 𝑣𝑎𝑟[𝒑2𝑞]

Compute pair-wise relative noise by aggregating 𝜁𝑝𝑞:

• 𝑐𝑝𝑞 = exp (−𝜅𝑝𝑞 𝒑1𝑝 − 𝒑2𝑞2

), confidence that (𝑝, 𝑞) is a true correspondence

• For computational efficiency, we selected the best 5 𝑞 s for each 𝑝

𝜌12∗ =

𝑐𝑝𝑞𝜁𝑝𝑞𝑝,𝑞

𝑐𝑝𝑞𝑝,𝑞

I1 I2

Selected References

We show one example below with estimated noise levels and denoised result using BM3D + our method for noise parameter

More subjects

Based on BM3D denoised result, decide which one is preferable

Ran on 71 images, each is evaluated by 3 users

Add synthetic Gaussian noise with different parameters Compare estimated noise levels and denoised result by BM3D

σ 19.25 20.31 27 23

PSNR 22.33 34.65 34.53 34.65

Clean image Synthetic noise Our method Metric-Q Best BM3D

C. Liu, R. Szeliski, S. Kang, C. Zitnick, and W. Freeman. Automatic estimation and removal of noise from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2), 2008 X. Zhu and P. Milanfar. Automatic parameter selection for denoising algorithms using a no-reference measure of image content. IEEE Transactions on Image Processing, 19(12), 2010.

We thank MIT Graphics and Vision group for helpful discussion. We would like to thank the volunteers who participated in the user study.

24%

41%

35%

Metric-Q

Ours

Equally good