Download - Noise Estimation from a Single Image
![Page 1: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/1.jpg)
Noise Estimation from a Single Image
Ce Liu William T. Freeman
Richard Szeliski Sing Bing Kang
![Page 2: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/2.jpg)
Parameter Tweaking in Computer Vision
Computer vision algorithms suffer from hand tuning parameters for particular images or image sequences
We want vision algorithms that behave properly under varying lighting conditions, blur levels and noise levels
Our work is one step in that direction
Given an image, estimate the noise level
Modify vision algorithms to be independent of noise
![Page 3: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/3.jpg)
Image Noise Is Important in Vision
In image denoising the noise is assumed to be known as Additive Gaussian White Noise (AWGN)
However, in real applications the noise is unknown and non-additive
Many other computer vision algorithms also explicitly or implicitly assume the type and level of image noise
Hard to make vision algorithms fully automatic without knowing noise
![Page 4: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/4.jpg)
Noise Level Function (NLF)
The standard deviation of noise is a function of image brightness I
Measurable by fixing the camera and taking multiple shots of a static scene
For each pixel: Mean: I Standard deviation:
NLF depends on camera, ISO, shutter speed, aperture
Our goal is to estimate NLF from a single image How to estimate noise without separating noise and signal?
I
![Page 5: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/5.jpg)
An Example Image
![Page 6: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/6.jpg)
Piecewise Smooth Image Prior
Patch
= +
Signal Residual
For each RGB channel:
Brightness mean I
Standard deviation
Red Green Blue
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
Brightness
Sta
nd
ard
d
evi
ati
on
Affine model
![Page 7: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/7.jpg)
Piecewise Smooth Image Prior
Patch
= +
Signal Residual
Red Green Blue
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
Brightness
Sta
nd
ard
d
evi
ati
on
![Page 8: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/8.jpg)
Piecewise Smooth Image Prior
Patch
= +
Signal Residual
Red Green Blue
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
Brightness
Sta
nd
ard
d
evi
ati
on
![Page 9: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/9.jpg)
Segmentation-based Approach
Observed image
![Page 10: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/10.jpg)
Segmentation-based Approach
Over-segmentation
![Page 11: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/11.jpg)
Segmentation-based Approach
Signal
![Page 12: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/12.jpg)
Segmentation-based Approach
Residual=
noise + unmodelled
image variation
![Page 13: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/13.jpg)
Estimate NLFs
Assume brightness mean I is accurate estimate
Standard deviation is an over-estimate: (may contain signal)
The lower envelope is the upper bound of NLF
III
Brightness
Resi
du
al st
d.
dev.
![Page 14: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/14.jpg)
Issues
Should the curve be strictly and tightly below the points?
III
Brightness
Resi
du
al st
d.
dev.
![Page 15: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/15.jpg)
Issues
Should the curve be strictly and tightly below the points?
How to handle the missing data?
III
Brightness
Resi
du
al st
d.
dev.
![Page 16: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/16.jpg)
Issues
Should the curve be strictly and tightly below the points?
How to handle the missing data?
Correlation between RGB channels?
III
Brightness
Resi
du
al st
d.
dev.
![Page 17: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/17.jpg)
Solutions
Formulate the inference problem in a probabilistic framework
Learn the prior of noise level functions
III
Brightness
Resi
du
al st
d.
dev.
![Page 18: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/18.jpg)
Outline
Over-segmentation and per-segment variance analysis
Learning the priors of noise level functions (NLF) Synthesize CCD noise Sample noise level functions Learn the prior of noise level functions
Inference: estimate the upper bound of NLF Bayesian MAP to estimate NLFs for RGB channels
Applications Adaptive bilateral filtering Canny edge detection
![Page 19: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/19.jpg)
Noise model
Camera response function (CRF) f: download from Columbia camera response function database (used 196 typical CRFs)
Tsin et. al. Statistical calibration of CCD image process. ICCV, 2001
Camera Noise
Atmospheric Attenuation
Lens/geometricDistortion
CCD Imaging/Bayer Pattern
Fixed PatternNoise
Shot Noise
Thermal Noise
Interpolation/Demosaic
WhiteBalancing
GammaCorrection
A/D Converter
Dark Current Noise
t
QuantizationNoise
SceneRadiance
DigitalImage
CameraIrradiance
)( cs nnLfI
I
L
2)(Var,0)(E sss Lnn Dependent noise:Independent noise: 2)(Var,0)(E ccc nn
![Page 20: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/20.jpg)
Synthesize CCD Noise
Estimate NLF )(),,;( IIEfI Ncs
I NI
Camera response function: f
Dependent noise:
Independent noise:s
c
![Page 21: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/21.jpg)
Sample NLFs by Varying the ParametersCamera response function (CRF) f s
c
Dependent noise:
Independent noise:s
c
0.02
0.18
0.02
0.18
0.02 0.04 0.06
![Page 22: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/22.jpg)
The Prior of NLFs
![Page 23: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/23.jpg)
Likelihood Function
The estimated standard deviation should be probabilistically bigger than and close to the true value
Bayesian MAP inference
nh (n+1)h
)ˆ,ˆ( nnI
I
)|ˆ( p
![Page 24: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/24.jpg)
Validation (1): Synthetic Noise
Add synthetic CCD noise, estimate, compare to the ground truth
— ground truth
estimated———
![Page 25: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/25.jpg)
Validation (2): Measure NLF of a Real Camera
29 images were taken under the same settings (the camera is not in the database for training)
The real NLF is obtained by computing mean and variance per pixel
![Page 26: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/26.jpg)
Validation (3): Robustness Test
Verify that different images from the same camera give the same estimated NLF (camera not in the database for training)
![Page 27: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/27.jpg)
Application (1): Adaptive Bilateral Filtering
Bilateral filter is an edge-preserving low-pass filter Spatial sigma and range sigma
Adaptive bilateral filter Down-weigh RGB values by signal and noise covariance matrices The range sigma is set to be a function of the estimated standard
deviation of the noise
From Durand and Dorsey, SIGGRAPH 02
Input noisy image Smoothing kernel Denoised image
![Page 28: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/28.jpg)
Test on Low and High Noise
low noise high noise
Red Green Blue Red Green Blue
![Page 29: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/29.jpg)
Results—Adaptive Bilateral Filtering
Standard bilateral filtering
Adaptive bilateral filtering
low noise high noise
![Page 30: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/30.jpg)
Results—Adaptive Bilateral Filtering
Zoom in high noise
Standard bilateral filtering
Adaptive bilateral filtering
![Page 31: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/31.jpg)
Application (2): Canny Edge Detection
low noise high noise
Red Green Blue Red Green Blue
![Page 32: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/32.jpg)
Results—Canny Edge Detection
low noise high noise
Parameters adapted in MATLAB
Parameters adapted by estimated noise
![Page 33: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/33.jpg)
Conclusion
Piecewise-smooth image prior model to estimate the upper bound of noise level function (NLF)
Estimate the space of NLF by simulating CCD camera on the existing CRF database
Upper bounds are verified by both synthetic and real experiments
An important step to automate vision algorithms independent of noise
![Page 34: Noise Estimation from a Single Image](https://reader036.vdocument.in/reader036/viewer/2022062423/56814527550346895db1ee58/html5/thumbnails/34.jpg)
Thank you!
Ce Liu William T. Freeman
CSAIL MIT
Rick Szeliski Sing Bing Kang
Microsoft Research
Noise Estimation from a Single Image