high-quality computational imaging through simple lens

High-Quality Computational Imaging Through Simple Lenses

F. Heide1, M. Rouf1, M. Hullin1, B. Labitzke2, W. Heidrich1, A. Kolb2

1University of British Columbia, 2University of Siegen

(ACM Transactions on Graphics, 2013)

Presented by Monica Drăgan

Advanced Methods in Computer Graphics, SS2014Freitag, 4. April 2014

Correct for :● Geometric distortion● Spherical aberation● Chromatic aberation● Coma



expensive, large, heavy



expensive, large, heavy

Alternative

approach

to high-quality

photography


Correct for :● Geometric distortion● Spherical aberation● Chromatic aberation● ComaCOM

PUTATIONALLY

Simple lenses:● Plano-convex● Biconvex● Achromatic doublets


Blurred captured image


Corrected image


Outline

● What are the challenges?

● Previous work

● Paper contribution

● Current approach

● Performance

● Applications

● Future work


Point spread function (PSF)

f/2.0 f/4.5


Point spread function (PSF)

● Spatially large (50x50px)

● Spatial variation

● Wavelength dependent


Image convolution

Blur

kernel

=Observed

blurred imageUnderlying

sharp image

*


Deconvolution

Blur

kernel

=Observed

blurred imageUnderlying

sharp image

*-1


Deconvolution

*-1 =

*-1 =

*-1 =


Outline


● Previous work



● Performance

● Applications

● Future work


Previous work

Levin et al. '07

Idependent

deconvolution

on each

color channel

Schuler et al. '11

Deconvolution in YUV space

Cossairt & Nayar '10

Use chromatic

aberations to

increase DOF


Outline


● Previous work



● Performance

● Limitations

● Future work


Paper contribution

● Cross-channel instead of channel-independent deconvolution


Paper contribution


Observations: ● hue changes are sparse and occure near the edges● edges appear in the same place in all channels

Blurred image Sharp image


Paper contribution


Severe ringing

Residual blur

Levins '07 Current approachBlurred image


Paper contribution


● Efficient convex optimization solver [Chambolle & Pock '11]


Paper contribution



● Robust approach for per-channel PSF estimation


Paper contribution



● Robust approach for per-channel PSF estimation

White noise calibration pattern


Outline


● Previous work



● Performance

● Applications

● Future work


Current approach

*-1 =

*-1 =

*-1 =

estimate

PSF

(lens specific)


Current approach

*-1 =

*-1 =

*-1 =

formulate

the optimization

problem

estimate

PSF

(lens specific)


-*

Optimization problemLeast squares

data fitting


Optimization problemSparse

image gradient


Optimization problem

Cross-channel prior


Least squares data fitting

Cross-channel priorSparse

image gradient


Weighted contributions




image gradient

PROBLEM IS CONVEX





image gradient

PROBLEM IS CONVEX


Efficiently solvable by standard forward – backward splitting methods

[Chambolle & Pock '11]



Original Resulted


Regularization for

low intensity areas

needed

Original


Resulted



Original Resulted

Unscaled gradients


Improved result


Original Initial result


Least squares

data fittingCross-channel prior

Regularization for

low intensity pixels

Optimization problemSparse

image gradient


PSF estimation

● PSF calibration – with white noise pattern● for a certain aperture

I J

f/2.0 f/4.5


PSF estimation

● PSF calibration – with white noise pattern● for a certain aperture

● for each tileI J

f/2.0 f/4.5


PSF estimation

● PSF calibration – with wite noise pattern● for a certain aperture● for each tile

I JB

* ? =


PSF estimation


● Non-blind deconvolution

Least squares

data fitting

I JB


PSF estimation



Least squares

data fitting

Energy

conservation

Gradient

total variation

I JB


PSF estimation



Efficiently solvable by standard

forward – backward splitting methods

[Chambolle & Pock '11]I JB


PSF estimation




PSF estimation

● Once per lens

● Accurate (two consecutive shots with different apertures)


Outline


● Previous work

● Tool's magic


● Performance

● Applications

● Future work


Performance

● Outperforms other state-of-art methods

● Postprocessing image quality comparable to that of a compact camera (at f/4.5)

● Improves also images taken with compact cameras


OriginalDeblurred

LevinDeblurred Schuler

Current approach


Outline


● Previous work

● Tool's magic


● Performance

● Applications

● Future work


Applications

● Deconvolution for multispectral cameras

● Remove residul blur in regular cameras


Outline


● Previous work

● Tool's magic


● Performance

● Applications

● Future work


Future work

● Use drastically simpler lens design

● Optimize lenses to generate blur that is easier to remove

● Calibrate full depth-dependent PSFs

● Speed up the computation (distributed computing)● Running time: ~10-20 s for a 8MP image


Thank you!

& special thanks to Marios Papas


Regularization for

low intensity pixels

Efficiently solvable by standard forward – backward splittin methods

[Chambolle & Pock '11]

Least squares

data fittingCross-channel prior

1. Optimization problemSparse

image gradient


Original

Original


Deblurred


Running time

high-quality computational imaging through simple lens

Education

computer graphics

heavyadvanced methods

dofadvanced methods

monica drganadvanced

heavyalternative approach

previous worklevin

function psff2

idependent deconvolution