Diffusion Coding Photography

for Extended Depth of Field

SIGGRAPH 2010

Ollie Cossairt, Changyin Zhou, Shree NayarColumbia University

Conventional Camera (F/1.8)

Focused Image PSF Image Noise Captured Image

Camera Blur ModelSpati

al dom

ain

Focused Image MTF Image Noise Captured ImageFrequency

dom

ain

Deblurring Problems

MTFCaptured image

Low SNR

low MTF values

Problem 1:

Focused image

Lens

Sensor

P

Object

Q

Problem 2:

Variation with depth

Extending Depth of Field: Previous Work

Focus Sweep Cameras[Hausler ’72]

[Nagahara et al. ’08]Lens Focal PlaneSensor

Wavefront Coding Cameras

[Dowski and Cathey ’95] [Chi and George ’01] [Garcia-Guerrero et al. ‘07]

Other Related Work[Levin et al. ’07][Veeraraghavan et al. ’07] [Levin et al. ’09]

Focus Sweep Camera

Lens

Scene

Sensor

+ + + + + + =InstantaneousPSF

t = 1 t = 2 t = 3 t = 4 t = 5 t = 6 t = 7

Final PSF

[Hausler ’72][Nagahara et al. ’08]

Focus Sweep Camera

+ + + + + + =InstantaneousPSF

t = 1 t = 2 t = 3 t = 4 t = 5 t = 6 t = 7

Final PSF

InstantaneousPSF

t = 1

+

t = 2

+

t = 3

+

t = 4 t = 5

+

t = 6

+

t = 7

+

depth

1depth

2

[Levin et al. ’09]

2D MTF

=

Final PSF

Lens

Scene

Sensor

[Hausler ’72][Nagahara et al. ’08]

Wavefront Coding

Lens Scene

Sensor

[Levin et al. ’09]

2D MTF

Cubic Phase Plate

x u

[Dowski and Cathey ’95]

y

Ambiguity Function

slice

MTF

Lens Resolution Target

Sensor

Focu

s S

weep

Wavefr

on

tC

od

ing

EDOF Camera Comparison

depth

Focu

s S

weep

Wavefr

on

tC

od

ing

Deblurred image

EDOF Camera Comparison

Deblurring Error vs. Depth

Deblu

rrin

g E

rror

noise

Deblurring Error

Deblurring Error

Depth

Wavefront Coding

Focus Sweep

Focus Sweep Wavefront Coding

Is it possible to achieve the performance of focus sweep

without moving parts?

Optical DiffusersCircular diffuser

[http://www.luminitco.com]

Diffuser sheets

w

SensorDiffuser

Light ray

x

x

Scatter function

w

SEM image

Diffuser Kernels

LensSensor

x u

A

u

x

A/2

-A/2

With diffuser

A/2

-A/2

u

x

Without diffuser

Light

field

space

w w

A/2

-A/2

u

x

Without diffuser

Diffuser Kernels

LensSensor

w

u

x

With diffuser

w

Light

field

space

x u

Diffuser Kernels

LensSensor

w

u

x

A/2

-A/2

u

x

u

x

Diffuser kernel

Light field Diffuser kernel

Coded light field

x u

Without diffuser With diffuser

Light

field

space

Diffusion Coded PSF

A/2

-A/2

u

x

Without diffuser

u

x

With diffuser

u

x

Diffuser kernel

x

project

x

project

Light

field

space

x

project

Senso

r sp

ace

Camera PSF Scatter function

Coded PSF

Radially Symmetric Light Field

Lensv

u

Sensor

For an on-axis, isotropic point source:

Radially Symmetric Diffuser

For a radially-symmetric diffuser kernel

Lens Sensor Sensor

Radially Symmetric Diffuser PSFs

Radially symmetric diffuser

Coded PSF Scatter function

Camera PSFCoded PSF Scatter function

Camera PSF

Conventional diffuser

Normalized frequencyNormalized frequencyNormalized frequencyNormalized frequency

PSF Vs. Depth

MTF Vs. Depth

k7

k9

k11

k13

K7

K9

K11

K13

-50px 50px -50px 50px -50px 50px -50px 50px

depth

depth

Diffusion Coding Performance

Deblurring Error vs. Depth

Depth

Wavefront CodingFocus Sweep

noise

Diffusion Coding (light field)Diffusion Coding (wave optics)

Similar performance to focus sweep without moving parts

Diffuser ImplementationDiffuser scatter function

)(rf

r (mm)110

Th

ickn

ess

(u

m)

3

863

2

1

)(rt

r (mm)

[www.rpcphotonics.com]

Diffuser heightmap Fabricated Diffuser

Diffuser surface profile

[Sales et al. ‘03]

Garcia-Guerrero

Comparison with Prior Work

Deblurring Error vs. Depth

Depth

Diffusion Coding

Diffusion coding significantly outperforms prior work

Diffusion Coding Experiments

Fabricated DiffuserCannon 50mm EF lensCannon 450D Sensor

Experimental Setup

Measured PSFs

depth

Without diffuser

with diffuser

BM3D Deblurring Algorithm

[Dabov et al. ‘08]

Examples

Conventional Camera

f-number = 1.8, exposure time = 16ms

Conventional Camera

f-number = 18 , exposure time = 16ms

Diffusion Coding

Captured

f-number = 1.8, exposure time = 16ms

Diffusion Coding

Deblurred

f-number = 1.8, exposure time = 16ms

Conventional Camera

• f-number = 1.8• exposure time = 10ms

Diffusion Coding

• f-number = 1.8• exposure time = 10ms

Captured

Diffusion Coding

• f-number = 1.8• exposure time = 10ms

Deblurred

Conventional Camera

• f-number = 1.8• exposure time =

12.5ms

Diffusion Coding

• f-number = 1.8• exposure time =

12.5ms

Captured

Diffusion Coding

• f-number = 1.8• exposure time =

12.5ms

Deblurred

Conventional Camera

f-number = 1.8, exposure time = 16ms

Diffusion Coding

Captured

f-number = 1.8, exposure time = 16ms

Diffusion Coding

Deblurred

f-number = 1.8, exposure time = 16ms

LimitationsConventional Camera Diffusion Coding

•Loss of image texture

•Loss of contrast

•Occlusion errors

Conclusions

Diffusion Coding Theory

Diffusion Coding Examples

Radially Symmetric Diffusers

LensSensor

Diffuser

Diffusion Coding Implementation