virtual computational camera changyin zhou [email protected] computational photography,...

9
Virtual Computational Camera Changyin Zhou [email protected] Computational Photography, Spring 2009

Post on 22-Dec-2015

226 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

Virtual Computational Camera

Changyin Zhou

[email protected]

Computational Photography, Spring 2009

Page 2: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

What is Camera?

Lens, Lens Array, Coded Aperture, 3D Phase plate ….

- Camera is a projection: [4D light field] [2D image]

2D Sensor

4DLight Field

Shape, BRDF

Illumination

- Optical Elements are transforms: [4D Light Field] [4D Light Field]

Input: 4D light field Input: Transforms … Ouput: 2D image

Page 3: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

Motivation

From Scratch(using Matlab, Python, C, …)

- Ray tracing

- Not well structured for optical design

- Not easy to develop from scratch (little intuition, time-consuming, …)

Optical Simulation Software(Zemax, ASAP…)

- Optical Element Based- Define Optical Elements Physically (e.g. curvatures of surface, refractive index)

Works as a real system in most cases

Disadvantage:

- No light field …- A big gap between physics and math- Designed for conventional optical elements

• Coded aperture?• Focal Sweep?

Two Typical Ways to Build Virtual Cameras

Page 4: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

Build An Abstract-Level Virtual Camera

Input: 4D light field Input: Transforms… Output: 2D image

An Abstract-LevelVirtual Camera

1. Light Field Based

2. “Object-Oriented”: Optical Elements

3. Concept-Level: Defined mathematically

Purpose:

Serve Computational Camera Research

Features:

Scene:4D Light Field

Optical Element:Transform of 4D Light Field

Sensor:Projection: 4D Light Field 2D Image

4. Pipeline

Page 5: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

Build An Abstract-Level Virtual Camera

Scene:LF(u, v, s, t)

Lens:

function outLF = Lens(inLF, arg);

For each [u, v] outLF{u, v} =

interp2(X, Y, inLF{u, v}, X-u/f, Y-v/f);end

Any other element:

function outLF = Other(inLF, arg);

….….….

Sensor:

function outIM = sensor(inLF, inDist);

For each [u, v] outIM(round(XX(1, :)*desU), round(YY(:, 1)'*desV) … = outIM(round(XX(1, :)*desU), round(YY(:, 1)'*desV)… + interp2(XX, YY, inLF{u, v}, … round(XX*desU)/desU, round(YY*desV)/desV);

end

Page 6: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

Build An Abstract-Level Virtual Camera

Function VirtualCamera(parameters);

LF = loadLF(filename);LF = lens(LF, arg);LF = codedAperture(LF, coding, arg);LF = propagation(LF, distance);

LF = otherOptics(LF, arg);

LF = propagation(LF, distance);

IM = sensor(LF);

END

Camera

Page 7: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

Challenges

• Resolution– Huge data (Constrained to 10 x 10 x 1000 x 1000 input light fi

eld in this project)

– Angular/spatial resolution balance could be different at every layer.

• An effective Framework (user interface, data structure, function interface, …)

• Ray Interpolation

Page 8: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009

Thank you!

Page 9: Virtual Computational Camera Changyin Zhou changyin@cs.columbia.edu Computational Photography, Spring 2009