mit media lab | camera culture

MIT Media Lab | Camera Culture Shoot Now, Relight Later Illumination Multiplexing with Lock-In Time of Flight Sensors Achuta Kadambi, Ayush Bhandari, Refael Whyte, Adrian Dorrington, Ramesh Raskar [email protected] How Can We Relight a Photograph? In this paper we propose a fusion of two popular contexts, time of flight range cameras and illumination multiplexing. Time of flight cameras are a low cost, consumer-oriented technology capable of acquiring range maps at 30 frames per second. Such cameras have a natural connection to conventional illumination multiplexing strategies as both paradigms rely on the capture of multiple shots and synchronized illumination. While previous work on illumination multiplexing has exploited coding at millisecond intervals, we repurpose sensors that are ordinarily used in time of flight imagstrategiesing to demultiplex via nanosecond coding. ICCP 2014 Illumination Multiplexing Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007 Time of Flight 3D Sensors Time of Flight 3D cameras are an emerging camera technology. Such technology forms the basis for the new Kinect. Time of Flight Operating Principle Operation of ToF range cameras. The time diﬀerence of arrival between the emitted and received code is encoded in the phase oﬀset. Thus, phase encodes depth. Nanosecond vs Millisecond Coding Case 1: Conventional High Speed Multiplexing Case 2: Using a lock-in ToF sensor allows for nanosecond coding. Case 3: Optimizing the Nanosecond Codes is the goal. Illumination Multiplexing Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007 Demonstration: Relighting Post-Capture Demonstration: Scene Relighting. Demonstration: Multiplexing Wavelength Demonstration: Color ToF camera by multiplexing RGB lights. Hardware Prototype We validate our technique with a hardware prototype. It consists of an FPGA, Lock-in Sensor, and Solid State Illumination sources. www.media.mit.edu/~achoo/demux

Upload: nen

Post on 24-Feb-2016

41 views

Category:

Documents

2 download

Report

Download

Tags:

Embed Size (px):

DESCRIPTION

MIT Media Lab | Camera Culture. Shoot Now, Relight Later. [email protected]. Illumination Multiplexing with Lock-In Time of Flight Sensors. www.media.mit.edu/~achoo/demux. ICCP 2014. Achuta Kadambi , Ayush Bhandari, Refael Whyte, Adrian Dorrington , Ramesh Raskar. - PowerPoint PPT Presentation

TRANSCRIPT

MIT Media Lab | Camera Culture

Shoot Now, Relight LaterIllumination Multiplexing with Lock-In Time of Flight SensorsAchuta Kadambi, Ayush Bhandari, Refael Whyte, Adrian Dorrington, Ramesh Raskar

[email protected]

How Can We

Relight a

Photograph?In this paper we propose a fusion of two

popular contexts, time of flight range cameras

and illumination multiplexing. Time of flight

cameras are a low cost, consumer-oriented

technology capable of acquiring range maps

at 30 frames per second. Such cameras have

a natural connection to conventional

illumination multiplexing strategies as both

paradigms rely on the capture of multiple

shots and synchronized illumination. While

previous work on illumination multiplexing has

exploited coding at millisecond intervals, we

repurpose sensors that are ordinarily used in

time of flight imagstrategiesing to demultiplex

via nanosecond coding.

ICCP 2014

Illumination Multiplexing

Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007

Time of Flight 3D Sensors

Time of Flight 3D cameras are an emerging camera technology. Such technology forms the basis for the new Kinect.

Time of Flight Operating Principle

Operation of ToF range cameras. The time difference of arrival between the emitted and received code is encoded in the phase offset. Thus, phase encodes depth.

Nanosecond vs Millisecond Coding

Case 1: Conventional High Speed MultiplexingCase 2: Using a lock-in ToF sensor allows for nanosecond coding.Case 3: Optimizing the Nanosecond Codes is the goal.

Illumination Multiplexing

Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007

Demonstration: Relighting Post-Capture

Demonstration: Scene Relighting.

Demonstration: Multiplexing Wavelength

Demonstration: Color ToF camera by multiplexing RGB lights.

Hardware Prototype

We validate our technique with a hardware prototype. It consists of an FPGA, Lock-in Sensor, and Solid State Illumination sources.

www.media.mit.edu/~achoo/demux

OPTICAL METROLOGY · Extreme Computational Imaging: Photography, Health-tech and Displays Ramesh Raskar, MIT Media Lab., United States The Camera Culture Group at the MIT Media Lab

Sam Ford Project Manager MIT Convergence Culture Consortium Program in Comparative Media Studies

MIT Media Lab Camera Culture Ramesh Raskar Jack Tumblin Computational Photography: Advanced Topics Computational Photography: Advanced Topics Paul Debevec