Phase Messaging Method for Time-of-flight Cameras

Wenjia Yuan † Richard Howard † Kristin Dana † Ramesh Raskar ‡

Ashwin Ashok † Marco Gruteser † Narayan Mandayam †

† Rutgers University

‡ Massachusetts Institute of Technology

CNS-1065463

5/3/2014 2

Camera-Display Communication

Displays Cameras

Related work: QR codes

5/3/2014 3

[1] Kamijo et al. ICIP. 2008.

Related work: Visual MIMO

5/3/2014 4

[1] Ashok et al. MobiCom. 2010. [2] Yuan et al. WACV. 2012. [3] Yuan et al. GlobalSip.2013

Other related work

5/3/2014 5

[1] Schmid et al. MobiHoc. 2013. [2] Perli et al. MobiCom. 2010. [3] Mondal et al. APCC. 2012.

Our work

5/3/2014 6

Time-of-flight Cameras

Phase Messaging Array (PMA)

Time-of-Flight Cameras

5/3/2014 7

PMD CamBoard Nano Swiss Ranger SR4050

ToF camera sensor

SoftKinetic DS311

Time-of-Flight Cameras

5/3/2014 8

PMD CamBoard Nano Swiss Ranger SR4050

ToF camera sensor

SoftKinetic DS311

Infrared LEDs

Time-of-Flight Cameras

5/3/2014 9

phase difference

depth

SoftKinetic DS311

delay 𝜏

5/3/2014 10

System Overview

5/3/2014 11

Extracted Message Stream

Depth Image

LEDs of the ToF camera

Depth Recovery

Sensor of the ToF camera

Photo Detector

Amplifier

LED

Phase Shifting

Multiplexer

Message Stream

FIFO

Data Format

Converter

Shift Register

Light Detection Phase Modulation

……

p

p0 pn-1

A0

Am-1

Data Processing

Transmitter PMA

Synchronized

Receiver ToF Camera

Phase Messaging Array (PMA)

5/3/2014 12

Light Detection

LED

Ph

ase

Mo

du

lati

on

Data Processing

Intuitive Solution

5/3/2014 13

LED

Light Detection

LED

Ph

ase

M

od

ula

tio

n

Data Processing

Time-of-flight Camera

Our Solution

5/3/2014 14

Light Detection

LED

Ph

ase

M

od

ula

tio

n

Data Processing

Photo Detector

LED

Phase Modulation

• Phase Shifting

5/3/2014 15

Light Detection

LED

Ph

ase

M

od

ula

tio

n

Data Processing

185o 280o 250o 320o

Phase Modulation

• Messaging and Signal Selection

5/3/2014 16

Light Detection

LED

Ph

ase

M

od

ula

tio

n

Data Processing

Phase Modulation

• Message Recovery

5/3/2014 17

Light Detection

LED

Ph

ase

M

od

ula

tio

n

Data Processing

p2 p0 p1 p3 p2 p1 p0 p3 p1

10 00 01 11 10 01 00 11 01

Experiments

5/3/2014 18

PMA SoftKinetic

DS311

Five independent experiments 8000 random bits/experiment Average accuracy 97.8%

Experiments

• Data rate analysis

– 9 LEDs

• One for synchronization indicator

• Another 8 for messaging

– 4 candidate phases => 2 bits/LED

– ToF camera

• 60 fps => 8x2x60 = 960bps

5/3/2014 19

Applications

• Dual use of cameras

– Imaging (original use)

– Communication (added channel)

• Applications

– Object recognition: ID beacon

– Navigation: path marker

– Robotics: line of sight communication

5/3/2014 20

Conclusions

• Propose a camera-display communication methodology based on a low-cost continuous-wave ToF camera

• Implement Phase Messaging Array (PMA)

• Demonstrate high messaging accuracy in experiments

5/3/2014 21

5/3/2014 22