mitsubishi electric research labs (merl) cambridge, ma

Mitsubishi Electric Research Laboratories August 2006

Mitsubishi Electric Research Labs (MERL)Cambridge, MA

Instant Replay: Inexpensive Instant Replay: Inexpensive High Speed Motion CaptureHigh Speed Motion Capture

Hideaki Nii, Jay Summet, Yong Zhao, Jonathan Westhues, Paul H. Dietz, Shree K. Nayar, John Barnwell, Michael Noland, Vlad Branzoi, Erich Bruns, Masahiko Inami, & Ramesh Raskar


Demo VideoDemo Video


Optical CaptureOptical Capture

• Goal– Find 2D or 3D location of tags

• Approach– Each tag has a photosensor– Label space using zones created by special

high speed projectors– Find zone in which the tag lies


Key FeaturesKey Features

• 500 Hz Tracking• Id for each Marker Tag• Capture in Natural Environment

– Visually imperceptible tags– Photosensing Tag can be hidden under clothes– Ambient lighting is ok

• Unlimited Number of Tags Allowed• Base station and tags only a few 10’s $


Potential ApplicationsPotential Applications

• High Speed Motion Capture & Tracking– Motion Capture

• Character Animation• CG alignment

– Industrial applications• Crash Test Dummies• Robot Guidance

– Indoor Positioning for Ubiquitous Computing• Locating users and devices• 3D User Input


Existing SolutionsExisting Solutions

• Optical: Uses expensive cameras to locate tags in the environment.– Passive tags give location, but no identity,

leading to reacquisition problems• Magnetic

– Environmental interference, expensive• Inertial Tracking (Gyro/ Compass)

– Drift, Calibration


Solution OverviewSolution Overview• Use inexpensive solid state projectors

to “label space”.– Solid State lighting (LED’s) can be

modulated extremely quickly (MHz to GHz).• Each point in space receives a different

code sequence.• Inexpensive and small tags receive the

codes, and can detect their own location.


Labeling SpaceLabeling Space

Each tag receives a

unique signal based upon it’s

physical location.

P rojec tor

T ags

P os =0

P os =255


How Labeling WorksHow Labeling WorksLightsourc

e

GrayCode Mask

Optics

Screen

Light source blink one by one and each position on the screen has different light pattern.4 light make 4 bit position resolution

pos=0 pos=15


System HardwareSystem Hardware

• One projector can send 10 bit one dimensional position data.

• No limitation on the number of tags.

PIC16F873+LED driver

IR LE D IR rece ive rT S O P 70 00

SY N C IN

S Y N C O U T

P IC 18F26 20

M em ory

W irelesscom m uni

ca tion

P ro jecto r Tag

455kHzm odulatedIR signal


Inside of ProjectorInside of Projector

The Gray code pattern

Focusing Optics

Gray code Slide

Condensing Optics Light Source


Inside of TagInside of Tag


SpecificationSpecificationName Value RemarksMeasurement time

1 mS Per one dimension

FOV 26.7 deg.Resolution 10 bitOperational Range

1m ~ 5m Good quality in 2m~4m area.


Low costLow cost

Proto type system Mass Produced System•Projector

Electrical parts:$50Projector body:$10Optics: $ hundreds(4 custom glass lenses)•TagElectrical parts:$50Wireless unit:$50

•ProjectorElectrical parts:$30Projector body:$10Optics:$10 (Plastic lens)•TagElectrical parts:$10Wireless unit: Optional


Expand to 2D/3D SystemExpand to 2D/3D System

2D measurement 3D measurement

Y data

X data

Y data

X dataX2 data


Result VideosResult Videos


Tags Hidden Under CostumeTags Hidden Under Costume

• Tags can be hidden under an actors costume with only small sensors exposed.• This enables motion capture during primary filming.


High Speed TrackingHigh Speed Tracking


Flying Tags!Flying Tags!


E-Tech DemoE-Tech Demo


Key FeaturesKey Features

• 500 Hz Tracking• Id for each Marker Tag• Capture in Natural Environment

– Visually imperceptible tags– Photosensing Tag can be hidden under clothes– Ambient lighting is ok

• Unlimited Number of Tags Allowed• Base station and tags only a few 10’s $


Take Away PointsTake Away Points• Very Inexpensive Projectors and Tags.

• Reasonable accuracy which will be improved with better optical design.

• Very fast sampling rates possible!

• We want to license the technology!


Thank you!Thank you!

Contact:

Ramesh Raskar [email protected]


New Smaller TagNew Smaller Tag

mitsubishi electric research labs (merl) cambridge, ma

Documents