Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa

Ishikawa-Namiki-LaboratoryParallel Processing for Sensory information

University of Tokyo

• remove the need for any physical I/O space at all

• remove the need of any additional input device

…Next step ?

UI : “WIMP” + alphanumeric inputI/O Device: touch sensitive

screen+stylus

Today’s approach

nice: input space is merged with viewing space

I. The problem:

physical interaction

Input and interaction with

“ever-shrinking” portable

electronic devices

(*) windows, icons, menus and pointer

”ergonomics-independent” miniaturization

How to remove the input space?

In a word: passive-vision input versatile but not mature yet…

…in any case overkill for the problem at hand! (“inputspace-less” WIMP)

• May require intensive computation • Not robust enough• Sensitive to external illumination conditions/backgrounds...

drawbacks:

(non-invasive techniques!)

...By providing the PDA with intelligent sight:

- handwritten character recognition- gestures recognition, etc

Low, medium or high-level vision

- hand/finger position

Closer look at the problem...

-Hand/finger closer than anything else-fingertip always visible (no occlusion)-fingertip “shape” simple and stable

animal antenna

proximity sensor...

+ ...angular information

should provide sufficient data for

tracking!

cane for the blind

what a short-sighted PDA would actually

see…

cost-efficient, ROBUST, FAST!

…but cumbersome, annoying?

- laser can be used as output device (on any external surface, including the retina: VRD)

And...

The proposed “antenna”:

a directive beam of light

advantages:

- active lighting: provides and control illumination

- real-time, precise 3D measures (telemetry)

- measurement of surface roughness, speed, even biometric data.

Inconvenients: no haptic feedback (just like passive vision, but visual cues possible).

... but how realistic is the integration of a LASER RADAR on a PDA?

VERY! Thanks to today’s “MOEMS” technology...

The realm of “MOEMS” (micro-opto-electro-mechanical systems):

Electronic circuit, photodetectors, laser diodes sources and mirrors integrated in the SAME chip.

II. Smart Laser Scanner Prototype

•laser diode,

•a pair of steering micro-mirrors,

•single non-imaging photodetector.

Hardware (discrete components):

circular laser “saccade”

Tracking Principle

smart laser scanner:

laser excursion is locked around the area of interest

Tracking sequence repeated every

millisecond.

kHz refreshing rate explains algorithmic extreme simplicity

Prototype Smart Scanner in Action

2 DOF (using fingertip)

(angular precision: better than tenth of a degree)

3 DOF (using fingertip)

Depth precision: (intensity based) around 1cm up to 2 m

a little farther away …

Simultaneous tracking

(up to four targets without hardware duplication)

Several users…

… unique user, higher DOF

Achieving 6 DOF (using three tracking points)

Fast 3d tracking of a ping-pong ball

Interactive laser annotation

UIs based on this 3D Input Device:Proof-of-principle examples

X/Y/Z translation

RIGTH CLICK

LEFT CLICK

Gestural command:

3D locator

X/Y translation

ZOOM

(-)

Interface for visualizing 2D data

LEFT CLICK

RIGHT CLICK

Gestural command:

Positional command:

ZOOM

(+)

X/Y translation

Fram

e

positio

n (-)

Zoom x2 Zoom /2

Gestural command:

Positional command:

Fram

e

positio

n

(+)

Visualization interface for video

Conclusion

The problem: design a 3D input device that does not interfere with technological miniaturization

Proposed solution: use laser “light antennas” to track one (or more) fingers.

Advantages:

•fast (no image processing),

•precise 3D measurements,

•insensitive to external light conditions,

•no markers necessary,

•can be used as output device.

Further Work

Study of adaptive saccade shapes (ex: follows the hand contour)

Prototype without mobile parts, using multiple laser beams...

MOEMS integration: PDA on a keychain, PDA on a chest-pin: touching the pin or activates tracking, a gesture can be used to ask for emails, and these are read on the hand...

Precision (cm)

Speed (3m/s)

multi-tracking (up to four fingers)

Interactive laser annotation

see more at both #102

ACR - 5.4.2005