Low-Cost Multi-Touch Sensing through Frustrated Total

Internal Reflection

Low-Cost Multi-Touch Sensing through Frustrated Total

Internal ReflectionJefferson Y. Han,New York UniversityJefferson Y. Han,New York University

Presented by: Cody Boisclair

Presented by: Cody Boisclair

IntroductionIntroduction Touch sensitivity is fairly common in electronics today…

…but only for a single point of contact at any given time.

Touch-sensitive devices that allow for multiple points of contact have thus far been expensive or difficult to produce.

Why multi-touch?Why multi-touch? Allows a user to interact with a system with more than one finger at onceChordingOperations using both hands

Also allows multiple users to interact on the same touch-sensitive platformInteractive wallsInteractive tabletops

Prior approaches to the problemPrior approaches to the problem Matrix of smaller sensors

Requires many connections, which severely limits resolution

Visually opaque - no touch-screen

Video camerasMeasuring of brightnessStereo camerasIdentifying tracking markers in gel

A new approach…A new approach…

FTIR

rustratedotalnternaleflection

Total Internal ReflectionTotal Internal Reflection When light encounters a medium with a lower index of refraction (e.g., going from glass to air), its refraction depends on the angle at which it hits the border.

Beyond a certain critical angle, light is not refracted, but instead reflects entirely within the material.This is the basis for fiber optics and other optical wave guides.

Frustrated Total Internal ReflectionFrustrated Total Internal Reflection If another material touches that within which the light is reflecting, the reflection is frustrated, causing the light to escape.

This has been used in the past:fingerprint imagingearly touch sensors (1970s!)tactile sensors for robotic grips

A Schematic of FTIRA Schematic of FTIR

(Kasday, 1984)

Using FTIR for touch sensitivityUsing FTIR for touch sensitivity A clear acrylic sheet is used as the touch surface.16 inches x 12 inches in prototype

Edges of surface lit by infra-red LEDs to produce total internal reflection.

A video camera is mounted under the surface and facing it.

When the surface is touched, the light escapes and registers on the camera.

Using FTIR for touch sensitivityUsing FTIR for touch sensitivity Basic image-processing techniques are performed on the camera output to identify the points of contact.

Computer-vision techniques are used to interpret the motion of contact points as discrete touches or strokes.

Processing easily handled in real-time by a 2 GHz Pentium IV processor

Advantages to this approachAdvantages to this approach High capture rate and resolution

30 frames per second640x480

True zero-force touch sensitivity

Inexpensive to construct Scalable to much larger (even wall-sized!) surfaces

Transparent: can be combined with rear-projection display

How is projection achieved?How is projection achieved?

DisadvantagesDisadvantages Requires significant space behind touch surface for camera

Gloves, certain types of styluses, and even dry skin may not registera function of refractive index

Residues on surface (e.g., sweat) also produce FTIR effect that may build up

Some applications…Some applications…

Video Clip at NYU (Local Copy)

Uses a 36x27-inch rear-projection screen.

Touch information is sent to programs using OSC (Open Sound Control) protocol

Any Questions?Any Questions?