Building a Prototype of Tactile Vision Substitution System with High-Speed CMOS sensor

26July2006

By: Arthur S. Agdeppa Jr

PROPRIETARY

“We see with the brain, not the eyes”(Bach-y-Rita, 1972)

There are 45 million blind people worldwide and growing.

The cure for blindness still doesn’t exist

Paul Bach-y-Rita and Kurt A. Kaczmarek invented the Tactile Vision Substitution System

PROPRIETARY

What is the Tactile Vision Substitution System (TVSS)?

A Sensory Substitution System.

Sends signal to one sense that is normally received by another

PROPRIETARY

Components of the TVSS?

CMOS active-pixel as the input device

Control box / circuit contains the receiver, multiplexer, clock, and switch.

Output device is an array of microelectrodes on flexible PCB

Input Control Circuit Output

PROPRIETARY

Output Resolution

Capable of performing complex perception

– facial recognition

– accurate judgment of speed and direction of a moving object.

High Resolution Output

CMOS sensor captures Edges detected Skin of

the image (Sketch) Blind Person

Brain

PROPRIETARY

Objectives of Research To explore the potential of a TVSS

– By building and testing a prototype of the device using high-speed CMOS

Collision avoidance for blind people

– By constantly sampling images at high rate (60 fps) and providing instant feedback

To provide real-time image sampling

– By reducing time lag between input and output device

PROPRIETARY

The Prototype Hardware

Input Device (COH-7200)

– CMOS sensor

– ½ “ monochrome

– 659 x 494

– 50 fps

– D-cam compliant 8-bit output

– IEEE-1394A output interface

PROPRIETARY

Prototype Hardware: Continuation

Logic (Control Circuit)

– Laptop Computer

– Linux Operating System (Open Source)

– Generic software

– C language

Output Device

– Light Emitting Diode Matrix

• Prevent Injuries

• Digital input

• Consists of 8 x 8 LED

• Serial Peripheral Interface

PROPRIETARY

Linux IEEE1394

libdc1394

Dependencies

Functions

Get/Query

Set

Print

Setup

Release

libraw1394 and kernel-2.6

PROPRIETARY

Output Pinout

Pin 1 – 5 V power source

Pin 2 – Data I/O

Pin 3 – Chip Enable

Pin 4 – Clock

Pin 5 – 0 V power source

PROPRIETARY

Challenges

Edge Detection Algorithm

– Canny edge detection algorithm

Formula to scale down image

– From 659 x 494 (input device) to 8 x 8 (output device)

Time

PROPRIETARY

Summary

Paul Bach-y-Rita and Kurt A. Kaczmarek invented TVSS since 1972

The prototype will have

– COHU-7200

– Linux box

– LED array

The prototype will be used to explore the potential of TVSS device

PROPRIETARY

My Future Agenda

Pursue Electronics and Computer Engineering Technology (ECET) in Maui Community College (MCC)

Be a part of Oceanit Ohana

PROPRIETARY

Acknowledgements

Oceanit

Curt Leonard

Leslie Bailey

Staff

CfAO

Malika Bell

Lisa Hunter

Hilary O’Bryan

MEDB

Isla Yap

MCC

Mark Hoffman

Wallette Pellegrino

PROPRIETARY

References

www.who.int (World Health Organization)

www.wicab.com (Official Website of Brainport)

www.linux.org (Official Website of Linux)

http://www.tldp.org/HOWTO/libdc1394-HOWTO/concepts.html (libdc1394)

…innovation through engineering and scientific excellence… since 1985