SAVI

Andrew EbaughSaurav Chatterjee

Shopping assistant for the visually impaired

Basic concept of SAVI

What does it do? SAVI allows blind people to locate and identify

items via RFID technology.

What is it good for? Blind people can shop by themselves. Facilitates in identifying items once they get

home. It can substitute for Braille signs and tags.

Currently, blind individuals cannot navigate a grocery store independently.

• Have to depend on others to help with shopping (Braille stickers on products)

• Little choice on prices and brand of items being purchased

• Have to depend on others when checking out at the register

• Organizing their items once they return home is a problem

Why is SAVI needed?

Product Demo

We will display three active pieces: Personal Server, ear piece, & scanning glove

Simple demo idea: Identify common grocery items with our system without using sight.

Plan to set up a display of several tagged items on shelves Generate a grocery list from among the several possible items Locate the correct items by handling the objects and scanning

the tags As our audio board also transmits over FM, we’ll have a

small stereo set up to broadcast what the device user will be hearing

Product Demo, cont.

Our original scenario was pretty close to this it was for an actual user in a grocery store It showed a user receiving audio feedback at the

register, which we will not demo It also showed the user putting away groceries at

home, which we will not demo

Implementation – Overall diagram

mot

e

Radio

JDBC

ODBSSQL

Audio

dB

MP3

PersonalServer

RFID

Linux

Processing

Input

Output

Implementation, cont.

mot

e

Radio

ODBS

PersonalServer

RFID

Linux

• RFID scanner – mote running TinyOS transmits RFID tag scan to a mote in a CF card on personal server

• CF card mote – accessible as a serial port on personal server, /dev/tts/3

• Serial forwarder service – dumps whatever is received on the serial port to listeners on a network port (4444 TCP)

• ODBS listener service – Connects to network port, and registers as a listening client, accepting forwarded packets.

Implementation, cont.

JDBC

ODBSSQL

dB

PersonalServer

Linux

• ODBS – Multi-threaded: port listening thread, database query thread, output thread

• Product information is stored in SQL database

• Upon receiving a RFID scan, the service queries the product database

• Additionally queries for brand, price, sale information

• Each piece of information returns an associated audio clip; from all of these, a presentation “playlist” is generated

Implementation, cont.

• Database is provided by sqlite and accessed by sqlite JDBC-connector

• Contains 7 tables: product, brand, measure, location, sale, alphanum

dB

Product table pseudo-SQL :

CREATE TABLE product (rfid, upc, brand_id, name, price, price_unit_id, price_quantity, sale_type, sale_quantity, sale_price, quantity, quantity_unit_id, location_id, section_id, clip)

Ex:INSERT INTO product VALUES ('E00781BCC1957A75', '04482008', 9, 'Oreo Cookies', '3.69', NULL, NULL, 2, NULL, NULL, '1.13', 0, 9, 18, 'product-11');

Implementation, cont.

• Presentation clip – Once the clip has been generated, ODBS makes a call to Runtime.exec() to execute an mp3 audio player

• Execution call takes the form of:madplay /mp3/brand-04.mp3 /mp3/product-02.mp3 /mp3/alphanum-39.mp3

• MP3 player talks to audio board as /dev/dsp

ODBS Run

time

Audio

MP3

PersonalServer

Linux

Implementation, cont.

• Slappy board – Audio is provided by board that plugs into personal server

• Board drivers are modular and with dependent modules (soundcore, ac97_codec) provide standard linux audio interface to be accessed by applications

• In addition to standard stereo output and line input, provides FM transmission on configureable frequency.

• All this together gives a range of options, from enhanced ODBS access of audio board, to a multitude of output devices

Audio

Implementation, cont.

What is done? All the pieces described so far are working

together System works end-to-end, from tag scan to audio

What isn’t? Mainly finishing things up and work on aesthetics Glove and personal server aren’t currently

something one would want to wear Last minute hardware glitches

Evaluation

How well does it work? Once the kinks are worked out, the system works

pretty well. The key metrics: Distance glove needs to be from tag Length of time glove needs to be reading tag Delay between successful scan and start of audio

(i.e. processing time) Length and relevance of audio clip

Evaluation, cont. – metrics

Distance glove needs to be from tag Depends on how charged the batteries are, and how

sensitive the antenna is For our glove reader with a Mylar antenna in palm and

full batteries, the distance is 3-4 inches. Length of time glove needs to be reading tag

We have toyed with different reading frequencies to tune this metric

Currently check for a tag 4 times per second. At this speed, the reader can detect a tag when moved by it at an acceptable speed (i.e. as if you were feeling around the object)

Evaluation, cont. – metrics Delay between successful scan and start of audio

This was our main concern, that doing the processing and database queries would take time

Actually, this time is minimal From successful scan, audio will start in under a second The longest portion of this is starting up the mp3 player,

suggesting a possible point for improvement Length and relevance of audio clip

Probably needs the most work The overall clip is built on subclips. Currently output is:

brand, name, price, any sale information It would be best if this was configurable, because

playing this information can take some time

Future Work

Lots to be done, lots of possibilities Database contains a lot of information not currently used:

UPC code, location, quantity There should really be an audio output component always

loaded, rather than making an external call The product database could be a central store database

accessed over wi-fi or similar Could integrate a shopping list or store navigation aid Would like to have a scanner at the checkout register that

relates product name, price, subtotal as rung up The glove we have is definitely a prototype: should have a

lightweight, chargeable battery, and several buttons Possible text-to-speech for audio generation, and linking with

existing UPC-based product databases