gesture recognition / sign language
DESCRIPTION
Gesture Recognition / Sign Language. Lukas Bloder Johannes Bannhofer SE09 MUS2 SS10. Overview. Hardware Sign Language Live Demo System Architecture System Tools Technologies Problems Fazit. Hardware. P5 Glove API: http://www.robotgroup.org/index.cgi/P5Glove. CyberGlove : - PowerPoint PPT PresentationTRANSCRIPT
GESTURE RECOGNITION / SIGN LANGUAGE
Lukas BloderJohannes BannhoferSE09 MUS2 SS10
Overview
- Hardware- Sign Language
- Live Demo
- System Architecture- System Tools - Technologies- Problems- Fazit
Hardware
P5 GloveAPI: http://www.robotgroup.org/index.cgi/P5Glove
CyberGlove:http://www.immersion.com
http://www.golem.de/0512/42086.html
MIT Color Glove Handtrackinghttp://people.csail.mit.edu/rywang/handtracking/
Hardware
CyberGlove :
Hardware
Number of sensors: 18 or 22Sensor Resolution: 0.5 degrees (typical)Sensor Data Rate: 90 records/sec minimum (100 records/sec typical).Operating system and hosts: Windows 2000 and XPOperating Range: 30 ft radius from USB portInterface: USB port for the wireless receiver
CyberGlove II:
Sign Language
American manual alphabet
Sign Language
Substitution signs
-Dynamic signs: J, Z
Additional SignsSpace, enter, delete, various commands
Demo Time!
System
C++ API (Partially from original source of 1998)JNI Bridge
Application:Exchangeable Processing (Matlab, weka)Rules (substitution signs, comamnds)Clients (Commandline, TTS, Graphical)
System Architecture
Classification using ANN
Matlabnntool
Classification using ANN
Matlab – Erros recognizing letters
Processing Rules
Rules to process more complex signsRecognition splitted to Wrist/FingersEvaluation with rules
System Tools
- Data Collector- Data Aggregator
Technologies Used
-C++ / Java-Matlab-MaryTTS
Problems
-Old API -Matlab /generating JAR Files-API license problems-Training data-Inconsistent sensor data
Fazit
-Old Hardware still does the job -Don’t touch machine generated code-Generating good training data -> hard work
THANKS FOR YOUR ATTENTION!
Lukas BloderJohannes BannhoferSE09 PEG SS10