evaluation of 3d gesture interfaces
DESCRIPTION
Interacting with computational devices was limited to using a keyboard and mouse for many years. More recently, touchscreens have become popular, especially in mobile computing. However, accurate three dimensional gestures are still difficult to recognize. One approach is the use of depth cameras, which are currently receiving much attention again in academic and in commercial fields due to the release of inexpensive consumer electronics like Microsoft’s Kinect in 2010. This presentation reflects depth camera technologies, and how gestures can be used for new interface designs. The first part establishes a framework to evaluate the suitability of depth cameras for gesture recognition. The second part focuses on interactions in three dimensional virtual spaces. I developed a gesture interface for an existing application to navigate in virtual spaces and to construct LEGO models. A user study was performed to compare the gesture input with traditional mouse and keyboard input. Finally, I discussed the influences of the findings for future research in gesture interfaces.TRANSCRIPT
hand gesture
depth image sequence
hand 1: U movehand 2: show 5
position tracking
hand gesture
depth image sequence
gesture description
gesture recognition
Experiment: Gestures based interfaces Which gestures are suitable to control a computer?
Conclusion for gesture interfaces Do gestures in"uence future interactions?
Agenda
Range-imaging technologies How can a depth map be computed?
Range-imaging technologies
Range-imaging for gesture interfaces § What are suitable devices for gesture interfaces
to be widely used?
a#ordable
Range-imaging for gesture interfaces § What are suitable devices for gesture interfaces
to be widely used?
a#ordable compact
Range-imaging for gesture interfaces § What are suitable devices for gesture interfaces
to be widely used?
a#ordable compact mobile
Range-imaging for gesture interfaces § What are suitable devices for gesture interfaces
to be widely used?
a#ordable compact mobile markerless
Range-imaging for gesture interfaces § What are suitable devices for gesture interfaces
to be widely used?
Range-imaging technologies
Structured light Time of "ight Amplitude modulation
Range-imaging technologies
Structured light Time of "ight Amplitude modulation
Range-imaging technologies
Structured light Time of "ight Amplitude modulation
Triangulation Geometric distance computation
Structured Light Parallel light sources
Time of "ight Light distance di#erences
Amplitude modulation Sending light waves coherent light source signal amplitude modulated light
Comparison Suitable depth cameras Range Accuracy Image resolution
Structured light
Time of "ight
Amplitude modulation <
<
<
Structured light
Time of "ight
Amplitude modulation
Time of "ight
Amplitude modulation
Structured light
Comparison Suitable depth cameras § Depth resolution and range
§ Update rate and latency
§ Robustness
§ Spatial resolution of depth image
0.01mm
Experiment: Gesture based interfaces
Semiotic gestures Navigation
ZoomingRotation
Selecting Dragging
Dropping Flipping
Pseudo-ergotic gestures Manipulation
Results Times for traditional and gesture input
0
5
10
15
20
25
30
35
Mouse Gesture
time [min]
Results Further results from questionnaires § Gesture input is more fun, but less e$cient and
accurate
§ Correlation between participants who never played
physical LEGO and long construction times for gesture interface
§ Semiotic gestures are easy to use
§ Most participants prefer a combination of gesture input
together with traditional input
Conclusion for gesture interfaces
Navigation Accuracy and precision § Imprecise actions can be controlled by gestures
§ Gestures might support creativity
Manipulation Accuracy and precision § Gesture input cannot replace precise mouse and
keyboard input
Conclusion Gesture Interpretation § Most current gesture interfaces are built for controlling
the computer; however, gestures are a subconscious expression of human feelings
§ Experience of haptic feedback in ergotic physical
gestures is helpful for simulating the haptic feedback in related pseudo-ergotic gestures
Questions?
http://www.vriendinnen.be/user%les/images/maatschappij/
2011/april2011/interactiefwinkelraam.jpg
http://static.giantbomb.com/uploads/original/
8/86760/1804618-image_kinect_fun_lab_15890_2316_0007.jpg
https://www.leapmotion.com/media/leap_motion_media.zip
http://media.salon.com/2013/01/gadget-show-
intel.jpeg1-1280x960.jpg
Photos
http://www.misure.unisa.it/uploads/2814/scanner_struct.jpg
http://miac.unibas.ch/PMI/04-BasicsOfUltrasound-media/%gs/
time-of-"ight.png
https://www.leapmotion.com/media/leap_motion_media.zip
http://images.businessweek.com/ss/
06/11/1129_makingof_lego/image/intro.jpg
Photos