Separating Gesture Detection and Application ControlConcerns with a Multimodal Architecture

Luís Fernandes et al.INESC TEC & UTAD - PORTUGAL

Problem“Most gestures are neither natural nor easy to learn or remember. (…) Even the simple headshake is puzzling when cultures intermix” D.A. Norman (2010)

Devices

Leap Motion Kinect 2

InMERSE Multimodal Architecture

InMERSE Multimodal Architecture - Data

Implementation

Reproduce this gestureto continue

Prototype

Final thoughtsResults are very preliminaryOpen issues and topics for reflection:

Access to low-level dataDiverse set of data formats

Bringing User Experience empirical data to gesture-control and somatic interaction in virtual reality videogames: an Exploratory Study with a multimodal interaction prototype

google it in December!

Subsequent paper – accepted for publication this November

Thank you! Any questions? Luís Fernandes – INESC TEC & UTAD lfernandes@utad.pt