the input modality problemgrad.csit.carleton.ca/seminar/2016f - lucas stephenson - input...
TRANSCRIPT
![Page 1: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/1.jpg)
THE INPUT MODALITY PROBLEMLucas Stephenson
![Page 2: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/2.jpg)
Finding the Problem
• Initial Research Focus• Enabling rehabilitative motions to
be used as computer input
• Hundreds of Research Projects• Motion Recognition
• Game(s)
• Reporting
• Thousands of variations of input modalities
• Each Project “starts from scratch”
• Validation and Reproduction Problems
![Page 3: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/3.jpg)
The Problem Re-Imagined
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
![Page 4: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/4.jpg)
Platform for Transforming Input
• Remove redundant development of “modules”• Development effort
• Consistency
• Flexibility
• Transform
Reporting
Applications
Intelligent Algorithms
System Input Mapping
Gesture Recognition
Sensor
![Page 5: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/5.jpg)
Aggregated Input Modality Engine• AIME…
• Tool to map different modules together
• Flexible/Extensible data types
• 3 Main Layers/Module (Conceptual Categories)• Input
• Processing
• Output
![Page 6: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/6.jpg)
Input
• Sensors!
• Gather external data
• Attached or external source
• Use existing data types• Button Press
• 1d,2d,3d Vectors
• Text
• …or define new data types
![Page 7: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/7.jpg)
Processing
• Gestures
• Range
• Learning systems (AI)
• Conditional switching
![Page 8: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/8.jpg)
Output
• Emulate system’s standard inputs• Keyboard/Mouse
• Report• Network
• Files
![Page 9: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/9.jpg)
Project Status
• C# Engine• External Modules
• Graphs are hard coded
• Several modules of each type, except conditional
• Allows piping of typed data across modules
• Working “graphs” for leg lifts, coin flips, accelerometer mouse control
• Would like to have:• UI for designing and manipulating graphs
• Module browser/repository
![Page 10: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/10.jpg)
Review: What Does it Do?
• Allows developers to implement smaller components and re-use other’s work, more easily1. Focus on new gesture recognition technique
• Use existing sensor modules
• Applications
2. Focus on new exer-game?
• Can use existing modules..
• Sensors
• Gesture recognition
• Output (game input)
![Page 11: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/11.jpg)
Research
• I plan to study the effects on users having additional computer control.• Can we increase the incidence of beneficial activities?
• Does allowing the user to specify what actions their physical based gestures apply to the computer impact the frequency of these activities?
• Study: Beneficial physical activity in office environment• 3 Groups
• Just ask them to do leg lifts (control)
• Leg lifts perform fixed action, (ALT-TAB)
• Leg lifts perform customizable discrete action
![Page 12: The Input Modality Problemgrad.csit.carleton.ca/seminar/2016F - Lucas Stephenson - Input Modality... · Finding the Problem •Initial Research Focus •Enabling rehabilitative motions](https://reader033.vdocument.in/reader033/viewer/2022041802/5e51d5a6d8395333c864200d/html5/thumbnails/12.jpg)
THANK [email protected]