![Page 1: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/1.jpg)
Leveraging the Kinect SDK to Control a Remote Device
Akhil Acharya and Sean FreemermanSummer Ventures in Science and Mathematics
Appalachian State University
![Page 2: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/2.jpg)
Computing Today
![Page 4: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/4.jpg)
Objective
● Original research on novel interaction techniques
● Develop new methodologies to interface the Kinect Software Development Kit with an Infrared (IR) transmitter
● Investigate the technology used by the Kinect to accurately track body parts
● Determine the viability of the Kinect platform
![Page 5: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/5.jpg)
Tools
● Kinect○ Tracks movement
● IR Toy○ Sends IR signals
● WinLIRC software package○ "Middle man" between Kinect Application and IR toy
● 2 AirSwimmer Remote Controlled Balloons○ System scalable to any IR device
![Page 6: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/6.jpg)
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
![Page 7: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/7.jpg)
![Page 8: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/8.jpg)
Microsoft Kinect
● Announced: 2009● Released: 2010● Full body motion controller for Xbox 360● Scatters multiple IR beams
○ Readings at discrete points○ Saves on processing power
● Tracks depth by measuring depth of focus ● Kinect SDK allows developers to create
applications using Kinect
![Page 9: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/9.jpg)
Why Kinect?
● Novel form of human-computer interaction● Relatively cheap
○ $150 to get started● Hands-Free● 3D Capabilities
![Page 10: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/10.jpg)
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
![Page 11: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/11.jpg)
Application Design
● Built using example application "Skeleton-Basics WPF"
● Communicates with the Kinect
![Page 12: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/12.jpg)
Control System (Kinect)
● Relative distance measured○ Distance between right shoulder and right hand○ User doesn't need to stand in the center of the
Kinect's FOV● Radius of 0.2 units - "null space"
○ Better differentiate commands○ Space to rest hand without performing action
![Page 13: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/13.jpg)
SystemKinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
![Page 14: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/14.jpg)
Control System (WinLIRC)
● If X and Y values exceed "null space" boundaries, command is sent to WinLIRC○ Done every 30 Frames (1 second)
■ Prevents WinLIRC from being overloaded○ Commands
■ Left/Right (X Values)■ Up/Down (Y Values)
● All commands defined as bytes in AirSwimmers.cfg file.
![Page 15: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/15.jpg)
![Page 16: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/16.jpg)
AirSwimmer
● Two models○ Shark ("Bruce")○ Clownfish ("Nemo")
● Lightweight control system○ Microcontroller with IR receiver○ Weighted ballast○ Servo○ IR remote
![Page 17: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/17.jpg)
Challenges
● Translating information● Debugging● Documentation
![Page 18: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/18.jpg)
Results
● It works! ● Movement occurs in near real time● Caveats:
○ Balloon requires line of sight■ Limited movement capability
○ Remotes avoid this by having higher power LEDs
![Page 19: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/19.jpg)
Next Steps
● Higher emission IR transmitter● Full-on voice control
○ Partially implemented already● Replicate with other motion control devices
○ Second Generation Kinect (Late 2013)○ Leap Motion
● Untapped potential○ Controlling non-IR devices○ Potential to control any device
![Page 20: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/20.jpg)
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
![Page 21: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/21.jpg)
Kinect SDK Application
WinLIRC (Always Running)
Position Data
Pass command to WinLIRC
Serial information over USB
IR signals
![Page 22: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/22.jpg)
Kinect SDK Application
Position Data
![Page 23: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/23.jpg)
Lessons
● Value of documentation● Powerful Kinect API● Potential of Gesture-based computing
![Page 24: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/24.jpg)
Acknowledgements
● Mentors: Dr. Rahman Tashakkori, Mr. Luke Rice, Ms. Bahar Akhtar
● Appalachian State University● AirSwimmer and IR Toy donated by Dan
Thyer.● UNC-Charlotte and the State of North
Carolina Undergraduate Research Symposium
![Page 25: Leveraging the Kinect SDK to Control a Remote Device](https://reader034.vdocument.in/reader034/viewer/2022052504/554a2609b4c9051b578b48c2/html5/thumbnails/25.jpg)
Thank You