developing for leap motion
DESCRIPTION
Developing for Leap Motion DotnetConf session here: http://www.youtube.com/watch?v=YixzSUxyGKU ( 1 hour) Video tutorial can be found here: Developing for Leap Motion in C# Part 1: http://www.youtube.com/watch?v=1Rn3q75mdns Developing for Leap Motion in C# Part 2: http://www.youtube.com/watch?v=-r_cAtHQzy8 GitHub repository for the Leap Motion demo app: https://github.com/IrisClasson/Leap-Motion/ Slides: http://www.slideshare.net/irisdanielaclasson/developing-for-leap-motion/TRANSCRIPT
![Page 1: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/1.jpg)
Taking the Leap with Leap Motion
Iris Classon
![Page 2: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/2.jpg)
About me
• Technical Evangelist at Telerik (WinRT)• Software Developer at Dotnet Mentor• Microsoft MVP C# and member of MEET• MCSD, MCPD, MCTS• Organizer Sweden Pluralsight Study Group (Gothenburg)
• Started learning programming from 0 summer 2011 • Licensed clinical Dietitian and Int.Lic. Personal Trainer
![Page 3: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/3.jpg)
Contact details
• www.irisclasson.com• Twitter• @IrisClasson
• Email• Telerik – [email protected]• Dotnet Mentor – [email protected]
![Page 4: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/4.jpg)
What is the Leap Motion?
![Page 5: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/5.jpg)
To get started
• Mac, Windows, Linux (latest OS – more or less)• Developer Unit of Leap Motion• Preorder or apply to get one• Purchase: release projected for 22nd of July• https://www.leapmotion.com/
• SDK • Can’t be shared
• Forum, samples and documentation available• Projects on GitHub
• Breaking changes introduced between SDK versions- might need to update the code
![Page 6: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/6.jpg)
Good to know
• Still in Beta• Developer units vs. consumer units:• The size and weight • Final hardware components & casing• Field of view approx 20% wider in consume version
![Page 7: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/7.jpg)
Do not forget
• Dispose Frame object when done with it• RemoveListener from Controller when app closes• Dispose Controller afterwards
• Add Leap.DLL, LeapCSHarp.DLL & LeapCSHarp.NET4.0.DLL• Set copy output to Copy Always• Reference the LeapCSHarp.NET4.0.DLL
![Page 8: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/8.jpg)
When in trouble w. the device
• USB-cable has been known to cause trouble• Change USB port and/or cable• Connect the Leap before running the executable• Make sure there is light and the device is clean• Calibrate screen• Use the Leap Vizualiser to check if it works
![Page 9: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/9.jpg)
The Leap story
• Small startup based in San Fransisco• Venture capital• Has grown from just a few people to 80 employees and still hiring• Asus and HP on board
• 40 000 Devs signed up• 12 000 dev units shipped so far• Still shipping
![Page 10: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/10.jpg)
Demo
![Page 11: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/11.jpg)
The Airspace Store
• Storefront to gather all the apps• Dev decides on pricing, distribution & platform supported• Non-exclusive distribution• Min price starts at jus below 1 USD• Industry standard at 70% to the developer• Apps manually tested before publishing
![Page 12: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/12.jpg)
Available languages
• C++• C#• Objective C• Java• Python• JavaScript• (Ruby => hack)
![Page 13: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/13.jpg)
How the API works
• Leap motion detects fingers/tools
• Data captured as frames continously
• Listener class handles events from the Frame class
• Controller is the connection between device and app
• To listen for the updates on the Frame data: controller.AddListener(listener)
• Gestures must be enabled through the controllercontroller.EnableGesture(Gesture.GestureType.TYPESWIPE)
![Page 14: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/14.jpg)
ID, Timestamp, Hands,Fingers, Tools,Gestures
Frame
Translation,RotationAxis,RotationAngle,Scalefactor
Motion factors
![Page 15: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/15.jpg)
Code – C#class SampleListener : Listener
public override void OnFrame(Controller controller) { // Get the most recent frame and report some basic information Frame frame = controller.Frame(); SafeWriteLine("Frame id: " + frame.Id + ", timestamp: " + frame.Timestamp + ", hands: " + frame.Hands.Count + ", fingers: " + frame.Fingers.Count + ", tools: " + frame.Tools.Count + ", gestures: " + frame.Gestures().Count);
if (!frame.Hands.Empty) { // Get the first hand Hand hand = frame.Hands[0];
// Check if the hand has any fingers FingerList fingers = hand.Fingers; if (!fingers.Empty) { // Calculate the hand's average finger tip position Vector avgPos = fingers.Aggregate(Vector.Zero, (current, finger) => current + finger.TipPosition); avgPos /= fingers.Count; SafeWriteLine("Hand has " + fingers.Count + " fingers, average finger tip position: " + avgPos); } ………
private static void Main(string[] args) {
var listener = new SampleListener(); var controller = new Controller(); controller.AddListener(listener);
// Remove the sample listener when done controller.RemoveListener(listener); controller.Dispose(); }
![Page 16: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/16.jpg)
Code – JavaScript // Display Frame object data var frameOutput = document.getElementById("frameData");
var frameString = "Frame ID: " + frame.id + "<br />" + "Timestamp: " + frame.timestamp + " µs<br />" + "Hands: " + frame.hands.length + "<br />" + "Fingers: " + frame.fingers.length + "<br />" + "Tools: " + frame.tools.length + "<br />" + "Gestures: " + frame.gestures.length + "<br />";
// Frame motion factors if (previousFrame) { var translation = frame.translation(previousFrame); frameString += "Translation: " + vectorToString(translation) + " mm <br />";
var rotationAxis = frame.rotationAxis(previousFrame); var rotationAngle = frame.rotationAngle(previousFrame); frameString += "Rotation axis: " + vectorToString(rotationAxis, 2) + "<br />"; frameString += "Rotation angle: " + rotationAngle.toFixed(2) + " radians<br />";
……
// Setup Leap loop with frame callback functionvar controllerOptions = {enableGestures: true};
Leap.loop(controllerOptions, function(frame) { // Body of callback function})
![Page 17: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/17.jpg)
Making sense of movement
![Page 18: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/18.jpg)
The screen
Screen class
- contains computer screen information
- Position, orientation & vector Leap/PC
- Information from OS
Can be invalid
Device should to be calibrated with the screen
![Page 19: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/19.jpg)
The tool
• Based on:• Length• Shape• Width
![Page 20: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/20.jpg)
The gestures
• Predifined gestures:• Circle• Swipe • Key Tap (downward movement)• Screen Tap (forward movement)
Turn, twist and tilt – and fist• Use Motion factors from frame class
• Translation,• RotationAxis,• RotationAngle,• Scalefactor
![Page 21: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/21.jpg)
UX guidelines
• Avoid complexity• Use what is considered natural• Think outside the box• Positive experience for the user• Feedback• No clutter• Desctructive action• Navigation and interaction – keep them apart
![Page 22: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/22.jpg)
Q and A
Infra red light safety concerns• IR occours naturally in the physical world• Widely used in a variety of devices• No health risk found with general exposure of IR
Using Leap as a mouse out of the box • Developer unit no, consumer unit yes
• CPU usage• Will be lower when released• New updates ’inactivates’ the device when not used, activated when it registers
motion
![Page 23: Developing for Leap Motion](https://reader033.vdocument.in/reader033/viewer/2022061300/54c879494a795964478b45fb/html5/thumbnails/23.jpg)
Thank youIris Classon