Getting started with Leap Motion AS3

Eugene Krevenets (aka Hyzhak)Lead Front End Developer of Realaxy

http://about.me/hyzhak

Time: 1m* Thanks for all you guys that comes to online conference. And also Serg for conferenceThanks Peter for so interesting presentation. shtif Stefcek. * Let me introduce myself before go deeper* Lead Front-End Developer of Realaxy team* Open-Source projects http://github.com/hyzhak/* lead Tech blog http://pressanykeytocreate.blogspot.com* http://about.me/hyzhak * QR-Code* I'll talk about LeapMotion and deep dive in some technical issues.* And if you already expert in LeapMotion I will show some architecture tricks to write down customize library for different 3D Engines

---

Lead Front-End Developer of Realaxy team

Also passionate about software developing, creating web services, mobile application. Have some open-source project on my http://github.com/hyzhak/ and tech blog http://pressanykeytocreate.blogspot.com. Always welcome to comments, pull requests and so on.All those information available on http://about.me/hyzhak just scan QR and you

Main Topics

About Leap Motion

APIs

WebSocket API vs Driver API

LeapMotionAS3

How it Works on Examples

time: 1m

* Focused on technical issues* We will have a Quick Glance on LeapMotion possibilities* Talk about API details* Overview deference between WebSocket API and Driver API* We will see how it implement this API for Action Script 3 - I talk about project LeapMotionAS3* Deep Dive in some Examples

About Leap Motion

Fingers Motion Sensor

Two cameras and three infrared LEDs

AirSpace

time: 4m + (2m)

* On the left side you can see my Instagram photo of LeapMotion that I have got as developer* Just device, wire and quick note about it* What is the LeapMotion?* Natural user interface (NUI)* Kinect - hole body, LeapMotion - fingers* Two tracking cameras and three infrared LEDs* Doesn't provide video information as Kinect - LeapMotion team have said that reason for it it need of additional permissions from user* Appstore to share desktop application AirSpace. It will launch soon maybe in a month, but I don't work in LeapMotion so I can't give you any precise instructions* Next we focused more on technical issues* If you scan this QR code you can open page from my tech blog where I've written down some additional information about LeapMotion and share some demo videos

API

Controller::onFrame

Hand (Palms)

PointableFinger

Tool

Gestures

Screens Calibration

time: 1.5m + (6m)

* Controller:onFrame We have some other events. But I focused more on this. It provide to use LeapMotion tracking information with some frequency, that doesn't correlate with EnterFrame event or something. I can't give you strict information about frequency It's depend on your LeapMotion settings and performance of your PC, but you must now it before translate tracking information right to screen - maybe you need add some buffer before show It on render even;* Hands - Palms;* Pointable (Finger, Tool)* Screen calibration information* Gestures

* let's look more closely to pointables

API. Pointables

Position, Velocity, Direction, Width, Length

Right-Handed coordinate system (like Alternativa3D)

* Position, Velocity, direction, width and length of pointable;* Right Handed coordinate system (like Alternativa3D) And if you use some other engines you need apply some additional transformation for inverting Z for example;* By the way for Alternative3D we also need rotation transformation because it's Z axis is directed to up. I'll show you in examples;

API. Gestures

Key tap

Circle

Swipe

Screen tap

* Gesture** Key tap;** Screen tap;** Circle;** Swipe;* I'll more focures on them in upcoming presentations

Web Socket API vs Driver API

Driver API (C++, C#, ObjectiveC, Java, Python)

WebSocket API for Web Application (AS3, JavaScript)

WebSocket's API doesn't have Screen API. No calibration information :(

time: 3.30m + (9m)

* Not like Kinect web application can track LeapMotion data without any additional drivers* Driver API for desktop application (C++, C#, ObjectiveC, Java, Python)* WebSocket API for web application (ActionScript3, Javascript everything that can run in browser)* Main difference now between WebSocket API and Driver API - it's a lack of Screen calibration information on WebSocketAPI* Leap Motion team have promised that them will fix this issues in upcoming SDK but still haven't;* Why we need calibration information?* If you ask user to interact with images on the screen you can't handle it, because you doesn't know where is screen situated in 3D environment;* here is some libraries in javascript that try to provide it's own calibration but I still think that don't good practice and we need one point for calibration not in any application;* so we are still missing calibration information;

Example. Raw Data

Connect to Socket

Handshake as WebSocket

Grab JSON

Sources:
https://github.com/hyzhak/leapmotion-3d-controller-as3/tree/master/src/examples/raw

time: 5m + (14m)* Connect to Socket;* Handshake as WebSocket;** http://en.wikipedia.org/wiki/WebSocket#WebSocket_protocol_handshake* Grab JSON;* LeapMotion AS3 have done this job for us

LeapMotionAS3

Sources: https://github.com/logotype/LeapMotionAS3

Wouter Verweirder (ASNUI, OpenCV, )
and Victor Norgren

Same API for AIR and web application

time: 1.30m + (12.30m)

* There two cool guys that have created excellent library for tracking LeapMotion data in actionscript* Sorry, for my pronunciation I afraid but I can't said them surname right* Wouter Verweirder (ASNUI about Kinect, OpenCV, ) and Victor Norgren* Same API for AIR and web application. Write ones - run everywhere!* There are very communicable and positive, so you can always connect with them* Huge respect for those guys;

Example. Drag'n'Drop

Build scene with Alternativa3D

Pool of Objects

Draw Pointables

Get Intersection Object3D with Pointables

Dragging

Sources: https://github.com/hyzhak/leapmotion-3d-controller-as3/tree/master/src/examples/drag-n-drop

### 11.1 Overrviewtime: 1.30m + (19m) overview* Here I will also talk about abstracting layer :)* We'are using Alternativa3D in our project Realaxy so I've choice it for example and control library, but you feel free to fork it and add any adapters for other 3D engines (like Flare3D, Away3D and so on) also I've be very glad for such initiative.* http://opengameart.org/ - all art in example was got from here.

### 11.2 Alternativa3D Scenetime: 2.30m + (20.30m) overview* Scene (Box and SkyBox);* Mouse Orbit Controller;* Can Customize for other 3D Engines;

### 11.3 Presentation of LeapMotion 3D Controllertime: 16m + (23.00m) overview* LeapMotion System;* Intersect system;* Gestures;* Fingers Visualization;* Drag'n'Drop Controller;

### 11.3 Demo time!time: 2m + (39.00m) overview* Let's eating Sushi!

Thanks

Sources: https://github.com/hyzhak/leapmotion-3d-controller-as3

Article:
part1: http://j.mp/10zccWx ,
part2: comes later

Questions?

Eugene Krevenets (aka Hyzhak)Lead Front End Developer of Realaxy

http://about.me/hyzhak