Eric ShangKuan (ericsk)

Technical Evangelist Microsoft Taiwan Corporation

Agenda

• Kinect for Windows

• PyKinect: How-to

• References

Kinect for Windows (K4W)

Kinect for Windows

How Kinect Works

Skeleton Tracking

Detailed Spec

Resolution 1280 x 960 (RGB Camera)

Viewing angle 43° vertical by 57° horizontal field of view

Tilt range ±27°

Frame rate 30fps

Audio format 16KHz, 24-bit PCM

Audio input A four-microphone array with 24-bit analog-to-digital converter (ADC) and Kinect-resident signal processing including acoustic echo cancellation and noise suppression

# of tracking users 4

# of tracking points 20

Different from XBox360 Kinect

• Near mode

– Enables the camera to see objects as close as

40 centimeters in front of the device without

losing accuracy or precision, with graceful

degradation out to 3 meters.

• Shortening USB cable and small dongle

• Support and software updates

Kinect Software Development

• Windows 7/8 RP

• Visual Studio 2010

• C++ or C# (w/ .NET 4.0)

• Kinect for Windows SDK (1.5)

• (optional) DirectX

• (optional) Microsoft Speech Platform Runtime

• (optional) Kinect for Windows Language Pack

K4W HW/SW Interaction

K4W SDK Architecture

1. Kinect hardware 2. Kinect drivers 3. Audio and Video Components 4. DirectX Media Object (DMO) for microphone array beam-forming and audio source

localization. 5. Win32 standard APIs

PyKinect

PyKinect Project

• A Python bindings of K4W SDK.

– K4W SDK is required.

• A part of Python Tools for Visual Studio (PTVS)

open source project.

• http://pytools.codeplex.com/wikipage?title=PyKi

nect

Enable PyKinect

• CPython 2.7 (32-bit)

• Kinect for Windows SDK (1.5 is OK)

• 2 ways to install PyKinect

– Install PTVS and PTVS – PyKinect Sample , then install PyKinect

through PTVS menu.

– Install through PyPI: http://pypi.python.org/pypi/pykinect/1.0

• (optional) PyGame

How to use PyKinect?

from pykinect import nui

# Create a kinect instance

kinect = nui.Runtime()

# Enable the skeleton engine

kinect.skeleton_engine.enabled = True

# skeleton frame ready event handling

kinect.skeleton_frame_ready += skeleton_frame_ready

# depth frame ready event handling

kinect.depth_frame_ready += depth_frame_ready

# video frame ready event handling

kinect.video_frame_ready += video_frame_ready

Skeleton Tracking

• Each joint (關節) is represented by 3-axis

coordination (x, y, z)

– RH axis

• Use JointId to ensure where the joint is.

Get Skeleton Data

def skeleton_frame_ready(skeleton_frame): skeletons = skeleton_frame.SkeletonData # index for users for index, data in enumerate(skeletons): # get head position head_positions = data.SkeletonPositions[JointId.Head] # handle limbs - left arm LEFT_ARM = (JointId.ShoulderCenter, JointId.ShoulderLeft, JointId.ElbowLeft, JointId.WristLeft, JointId.HandLeft) left_arm_positions = [data.SkeletonPositions[x] for x in LEFT_ARM]

Demo – Draw Skeletons

References

• Kinect for Windows SDK Programming Guide &

References:

http://msdn.microsoft.com/en-us/library/hh855347

• Check-out the PyKinect source:

http://pytools.codeplex.com/SourceControl/list/changes

ets