Capturing, Encoding and Rendering Gestures using

the Kinect

Adan PatienceSupervisor: Mr. Mehrdad Ghaziasgar

Co-Supervisor: Mr. Reginald McDonald DoddsMentor: Mr. Kenzo Abrahams

Overview

▪ Recap

▪ Design Decisions and System Changes

▪ Interface

▪ Implementation

▪ Tools Used

▪ Project Plan

Recap Introduction

▪ Many countries lack networking infrastructures which support video chatting applications such as Skype.

▪ Develop towards a video chatting application using the Kinect, that could possibly be used extensively by the deaf in the future.

Recap Work Flow

Recap High Level Design

Input

Video Frames

CaptureLocate

and Track User

Encode

Get Joint Coordinat

es

Write Coordinate

s to File

RenderGet

Frame and Read Coordinat

es

Inert Key Frame

Animate

Design Decisions and System Changes

Previously

▪ No Configuration File

▪ Append Coordinates to tracking file

▪ Used line number in tracking file to determine frame number

▪ Key frame every 24th set of coordinates

▪ No real-time

Currently

▪ Create Configuration file

▪ Overwrite tracking file containing coordinate points

▪ Create new file to store the frame count

▪ Key frame every 10th set of coordinates

▪ real-time key framing

User Interface

▪ Setting foundation for the application

▪ Application not at a mature enough state for a user interface

Implementation

Input

Update Skeleton

CaptureProcess Skeleton

Draw Skeleton

Encode

Get Joint Coordinates and Frame

Count

Write Coordinates to File and

Frame Count

Render

Configure User and

Get Frame Count

Get Coordinat

es

Map Coordinat

es

Insert Key Frame

Animate

Configure User

▪ Assume 3 Positions for the Configuration File– Rest Arms– Arms Up– Spread Arms

▪ Coordinates are stored in CONF.csv– Average of the centre shoulder position is

calculated– Minimum and Maximum of both x and y

coordinates are determined for both hands.

centre shoulder

Minimumy-position

Maximum x-position

Maximum y-position

Minimum x-position

Map Coordinates

▪ Right Wrist

▪ Left Wrist

Left Wrist

Centre ShoulderRight Wrist

-6.5 UnitsOrigin

6.5 Units

Tools Used

▪ Platform– Windows 7 x86

▪ Applications– Blender 2.49b– Visual Studio 2013– Kinect SDK

▪ Languages– Python 2.6– C++

Project Plan

Term1

• Identify User Requirements, Analyse User Requirements, Literature Survey

• Become Acquainted with OpenCV and C++

Term2

• Identify Objects and Classes in the OOD and analyse the Class’ methods and attributes, Design UI and Choose the Render Application

• Familiarise myself with the Kinect SDK and API

Term3

• Create the Capture and Encode Classes leveraging from the Kinect• Develop the mapping function. Create UI and finalise application

Term4

• Test the application against the User Requirements• Optimise where possible and iron out bugs