Controlling a Computer by hand

gestures

Supervisor: Yuchao Dai, Laurent Kneip, Hongdong Li

Student: Bohuai Jiang u4652552

2

Content

• Introduction

• Technique review

• Design and Implementation

• Experiment and Evaluation

• Conclusion and Future works

Introduction

• Overview

3

Pic from Multi-touch - Wikipedia, the free encyclopedia. 2014. Multi-

touch - Wikipedia, the free encyclopedia. [ONLINE] Available

at:http://en.wikipedia.org/wiki/Multi-

touch#mediaviewer/File:Multitouch_screen.svg. [Accessed 02

November 2014].

Introduction

• Overview

4

Pic from Multi-touch - Wikipedia, the free encyclopedia.

2014. Multi-touch - Wikipedia, the free encyclopedia. [ONLINE]

Available at:http://en.wikipedia.org/wiki/Multi-

touch#mediaviewer/File:Multitouch_screen.svg. [Accessed 02

November 2014].

Introduction

• Overview

5

Pics form Geekerific home, controlled by voice and hand

gestures - AGBeat. 2014.Geekerific home, controlled by

voice and hand gestures - AGBeat. [ONLINE] Available

at: http://agbeat.com/tech-news/geekerific-home-controlled-

voice-hand-gestures/. [Accessed 02 November 2014].

Introduction

• Objective– Build a robust and real time human hand pose estimation and

tracking system.

– Data collected form RGB-Depth Camera

– Mainly focused on :

6

hand Model construction Cost function computation optimization

3.How to use the RGB-D camera "Asus xtion pro" with ROS. - DASL MediaWiki.

2014. 3.How to use the RGB-D camera "Asus xtion pro" with ROS. - DASL

MediaWiki. [ONLINE] Available

at:http://dasl.mem.drexel.edu/wiki/index.php/3.How_to_use_the_RGB-

D_camera_%22Asus_xtion_pro%22_with_ROS.. [Accessed 02 November 2014].

Technique review

• Iteration Closet Point (ICP)

• Gradient descent

7

Technique review

• Iteration closest point (ICP)– Algorithm employed to minimize the difference

between two point clouds by iterations

8

Pic from Documentation - Point Cloud Library (PCL). 2014. Documentation - Point Cloud Library (PCL). [ONLINE] Available at:http://pointclouds.org/documentation/tutorials/interactive_icp.php. [Accessed 03 November 2014].

Technique review

9

Technique review

• Basic Iteration Closet Point

10

Technique review

• Iteration Closet Point in hand tracking and

optimization

– Point-model alignment task

– Require more complex ICP

– Gradient descent base ICP

11

Technique review

• Gradient descent

– Widely using for multivariable function

optimization.[1][2][3]

– Easy and simple to be understood

12

Gradient descent - Wikipedia, the free encyclopedia. 2014. Gradient descent - Wikipedia, the free encyclopedia. [ONLINE] Available at:http://en.wikipedia.org/wiki/Gradient_descent. [Accessed 03 November 2014].

Technique review

13

Image form Tuning the learning rate in Gradient Descent | Datumbox. 2014. Tuning the learning rate in Gradient Descent | Datumbox. [ONLINE] Available at:http://blog.datumbox.com/tuning-the-learning-rate-in-gradient-descent/. [Accessed 03 November 2014].

Technique review

14

Design and Implementation

15

Design and Implementation

• Model construction

• Cost function

• Gradient-base ICP

16

Design and Implementation

17

Design and Implementation

• Model – Adopt commonly used 26 degree of freedom model.

18

Image form Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. Realtime and

Robust Hand Tracking from Depth.2014

Design and Implementation

19

Design and Implementation

20

Design and Implementation

21

Design and Implementation

22

Design and implementation

23

Design and implementation

24

Design and implementation

25

Design and implementation

26

Design and implementation

27

Design and implementation

28

Design and implementation

29

Design and implementation

30

Design and implementation

31

Design and implementation

32

Design and implementation

• Gradient descent implementation

- Created self-adjust lambda

Time complexity:

O(10*|M|*|sub(P)|)

33

Design and implementation• Gradient descent base ICP

34

Experiment and Evaluation

35

Experiment and Evaluation

• ExperimentGround truth:

• Used from CVPR14 at

http://research.microsoft.com/en-/um/people/yichenw/handtracking/index.html

Goal:

1.Test gradient-descent base ICP performance for last frame

initialization.

2.Test gradient-descent base ICP performance for random

initialization.

36

Experiment and Evaluation

37

Experiment and Evaluation

• Random initialization.

– Steps:1. Add random noise between −1° 𝑡𝑜 1° to all rotation

variable from DOF26, and Add random value between -1 to

1 to all translation variable from DOF26

2. Run ICP and compute error

3. Increasing random value range. And Repeat step 1 and

step 2.

38

Experiment and Evaluation• Result for last frame initialization

39

Experiment and Evaluation

• Result for Random initialization

40

Conclusion and Future Work

• Conclusion– Gradient descent base ICP is not feasible for hand tracking and

pose estimation.

• It is acceptable to optimize last frame initialization hand model

• Performed poorly on random initialization

– Its performance should be improve using stochastic optimization

such as PSO.

– Gradient descent base ICP has a great potential of

improvements

41

Future work

• Replace random selection by other sample techniques,

such as systematic sampling.

• More scalable hand model need to be implemented so

that it can used for general data

• Set constraints to DOF26 to avoid counter-intuitive hand

pose generated by hand model

• implement stochastic optimization to out ICP

42

Thanks for listening

43

Reference

[1]Li, Z., Chen, J., Chong, A., Yu, Z., & Schraudolph, N. N.. Using stochastic gradient-descent scheme

in appearance model based face tracking. In Multimedia Signal Processing, 2008 IEEE 28 10th

Workshop on (pp. 640-645).in IEEE.2008

[2] Bray, M., Koller-Meier, E., Müller, P., Van Gool, L., & Schraudolph, N. N.. 3D hand tracking by rapid

stochastic gradient descent using a skinning model. In In 1st European Conference on Visual Media

Production in CVMP. 2004

[3] Hou, S., Galata, A., Caillette, F., Thacker, N., & Bromiley, P.. Real-time body tracking using a

gaussian process latent variable model. InComputer Vision, 2007. ICCV 2007. IEEE 11th International

Conference on (pp. 1-8). IEEE. In 2007

44