exploratory project

EXPLORATORY PROJECT

Hand Segmentation and Gesture Recognition

Under The Supervision of:Dr. Kishor Sarawadekar

By:Vandit Chauhan (14095076)Shivam Agarwal (14095063)Aman Soni (14095004)

ABSTRACT:Hand gesture recognition applications requires a

reliable identification of the hand region and its subdivision into fingers and palm areas.

The center of the palm and the hand orientation are identified.

Then circular and elliptical shapes are fitted on the extracted samples in order to reliably identify the palm and fingers area.

The proposed approach has been tested on a given dataset and preliminary results show its reliability.

Hand-Forearm Segmentation

Segmentation of the hand from an image is a necessary step for many applications. e.g. hand tracking, gesture recognition. One of the major problems is the hand-forearm segmentation. The experimental results prove that proposed algorithm is accurateand fast.


In the beginning, distance transform of the image is obtained, and then the pixel having the maximum value gives us the center point of the palm.


Then the orientation of the hand is determined, and in accordance to that the arc joining the palm with forearm is determined, as shown.


The point on the arc having maximum value of distance transform, gives us the wrist point, and a tangent is drawn at the circle at this point. The region below the tangent is eliminated.

How to segment palm and fingers ?

Palm-Fingers Segmentation

From the distance transform obtained previously, the centroid is chosen as the centre and the maximum value is chosen as radius, for drawing the circle shown in the figure.


Now the circle is traversed and at each point, the region is grown by choosing the point as center and the distance transform value of the point as the radius. The grown region is shown alongside.


Finally, the grown region is subtracted from the original image and the fingers are obtained as shown in the figure.

How to recognize the gesture?

Gesture Recognition

How to recognize the gestures ?

Understanding the dataset : • Divided into various classes based on number of fingers.

• Further divided into subclasses based on different types of gestures.

Gesture RecognitionClass 1: Gestures with one finger. Contains 3 Subclasses:

Subclass 1:

Subclass 2:

Subclass 3:

Gesture RecognitionClass 2: Gestures with two fingers. Contains 3 Subclasses:

Subclass 1:

Subclass 2:

Subclass 3:

Gesture RecognitionWhat next ? Create classifiers for various subclasses.

Train the classifiers using logistic regression with the help of data available.

Gesture Recognition

What features can be included:

Pixel value of fingertips. Centroid of palm. Relative angle b/w fingers. Centers of fingers. Orientation of hand. Angle b/w palm center and fingers.

Gesture Recognition

So how do we proceed ?

Create a hypothesis function for all subclasses.

The hypothesis function returns probability of an image, to belong to a particular subclass.

Gesture Recognition

The logistic regression hypothesis is defined as:where function g is the sigmoid function. The sigmoid function is defined as:

Gesture Recognition

The cost function in logistic regression is given as :

Then by using gradient descent to minimize the cost function, the various unknowns in theta matrix is determined. Hence the classifier is trained.

Hypothesis:

Parameters:

Cost Function:

Goal:

Gesture Recognition

Applications and Future Prospects:

Applications of Hand-Gesture Recognition

There are numerous applications of hand-gesture recognition. Some of them are listed below :• Sign Language for Blind• Hand-Gesture Controlled Robots• Virtual-reality Gaming• Gesture Control for TV


Gesture Control for TV:Now-a-days, some smart TVs have the feature where you can control the TV through hand gestures such as increasing/decreasing the volume, changing the channel, etc.


Gesture-Based Gaming:Again most of the actions in a game can be performed using hand-gestures which makes the game more realistic and adds fun.


Hand-Gesture Controlled Robots: Gesture recognition can be used to create a wireless-controlled robot.

The various classes can be assigned a specific task, such as locomotion.

The subclasses can be used for performing the operation, such as moving forward, backward, in case of locomotion.

References• Zhi-hua Chen, Jung-Tae Kim, Jianning Liang, Jing Zhang, and Yu-Bo Yuan

“Real-Time Hand Gesture Recognition Using Finger Segmentation”Hindawi Publishing Corporation

The Scientific World JournalVolume 2014, Article ID 267872, 9 pages

• Bosheng Wang, Jiaqi Xu“Accurate and fast hand-forearm segmentation algorithm based on silhouette”

2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems (Volume:02 )

• Giulio Marin, Marco Fraccaro, Mauro Donadeo, Fabio Dominio, Pietro Zanuttigh“Palm area detection for reliable hand gesture recognition”Department of Information EngineeringUniversity of Padova

exploratory project

Documents