Title

Emerging Technologies Department of Computer Science & EngineeringR V College of Engineering, Bangalore THARUN S 1RV12CS113

RVCE - Marching Ahead 4 Aug 2013Project TitleHand Gesture Recognition using Opticl flow techniquesRVCE - Marching Ahead 4 Aug 2013Literature survey1) Gihan Shin,; Junchul Chun,. Vision-based Multimodal Human Computer Interface based on Parallel Tracking of Eye and Hand Motion 2007 International Conference on Convergence Information TechnologyThis paper presented a vision-based multimodal human computer interface system using eye and hand motion tracking. Conventional vision-based human computer interface use eye or hand motion tracking individually.For motion tracking and recognition of eye and hand gesture optical flow method and template matching is used.

RVCE - Marching Ahead 4 Aug 2013Conventionally, tracking the motion of hand gesture and eye is considered a individual process for developing a different virtual interface. However, in this work, a vision-based multimodal human computer interface system which combines eye and hand motion tracking and their recognition was developed. The proposed vision-based virtual interface integrated the function of the motion tracking of eye blinking and hand gesture with the function of their recognition.The proposed interface could be easily used a substitute for a traditional devices such as mouse.RVCE - Marching Ahead 4 Aug 20132. Philippe Weinzaepfel,; Jerome Revaud., DeepFlow: Large displacement optical flow with deep matching 2013 IEEE international Conference on Computer Vision, (ICCV)Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik, DeepFlow, blends a matching algorithm with a variational approach for optical flow. it proposes a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to effi- ciently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable asset for integration into an energy minimization framework for optical flow estimation.

RVCE - Marching Ahead 4 Aug 2013Guojian Wei, Zhiqiang Hou, Wu Li, Wangsheng Yu Color image optical flow estimation algorithm with shadow suppression 2013 Seventh Internation Conference on Image and Graphics (ICIG)A new optical flow estimation algorithm for color image is proposed in this paper overcome the influence of shadows and improve the accuracy of optical flow estimation. The idea is to compute optical flow of the color invariant space and then fuse the result with optical flow of the RGB space. Firstly, it computes edge strength from color invariants of two adjacent frames to build three-channel color images, and then computes optical flow of the images we have built before, finally, fuses the result of color invariant optical flow with RGB image optical flow through L norm. The experimental results using this method show that the target regions detected by this method are more robust and accurate under shadow and illumination conditions.

RVCE - Marching Ahead 4 Aug 2013Research GapSo far, most of the research done in this field has been directed to computation of the flow field between the frames of black-and-white image sequences.Colour by itself carries more information about the objects than the grey value and provides us more information that we can use for computing the optical flow.

RVCE - Marching Ahead 4 Aug 2013MotivationHuman computer interactions has always been limited to few hardware devices.Allowing gestures to be a part of interaction with the help of image processing.The benefits of using gestures for human computer interaction.Optical flow algorithms are efficient and simpler to implement

RVCE - Marching Ahead 4 Aug 2013Motivation Motion is a rich source of information about the world. Tracking of a moving object over time is a challenging issue on video processing. Optical flow is very useful for detecting the moving objects in a video. Optical flow vector is calculated between adjacent frames in a video and we can simply track the path of moving object by simply following the arrows of optical flow vectors.RVCE - Marching Ahead 4 Aug 2013Problem statement and objectivesGesture handsOptical flowIncrease efficiencExplore new tecniques use papersUse existing methods and change shit

RVCE - Marching Ahead 4 Aug 2013Content and IdeasOptical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. Optical Flow estimation is used to estimate displacement vectors for each pixels in two frames of a video. This displacement vector says how quickly a pixel is moving across the image and direction of movement of each pixel.One of the key correspondence problems in Computer Vision is the computation of the optic flow field between two consecutive frames of an image sequence.RVCE - Marching Ahead 4 Aug 2013Optical Flow A video is a collection of frames and we are trying to estimate the direction and the velocity of the motion introduced in such an image sequence. In order to compute the motion we need to compare certain features that stay invariant in the sequence and that can help us to identify the objects and their movement in the scene.The 2D displacement field that describes the motion of a scene between two consecutive frames of an image sequence is called optic flow.

RVCE - Marching Ahead 4 Aug 2013The image is represented as a pixel matrix, where every pixel has a unique coordinate position. In order to represent the problem mathematically, we consider a scalar-valued image sequence I(x, y, t)where (x, y) denotes the coordinates of the pixel at time t.Let the image frame at time t be denoted by I(x, y, t).Then the image frame at time t + 1 will be denoted by I(x+u, y+v, t+1), where x + u and y + v show the new position of the pixel which was previously at (x,y) position. Optical flow is a flexible representation of visual motion that is particularly suitable for computers analyzing digital images. It associates each image pixel (x, y) with a two component vector u = [u(x, y), v(x, y)] indicating its apparent instantaneous 2D velocity. The presented paper uses a combination of feature match algorithm and super pixel segmentationRVCE - Marching Ahead 4 Aug 2013Sugitha, T.S.; Gladwin, J.K.T.,Optical Flow Based On Feature Match and Super Pixel Segmentation Advances in Computing and Commonications (ICACC), 2014

This paper describes optical flow estimation. The estimation is carried out using feature match method followed super pixel segmentation.Feature match algorithm is based on the matching between features of patches of adjacent frames in a video .This displacement vectors obtained by feature match algorithm are modified by using super pixel segmentation based optical flow estimation.The best optical flow associated with each superpixel is computed by optimizing a cost function.RVCE - Marching Ahead 4 Aug 2013Feature match algorithm computes the matching of patches between two frames and find out displacement vector for each pixel in the image .Then the super pixel segmentation is applied to the displacement vectors obtained from the Feature match algorithm and it improves the the result obtained by Feature Match algorithmSuperpixel based optical flow is computed by optimizing a cost function.Super pixel based optical flow estimation improves feature match algorithm and increase the quality of optical flow estimation. The resultant flow vectors is found to be more closer to the Ground Truth.

RVCE - Marching Ahead 4 Aug 2013The core element of patch matching methods is a repeated search of all patches in one image region for the most similar patch in another image region. In other words, given images A and B in figure 2, find for every patch in A the nearest neighbor in B . We call this mapping as nearest neighbor field mapping.Computing Approximate Nearest-Neighbour Field(ANNF)between a pair of images has become a major problem in Optical flow estimation.

RVCE - Marching Ahead 4 Aug 2013Feature match algorithmThe boxes (Red, Green, Blue and Yellow)in the figure denote similar patches in the pair of images. For each patch of size pp in the Target image, finding the closest patch in the Source image is the Nearest Neighbour problem between a pair of images. Since dealing in higher dimensions has become a major problem , we propose that p p image patches can be approximated intelligently to a low dimensional feature vector and then conventional k-nearest neighbour algorithms can be used effectively on this feature vector.

RVCE - Marching Ahead 4 Aug 2013Super pixel segmentation based optical flowIn computer vision, image segmentation is the process of partitioning a digital image into multiple segments or sets of pixels called super pixels. The goal of super pixel segmentation is to simplify or change the representation of an image into something that is more meaningful and easier to analyze by using this super pixel segmentation , we can assign a label to every pixel in an image such that pixels with the same label share certain visual characteristics.Optical flow result can be improved by using super pixel segmentation. Displacement vector obtained by using Feature match algorithm can be improved by using super pixel segmentation based on the assumption that a super pixel should have same velocity vectors. Same velocity vectors is assigned to a super pixel by using a cost function.

RVCE - Marching Ahead 4 Aug 2013Testing

RVCE - Marching Ahead 4 Aug 2013

RVCE - Marching Ahead 4 Aug 2013PerformanceThe most commonly used measure of performance for optical flows are average angular error (AAE)and average end point error(AEE). The angular error between two flows (u0, v0)and (u1, v1) is the angle in 3D space between (u0, v0, 1) and(u1, v1, 1). The AE is usually computed by normalizing the vectors, taking the dot product, and then and then taking the inverse cosine of their dot product .

RVCE - Marching Ahead 4 Aug 2013Thank youRVCE - Marching Ahead 4 Aug 2013