HUMAN ACTION RECOGNITION USING LOCAL SPATIO-TEMPORAL DISCRIMINANTEMBEDDINGKui Jia and Dit-Yan Yeung, IEEE Conference on Computer Vision and Pattern Recognition

Instructor: Jenn-Jier LienReporter: Mei-Hsuan Chao

2

OUTLINE

Introduction Related work Local spatio-temporal discriminant embedding Experiments Conclusion

3

INTRODUCTION

Recognizing human activities in videos has many important computer vision applications.

A human silhouette contains both instant spatial information about the body pose and dynamic temporal motion information of the global body and local body parts.

Human silhouettes can be considered as data points on nonlinear dynamic shape manifolds.

The aim in this paper is to find a manifold embedding method which can optimally make use of the discriminative temporal shape variation information between different types of actions.

4

RELATED WORK

LPP(Locality preserving projections) LPP constructs a nearest neighbor graph. By using the Laplacian of the graph, LPP can find a mapping

which optimally preserves the local neighborhood information.

LSDA(locality sensitive discriminant analysis) LSDA first constructs one nearest neighbor graph, and then

splits it into the within-class graph and the between-class graph.

5

LOCAL SPATIO-TEMPORAL DISCRIMINANT EMBEDDING

Neighbor graph G

Short video segment Si

6

LOCAL SPATIO-TEMPORAL DISCRIMINANT EMBEDDING

Objective functions

7

LOCAL SPATIO-TEMPORAL DISCRIMINANT EMBEDDING

Principal angles between Si and Sj

8

LOCAL SPATIO-TEMPORAL DISCRIMINANT EMBEDDING

Optimal embedding

The columns of an optimal A can be obtained as the generalized eigenvectors corresponding to the l largest eigenvalues.

9

LOCAL SPATIO-TEMPORAL DISCRIMINANT EMBEDDING

Iterative learning

10

EXPERIMENTS

Data setting

Design of two-stage recognition scheme

Frame by frame basis

Short segment

basis

Test silhouette

frame

Recognitionresult

11

EXPERIMENTS

12

CONCLUSION

Propose a novel local spatio-temporal discriminant embedding (LSTDE) method.

Perform recognition on a frame-by-frame or short video segment basis.

Experimental results demonstrate that the proposed method can accurately recognize human actions, and outperforms some representative manifold embedding methods.