3d motion capture assisted video human motion recognition based on the layered hmm myunghoon suk...
TRANSCRIPT
![Page 1: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/1.jpg)
3D Motion Capture Assisted Video human motion recognition
based on the Layered HMM
Myunghoon Suk & Ashok Ramadass
Advisor : Dr. B. Prabhakaran Multimedia and Networking LabThe University of Texas at Dallas
![Page 2: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/2.jpg)
Contents
• Motivation• Previous Work• Current Work
– Extracting 2D feature data (MHI)– Classifying human motions
![Page 3: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/3.jpg)
Motivation
CleanedSemantic Data
Easy to get, but
Quite noisy
3D MOCAP data
Video Human Motion data
Recognizing Video Human Motion
+
![Page 4: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/4.jpg)
HMM Modeling
T 1 2 3 4 … t
1s1O2O
3O 4OmO
2s3s
4s
ns
MHI
K-means (WEKA)
2D Motion Shape data
3D Motion Capture data
ObservationSequence data
Hidden state-transition Sequence data
Quantization
Which Motion?Forehand, Backhand, Smash,
Left kick, Right Kick, Left punch, Right punch
Test data
3D Motion Capture Assisted Video Human Motion Recognition Enhancement
![Page 5: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/5.jpg)
Current Work
• The system for falling-down detection of elderly or patient at home
• Lower layered HMMs with 3D motion capture data are to estimate one of atomic activities (e.g. movement of human hip portion)
• Higher layer recognizes exactly the falling-down motion with much longer time granularity
![Page 6: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/6.jpg)
Layered HMM
HMM (A)HMM (A)
HMM (B) (Baum-Welch)HMM (B) (Baum-Welch)
2D Feature Vector2D Feature Vector
Horizontal directionUp directionDown direction
Normal ActionAbnormal Action(Falling down)
Classification ResultsClassification Results
Position of human hip
Movement directions
![Page 7: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/7.jpg)
Background Techniques
• Extracting 2D feature (Computer Vision)– Motion History Images (MHI)
• Classification (Machine Learning)– Hidden Markov Model (HMM)– Layered Hidden Markov Model (LHMM)
![Page 8: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/8.jpg)
Motion History Images
• Keywords:– Motion Energy Image (MEI)– Motion History Image (MHI)– 2D Image feature data with suggestion of possible
actions.
![Page 9: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/9.jpg)
Motion History Images
• Motion Energy Image :-– Describes the motion energy for a given view of
action– Spatial distribution of motion – WHERE
![Page 10: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/10.jpg)
Motion History Images
• Motion History Image :-– Pixel intensity – HOW the spatial distribution has occurred
![Page 11: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/11.jpg)
Motion History Images
MEIMEI MHIMHIWHEREWHERE HOWHOW
2D Image Feature Date
2D Image Feature Date
Suggestion of Possible actions
Suggestion of Possible actions
![Page 12: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/12.jpg)
Motion History Images
• Reference Paper :-– Hierarchical Motion History Images for
recognizing Human Motion.
![Page 13: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/13.jpg)
Project - Face detection using HAAR like Features & AdaBoost algorithm
• deals with the application of one of the four AdaBoost algorithms in boosting the classifiers based on the paper "Robust Real Time Face detection by viola & jones“
• OpenCV Visual C++• Available Source files: Face detection using available HAAR like Features. • PreRequisites: Basic knowledge of using OpenCV library. Knowledge on AdaBoost(Adaptive Boosting) – A Machine Learning
Algorithm.• Other References:
http://cmp.felk.cvut.cz/~sochmj1/adaboost_talk.pdf
![Page 14: 3D Motion Capture Assisted Video human motion recognition based on the Layered HMM Myunghoon Suk & Ashok Ramadass Advisor : Dr. B. Prabhakaran Multimedia](https://reader031.vdocument.in/reader031/viewer/2022032708/56649e5d5503460f94b56a58/html5/thumbnails/14.jpg)
Project – Contour detection using Background Subtraction and Edge Detection Techniques
• OpenCV Visual C++• Available Source files:
Reading a video file.• PreRequisites:• Basic knowledge of using OpenCV library.• Other References:• Introduction to opencv programming -
http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html