temporal segmentation of egocentric...
TRANSCRIPT
![Page 1: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/1.jpg)
Temporal Segmentation of Egocentric VideosVIVEK PRADHAN
![Page 2: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/2.jpg)
Problem Statement
![Page 3: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/3.jpg)
DataSet
![Page 4: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/4.jpg)
DataSetShow External Video.
![Page 5: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/5.jpg)
Problem Statement
![Page 6: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/6.jpg)
Problems with Short Term motion detectors
![Page 7: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/7.jpg)
Problems with Short Term motion detectors
![Page 8: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/8.jpg)
Cumulative displacement curvesDivide Image into 10x5 cells and compute displacement of cells.
Make it cumulative over time to cancel out head motions.
![Page 9: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/9.jpg)
Properties of CD curves
![Page 10: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/10.jpg)
Properties of CD curves
![Page 11: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/11.jpg)
Properties of CD curves
![Page 12: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/12.jpg)
SVM features from CD curves.Radial Projection Response
Motion Clusters
FOE estimation
![Page 13: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/13.jpg)
Gaze Fixation through CD curvesCauses head motion to stop.
This can be exploited to find the gaze fixation.
![Page 14: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/14.jpg)
Experiments and Results
![Page 15: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/15.jpg)
Experiments and Results
![Page 16: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%](https://reader036.vdocument.in/reader036/viewer/2022070803/5f03244b7e708231d407bea9/html5/thumbnails/16.jpg)
CD curves for other classes and tasksMight have very different results for chest mounted cameras.
Authors have performed the same experiment with 3D CNNs.
Activity recognition.