taxonomic classification for web-based videos
DESCRIPTION
Taxonomic classification for web-based videos. Author: Yang Song et al. (Google) Presenters: Phuc Bui & Rahul Dhamecha. 1. Introduction. Taxonomic classification for web-based videos. Web-based Video Classification. Web-based Video (e.g. Youtube ) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/1.jpg)
Taxonomic classification for web-based videos
Author: Yang Song et al. (Google)Presenters:
Phuc Bui & Rahul Dhamecha
![Page 2: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/2.jpg)
1. Introduction
![Page 3: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/3.jpg)
Taxonomic classification for web-based videos
![Page 4: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/4.jpg)
Web-based Video Classification
• Web-based Video (e.g. Youtube)– Over 800 million unique users visit / month– Over 4 billion hours of video are watched / month– 72 hours of video are uploaded / minute
• Classification– Improve User experience– Increase Website profit
![Page 5: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/5.jpg)
What’s interesting?
• Large-scale classification– Taxonomy of categories– Unlimited domain
• Combined Approach– Text
• Labeled Web documents• Labeled Video
– Video• Content-based features
![Page 6: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/6.jpg)
Overview Approach
• Multi-labels Classification– One classifier for each category
• Classifiers– Text-based Classifier• from Web-based Documents
– Combined Classifier• Text-based Classifier• Video content-based features
![Page 7: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/7.jpg)
2. Algorithms
![Page 8: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/8.jpg)
TAXONOMIC CLASSIFICATION: - THE VARIOUS CATEGORIES.
![Page 9: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/9.jpg)
TRAINING SET OF EACH CATEGORY
![Page 10: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/10.jpg)
Pre-trained text based classifiers of each category used for porting videos Labeled Video data is used for training these classifiersNo. of Classifiers = No. of CategoriesAda-boosting is deployed to aggregate these weak classifiers to a Strong Classifier
MIGRATION FROM TEXT TO VIDEO
![Page 11: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/11.jpg)
Feature Extraction
Text Based Features.
President Obama: the Real Mitt Romney - Denver, Colorado
Title
Description
Keywords
![Page 12: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/12.jpg)
Content Based Features
Moments from multi-scale analysis
Color HistogramMean, variance of each channel.
Difference between mean of center and boundary
![Page 13: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/13.jpg)
Content Based Features contd…
Edge DetectionCanny Edge Detection Algorithm
![Page 14: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/14.jpg)
Content Based Features contd…
Color Motion Features
Cosine Difference of the histograms of subsequent frames.
![Page 15: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/15.jpg)
Content Based Features contd…
Shot Boundary Features
TypesHard CutFadeDissolveWipe
![Page 16: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/16.jpg)
Hard Cut
instantaneous transition from one scene to the next
![Page 17: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/17.jpg)
Fade
A Fade which is a gradual between a scene and a constant image (fade-out) or between a constant image and a scene (fade-in).
![Page 18: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/18.jpg)
Dissolve
A Dissolve is a gradual transition from one scene to another in which the first scene fade-out and the second scene fade-in. so it is a combination of fade-in and fade-out.
![Page 19: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/19.jpg)
Wipe
A Wipe is a gradual transition in which a line move across the screen, with the new scene appearing behind the line.
![Page 20: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/20.jpg)
Integration
Labled Videos
Text Based Feature
Extraction
Apply Pre-trained Text Classifiers
F Score from Classifiers
Labled Videos
Content Based Feature
Extraction
F Score and Content Based Features are
combined
A new Classifier is trained.
![Page 21: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/21.jpg)
3. Experiments
![Page 22: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/22.jpg)
Data
• 5789 videos• 9087 labels• 565 categories
• 80% training• 20% evaluation
![Page 23: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/23.jpg)
Evaluation
• Precision
• Recall
• F-score
![Page 24: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/24.jpg)
Results
• Sample videos
![Page 25: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/25.jpg)
Results
• 80-category classifiers • 1037-category classifiers
![Page 26: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/26.jpg)
Results
![Page 27: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/27.jpg)
Results
• Adaption + Content-based features classifiers
• Content-based features-only classifiers
![Page 28: Taxonomic classification for web-based videos](https://reader035.vdocument.in/reader035/viewer/2022062323/56816263550346895dd2c4fa/html5/thumbnails/28.jpg)
4. Conclusion & Dicussion
• Video features– Content-based– Associated texts
• Web-documents based text classifier
• Semi-supervised learning
• Image-based classifiers– ImageNet