Commentary-based Video Categorization and Concept Discovery

By Janice Leung

Agenda

Introduction to Video Sharing Sites Current Problem Previous Works Commentary-based Video Clustering Conclusion Future Works

Video Sharing Sites

Allows users to upload videos Shares videos worldwide Example:

Dailymotion YouTube MySpace

De Facto

YouTube More than 65,000 new videos every day 100 million videos views daily 20 million unique visitors per month

Immense amount of videos

Incredible growth of videos

How to search for desired video?

YouTube: Tags + simple Categorization

YouTube

Predefined categories Videos

Title Description Tags Category Comments

Provided by the one who uploads the video

Provided by many users

Related Works

Classify videos: Video features: color, grayscale

histogram, pixel information Keywords from description Tags

Find user interests: Object fetching information Tags

Problems

Video features Cannot tell exactly what the video is

about No users interest is considered

Keywords from description Description provided by the one who

uploaded the video Not sufficient information

Problems (Cont.)

Tags Not sufficient information May reflect users feelings on videos but

too brief to represent the complex idea of the videos

Object fetching information Reflects users interests but no

information about the videos at all

Video Categorization and Concept Discovery

Site: YouTube Videos: involving Hong Kong singers

Comment vs Tag

Comments Given by many users

Can be large amount Express users opinions Rich words describe

fine-grained level ideas

Tags Given by only one

person (the one who uploaded the video)

Few tags Describe the video in a

very brief way Singer name Song name

Comments

Include: Video content

Music styles Music ages

Singer description Appearance Style News etc.

Commentary-based Video Categorization

Objective: Categories videos based on user interests and discover the concept of videos

Cluster videos by using comments Group videos based on user interests Find video concepts Clustering algorithm: multi-assignment

NMF

Video clustering

Bi-clustering: videos and words Clusters videos and words into k

groups by matrix factorization Video-word matrix X as input

Video-word matrix X is derived by tf-idf

Tf-idf

Term frequency (tf) Suppose there are t distinct terms in

document j

where fi,j is the number of occurrence of term i in document j

Tf-idf (Cont.)

Inverse document frequency

where N is the total number of documents in dataset and ni is number of documents containing term i

Tf-idf (Cont.)

Importance weight of term i to document j

Matrix X as input to NMF is defined as

Video Clustering (Cont.)

Decompose X into non-negative matrices W and H by minimizing

where

Ref. : Document Clustering Based On Non-negative Matrix Factorization (Xu et al SIGIR’03)

0, HW

Video Clustering (Cont.)

NMF decomposition for video clustering

Video Clustering (Cont.)

Suppose Number of videos: N Number of distinct terms: M Threshold: β

W in size M x K wn,k: coefficient indicates how video n

belongs to cluster k

Video-cluster assignment

Videos can belongs to multiple groups Multi-cluster assignment Video n belongs to cluster k if Set of clusters that video n belongs to:

where K is set if all clusters

kknw ,

Video-cluster assignment (Cont.)

Threshold, β Many irrelevant videos for each cluster Coefficient distribution varies for different

clusters Coefficient distribution dependant Different for different clusters

Concept Discovery

Matrix H in size of K x M hk,m: how likely term m belongs to

cluster k Term belongs to a cluster describes

the videos in that cluster Concept words of cluster k videos

Top 10 words of cluster k

Experiment

19305 videos 102 Hong Kong singers 7271 users Number of cluster, k: 20

Experiment (Cont.)

Threshold, β Coefficient distribution dependant Threshold for cluster i is defined as

Experiment (Cont.)

Video coefficients may distribute in an extremely uneven manner

Cause poor result To compensate, threshold can be set

as

Experiment (Cont.)

0.700.300.00

0.310.640.05

0.130.220.65

0.010.640.35

0.120.230.65

0.700.300.00

0.310.640.05

0.130.220.65

0.010.640.35

0.120.230.65

Mean Coef.

0.33 0.406 0.254

Mean + SD Coef.

0.631 0.622 0.526

C1 C2 C3

V1

V2

V4

V3

V5

Experiment (Cont.)

Experiment (Cont.)

Experiment (Cont.)

Concept Words vs Tags

Concept Words vs Tags

Percentage of videos with tags covering concept words across groups

Singer Relationship Discovery

Comments on videos may talk about singers

Singer styles, appearance, news Singer clustering using comments Reveals relationships between singers Discovers hidden phenomenon

Singer Relationship Discovery (Cont.)

Conclusion

Captures user interests more accurately and fairly than that of the human predefined categories

Categories can be changed dynamically, user interest changes from time to time

Obtain clusters with fine-grained level ideas Ease the task of video search by

categorizing videos and refining index

Future Works

Extend to user clustering Obtain relationships videos, singers

and users of the entire social network Study the social culture Ease the job of advertising to target

customers Connect people who share the same

interests

Q & A

Questions?