A Human-Centered Computing Framework to Enable Personalized News Video Recommendation

오준혁 (Oh Jun-hyuk)

Questions• How to detect news topic from video?• How to measure inter-topic association?• How to measure interestingness of news topic?

Contents• Introduction• Related Work• User-Adaptive News Topic Recommendation

• News Topic Detection• Topic Association Extraction• Interestingness Scores of News Topics• Hyperbolic Topic Network Visualization• Personalized Topic Network Generation

• Personalized News Video Recommendation• Algorithm Evaluation• Conclusions

Contents• Introduction• Related Work• User-Adaptive News Topic Recommendation

• News Topic Detection• Topic Association Extraction• Interestingness Scores of News Topics• Hyperbolic Topic Network Visualization• Personalized Topic Network Generation

• Personalized News Video Recommendation• Algorithm Evaluation• Conclusions

Introduction• News video recommendation in CNN

• Not related to the current news topic and user profile• Users need to follow(subscribe) news topics manually

Introduction• Video recommendation in YouTube

• Related to the current video topic and user profile but not visual-ized

Introduction• Topic Network

• Visualize news videos and represent inter-topic association.

• Hyperbolic visualization• Enables interactive topic network navigation(browsing).• Recommend the news topics of interest according to the personal

preferences.

Contents• Introduction• Related Work• User-Adaptive News Topic Recommendation

• News Topic Detection• Topic Association Extraction• Interestingness Scores of News Topics• Hyperbolic Topic Network Visualization• Personalized Topic Network Generation

• Personalized News Video Recommendation• Algorithm Evaluation• Conclusions

Related Work

1. Automatic news topic detection• Identification of individual topics within a broadcast news video by

detecting the boundaries where the topic of discussion changes.• Special program structures and styles can be used to detect

boundaries.

Intro News Intro News Intro News … …

KBS news broadcast

Related Work

2. News visualization• Existing visualization systems disclose all the available news topics

to news seekers without considering interestingness of topics.

It will be better to provide a small number of interesting news!

Contents• Introduction• Related Work• User-Adaptive News Topic Recommendation

• News Topic Detection• Topic Association Extraction• Interestingness Scores of News Topics• Hyperbolic Topic Network Visualization• Personalized Topic Network Generation

• Personalized News Video Recommendation• Algorithm Evaluation• Conclusions

User-Adaptive News Topic Recommendation

• Goal : recommend the news topics of interest by incorpo-rating topic network and hy-perbolic visualization.

• To-do List• News Topic Detection• Topic Association Extraction• Interestingness Scores of News

Topics• Hyperbolic Topic Network Visual-

ization• Personalized Topic Network

Generation

Contents• Introduction• Related Work• User-Adaptive News Topic Recommendation

• News Topic Detection• Topic Association Extraction• Interestingness Scores of News Topics• Hyperbolic Topic Network Visualization• Personalized Topic Network Generation

• Personalized News Video Recommendation• Algorithm Evaluation• Conclusions

News Topic Detection• Define a set of over 4,000 elemental news topics.• Three major sources are integrated

• Audio, Video, Closed Caption

News Topic Detection

News Topic Detection – Closed Captions

• Natural Language Processing(NLP) is conducted.• Closed Captions are segmented into a set of keywords.• Special text sentences are removed by syntax parser

• ex) “CNN’s Andrew reports from Seoul” not related to the topic

• TreeTagger is used to extract the POS(part-of-speech) in-formation• POS : a linguistic category of words (lexical category)

• LingPipe is used to

extract keywords.

News Topic Detection – Closed Captions

News Topic

K-POP, Korean Music, PSY

LingPipe (Keyword Extraction)Korean, music, PSY, Gangnam Style, release, sequel

TreeTagger (POS Tagging)Korean[NN] music[NN] sensation[NN] PSY[FW] will[MD] release[VD] his[PP$] much[JJ] anticipated[JJ] sequel[NN]

to[TO] "Gangnam Style”[FW] Friday[NN].[SENT]

Original CaptionKorean music sensation PSY will release his much anticipated sequel to "Gangnam Style" Friday.

News Topic Detection

News Topic Detection – Audio• Automatic Speech Recognition(ASR) system is used to

translate the audio channel to a text transcription.• Audio Text processed in a similar way to closed cap-

tion.

Hidden Markov Model for speech recognition

News Topic Detection

News Topic Detection – Video• Detect video objects(text area, human face) because they

provide important clues about news story.• Confidence map is used to measure the importance of

video objects in video.

News Topic Detection

Contents• Introduction• Related Work• User-Adaptive News Topic Recommendation

• News Topic Detection• Topic Association Extraction• Interestingness Scores of News Topics• Hyperbolic Topic Network Visualization• Personalized Topic Network Generation

• Personalized News Video Recommendation• Algorithm Evaluation• Conclusions

Topic Association Extraction• Inter-topic association(inter-topic contextual relationship)

• d(Ci , Cj ) : The length of the shortest path between the news top-ics by searching the relevant keywords for news topic interpretation from WordNet.

• ψ(Ci, Cj) : the co-occurrence probability between the relevant news topics obtained in news topic detection process. The frequency of co-occurrence of two news topic keywords in the same video.• ex) In a news video, “PSY”, “Music” co-occurs.

Topic Association Extraction• WordNet : a lexical database for the English language.

• Provide a graph representing semantic relationship between words.

Topic Association Extraction• Topic network can be generated from topic association• News topics are organized according to the strength of

their association• Allow news seekers to easily recognize global overview of

large-scale news videos at the first glance.

Contents• Introduction• Related Work• User-Adaptive News Topic Recommendation

• News Topic Detection• Topic Association Extraction• Interestingness Scores of News Topics• Hyperbolic Topic Network Visualization• Personalized Topic Network Generation

• Personalized News Video Recommendation• Algorithm Evaluation• Conclusions

Interestingness Scores of News Topics

• Interestingness Score

• m(Ci) : the number of TV channels or news programs which have discussed the given news topic Ci. Popularity

• k(Ci) : the number of news topics linked with the given news topic Ci on the topic network. Importance (similar to PageRank)

• Used to highlight the most interesting news topics and eliminate the less interesting news topics for reducing the visual complexity for large-scale topic net-work visualization.

Interestingness Scores of News Topics

• PageRank Algorithm

Summary & Answers• How to detect news topic from video?

• Define a set of news topic.• Integrate multi-modal sources

• Closed caption : Natural language processing• Audio : Automatic speech recognition• Video : video object extraction and classification

• How to measure inter-topic association?• Keyword association : length of path between keywords in WordNet• Co-occurrence : the probability of co-occurrence obtained in news topic

detection process.

• How to measure interestingness of news topic?• Popularity : the number of TV channels or news programs which have

discussed the given news topic• Importance : the number of news topics linked with the given news topic

on the topic network.

THANK YOUQ&A