nexgentv: providing real-time insight during political debates in a second screen application

EURECOM – Campus SophiaTech, 450 Route des Chappes, 06410 Biot FRANCE www.eurecom.fr NexGenTV: Providing Real-Time Insight during Political Debates in a Second Screen Application Olfa Ben Ahmed 1 , Gabriel Sargent 2 , Florian Garnier 3 , Benoit Huet 1 Vincent Claveau 2 , Laurence Couturier 3 , Raphaël Troncy 1 , Guillaume Gravier 2 , Philémon Bouzy 3 , Fabrice Leménorel 4 [email protected], [email protected], [email protected], [email protected] ACM MULTIMEDIA 2017 October 23-27, 2017, Mountain View, USA (1) (2) (3) (4) Objectives Enrich viewers’ TV experience by providing relevant additional information and interactive content on their mobile devices. Leverage on latest multimedia content analytics and social media monitoring to fuel a second screen authoring platform. Provide an end to end platform from broadcast stream to front-end application on mobile devices. Motivation Exploit the convergence of TV and Internet to enable Broadcasters to produce and deliver Second Screen Applications and facilitate Social TV. Framework Content Analysis Technologies Functionalities to segment, describe, enrich TV streams based on content and social network analytics: Face detection and recognition Detecting and identifying key persons within a TV program in real time using deep facial representation Speaker Diarization Segmentation of the program based on speech turns Term Extraction Identification of the salient/representative terms describing a sequence Content Hyperlinking Providing links to related material based on subtitles Tweet collection and Analysis Online social activity monitoring allows to detect important events in the program and to detect the polarity of comments (sentiment analysis) Interfaces Back-Office Clip selection, assisted from speech and speaker turn detection Clip enrichment, using content analysis and exploiting an ontological description of political life Clip publication (pushed on the mobile app) Mobile App Clips pushed from the back-office appear on the timeline and can be viewed along with their description and additional hyperlinks Real-time stats are available for viewing during the program: speaking time, visual appearance duration, amount of reaction on social media, popularity of candidates… video subtitles Social Networks Internet tweets clip selection Broadcaster Operator BACK-OFFICE T0 Knowledge Base NEXGENTV Clip3 Clip2 Clip1 - Face Detection and Recognition - Speaker Diarization - Segmentation - Keywords Detection - Named Entity Recognition - Event – Clip Association - Tweet – Clip Association - Named Entity based Search - Topic based Search - HyperLinking End User Mobile App T-end http://www.nexgentv.fr/

Upload: benoit-huet

Post on 23-Jan-2018

170 views

Category:

Social Media

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

EURECOM – Campus SophiaTech, 450 Route des Chappes, 06410 Biot FRANCE www.eurecom.fr

NexGenTV: Providing Real-Time Insight during Political Debates in a Second Screen Application

Olfa Ben Ahmed1, Gabriel Sargent2, Florian Garnier3, Benoit Huet1 Vincent Claveau2, Laurence Couturier3, Raphaël Troncy1, Guillaume Gravier2, Philémon Bouzy3, Fabrice Leménorel4

[email protected], [email protected], [email protected], [email protected]

ACM MULTIMEDIA 2017 October 23-27, 2017, Mountain View, USA

(1)

(2)

(3)

(4) Objectives Enrich viewers’ TV experience by providing relevant additional information and interactive content on their mobile devices. Leverage on latest multimedia content analytics and social media monitoring to fuel a second screen authoring platform. Provide an end to end platform from broadcast stream to front-end application on mobile devices.

Motivation Exploit the convergence of TV and Internet to enable Broadcasters to produce and deliver Second Screen Applications and facilitate Social TV.

Framework

Content Analysis Technologies

Functionalities to segment, describe, enrich TV streams based on content and social network analytics:

Face detection and recognition Detecting and identifying key persons within a TV program in real

time using deep facial representation

Speaker Diarization Segmentation of the program based on speech turns

Term Extraction Identification of the salient/representative terms describing a

sequence

Content Hyperlinking Providing links to related material based on subtitles

Tweet collection and Analysis Online social activity monitoring allows to detect important events

in the program and to detect the polarity of comments (sentiment analysis)

Interfaces Back-Office

Clip selection, assisted from speech and speaker turn detection

Clip enrichment, using content analysis and exploiting an ontological description of political life

Clip publication (pushed on the mobile app) Mobile App

Clips pushed from the back-office appear on the timeline and can be viewed along with their description and additional hyperlinks

Real-time stats are available for viewing during the program: speaking time, visual appearance duration, amount of reaction on social media, popularity of candidates…

video

subtitles

Social Networks

Internet

tweets

clip selection

Broadcaster Operator

BACK-OFFICE

Knowledge Base NEXGENTV

Clip3 Clip2 Clip1

- Face Detection and Recognition - Speaker Diarization - Segmentation

- Keywords Detection - Named Entity Recognition