Lyndon Nixon, MODUL University

Annotating TV programming and

linking to related content on the Web

18 November 2014, Potsdam, Germany

http:/www.linkedtv.eu

Structure

Introducing Linked Television

Challenges for TV Annotation

Challenges for TV Enrichment

LinkedTV Solutions

Summary

What is Linked Television?

„... a true TV ecosystem must functionally integrate the apps with the

television programming service and content being offered to the consumer.“TVs can't be smart. Stop trying to make it happen. - WIRED Opinion, Gary Myer, Oct 2013

What is Linked Television?

„Television won't last because people will soon get tired of staring at a plywood

box every night.“

- Darryl Zanuck, movie producer, 20th Century Fox, 1946.

What is Linked Television?

Source: Tomorrow FOCUS Media „Smart-TV Effects 2014-1“

Source: Tomorrow FOCUS Media „Smart-TV Effects 2014-1“

What is Linked Television?

ARD/ZDF Online Study 2014 „Second Screen: Parallelnutzung von

Fernsehen und Internet“

08.11.2012

What is Linked Television?

37-52% of TV viewers are using a

companion device alongside the TV

program.

http://www.linkedtv.eu

Ever saw something

on TV and wanted to

know more about it,

but didn‘t even know

how to search for it?

What is Linked Television?

Scenario 1:LinkedNews app

Easy access to

background to the news

Focus on key concepts of

interest in each item

hbbTV single screen and

dual screen apps

Scenario 2: LinkedCulture app

Easy access to

information on the art

objects discussed in the

program

Secondary screen-

orientation, cast TV

program to main screen

Television annotations?

What we get?

Television annotations?

What we need?

Television enrichments?

What we get? What we want?

dbpedia.org/resource/Jan_Sluyters

LinkedTV Solution

linkedtv.eu

Examples of Paintings by Jan Sluijters

“...schilderij van Jan Sluijters....”

Video object and

text detection

Personalisation

engine

Expansion of

related concepts

Linking to

related content

Connection to

concepts

… and their digital images

Concept Detection

linkedtv.eu

“...schilderij van Jan Sluijters....”

Video object and

text detection

Scenario content specific video chapter detection (News Items, Art Objects)

Aggregated REST service for all analysis approaches in LinkedTV, integrated with the LinkedTV Platform

Keywords and text fragments can be connected to uniquely identified, distinct named entities using online entity extraction services (NERD, http://nerd.eurecom.fr)

Concept Connections

linkedtv.eu

Connection to

concepts

dbpedia.org/resource/Jan_Sluyters

Introduction Silver Tea Jar

Nelleke van der Krogt Friesland

Hessel van Martena Silver (zilver)

Museum Martena Tea jar (theebus)

Franeker Emiel Aardewerk

Tin coffee decanter (art object)

Meesterteken

Zilversmid

Sondersitzung Blackout

BER Berlin Brandenburgairport

Berlin Fire Brigade

Hartmut Mehdorn Frank Henkel

Jochen Großmann Vattenfall Europe

Linking to related content

linkedtv.eu

Digital images of Jan

Sluijters‘ paintings

Linking to

related content

• Information cards display relevant properties of annotated entities by type

• Links to related content are organised by dimensions

• Access to related video chapters on the LinkedTV Platform

• Specific enrichment services for • News articles• Cultural heritage objects

Linked Culture

Tussen Kunst en Kitsch Video

Chapter segmentation + tagging „Art Objects“

Entity extraction incl. Art Object annotation

Related

TKK

chapters

Related

art objects

(Europeana)

Related

White List

media

TVEnricher (TKK configuration)

Editor Tool LinkedTV Player

Entity Proxy

Art objects in TKK episodes

Art object: semantic model

http://data.linkedtv.eu/object/avro/8a8187f2-3fc8-cb54-0140-7dccd76f0001/2138

A silver tea jar

RDF Is-a Container http://vocab.getty.edu/aat/300045611

CRM Consists-of Silver http://vocab.getty.edu/aat/300010975

VRA locationCreationSite Friesland http://www.geonames.org/2755812

DCT temporal Start: 1690, End: 1742

Mapping to Europeana API

http://data.linkedtv.eu/object/avro/8a8187f2-3fc8-cb54-0140-7dccd76f0001/2138

A silver tea jar

RDF Is-a Container

CRM Consists-of Silver

VRA locationCreationSite Friesland

DCT temporal Start: 1690, End: 1742

what:(container+OR+houder+OR+b

ak+OR+tank+OR+blik)

proxy_dc_format:zilver

where:Friesland

YEAR:[1690+TO+1742]

Leeuwarden

Enrichment results

Linked Culture

https://vimeo.com/108891238

Linked News

RBB Video

Chapter segmentation + tagging as „News Item“

Named Entity Expansion over news items

Related

news

items

Related

news articles

Related

White List

media

TVEnricher (RBB configuration)

Editor Tool LinkedTV Player

Entity Proxy

Linked News

Asparagus

Harvest

Poland Beelitz

Linked News

http://showcase.linkedtv.eu/showcase3.html

Summary

Rich annotation of AV content can enable added value services

but

• Consider the purpose of the annotation – this changes the

requirements on the annotation model and approach

• Consider the domain of the content to be annotated – every

domain has different requirements on how to annotate it

Enrichments are an example of added value on top of

annotations but

• Enrichment sources also need to be annotated

• Relevancy depends on the right choice of annotation entities

No perfect, automated approach: room for ongoing research!