From file-based production to real-time co-production

How semantic technology enables the transition from file-based production

to real-time co-production

Belgian Broadcast DaysVilvoorde 2012/05/24

Recently, the focus of Technology Innovation in the media sector has been ‘digitisation’.

Digitisation would bring cost-efficiency, re-use of content and collaborative production.

Major production facilities worldwide have spent excessive amounts of money, due to the integration of digital isles.

State of the art technology is not scalable in terms of systems. Growing demands for co-production tools will make this worse.

Media Production relies on craftsmanship and informal processes, where digitisation has just started.

There is a vast potential for better integration of media production processes, in particular when it comes to co-production.

Potential Real-time collaborative production Different producers Integrated Intellectual Property New types of ‘customisable’ products

Solution Area Integrated or “paper-less” production Industry standards (EBU Core, NewsML)

Semantic Technology

News Production

NewsWireeXchange

Server

Newsroom Computer System

Media Asset Management System

Search Application

PoP Server

Media Manager

Journalist Archivist

Informally Structured Information Flow

Drama Production

Digital Media Flow

Script Writing (Screenplay) Breakdown Logging (Continuity)

Principal Photography Post -Production

Key Issue Non-integrated Process

Unstructured information ‘Document oriented’ Lack of interoperability Loss of information

Symptoms Re-use is expensive Content is hard to find

Traditional remedies Manual ‘annotation’ (which is not scalable!) Database Duplication (which creates inconsistency) A complex and expensive architecture which is difficult to

maintain

Design PrinciplesA B C E F GD

H JI I

A

B

C

E

D

...

XY

A B C

ED

... X Y

Media Asset Mgnt Semantic Integration Platform

Co-production - Semantic Integration - Media Production in the Cloud Available online (Software as a Service) Replace all paper documents by browser-based applications Ensure compatibility with existing MAM systems Interoperability through maximum support of industry standards Advanced search technology ensures retrievability Real-time(!) co-production

Value Proposition

9

Semantic technology enables system integration in a way that :

System integration remains scalable, capitalising on earlier investmentsInformation remains secured and retrievable when growing to large numbers

COST

ITEMS AND SYSTEMS

MAN

AGAB

ILIT

Y

ITEMS AND SYSTEMS

Co-production is all about scalability, in terms of systems and in terms of items!

Semantic technologies:

Tools to tag the virtual space!

Belgian Broadcast DaysVilvoorde 2012/05/24

A semantic project to tag the virtual space!

In everyday life, a human being, is browsing in six different areas:• The three axes of space (height, width, depth)• The time axis (time)• and two for the memories (primary and

secondary retentions)

The human cognitive modelThe human cognitive model

Grammar

Syntax

Data

Culture

Co

nte

nt

...

Thalys – Paris/ Brussels : 05/03/2012 – 5:37

camera man

(from a live Beethoven'32 sonates rendering)...)Play/Stop Media

From essence …

Media Wrapper

Semantic info

Structural info

Administrative info

Media Location

Play

Index

Browse

Read abstract

Select segment

Modify segment

Check

Trade

camera man

(from a live Beethoven'32 sonates rendering)...)

… to Media management (Modelling, Information,..)

The semantic paradygm shift!

<Author>< ….. ><\Author>

<Title>< …………… ><\Title>

<Book>< …………… ><\Book>

<Publisher>< …………. ><\Publisher>

<ISBN>< ………. ><\ISBN>

< ….. >< ………. ><\ ….. >

The classical “flat model” modelling:

The structural change in modelling:

From

“FLAT” models

(based on Cataloguing rules

and Data Records)

To “CONTEXTUAL – LINKED Data”

models (with alias)

Semanticallyequivalent to

the Flat model

To “SEMANTIC” Models : Based on

Networked ENTITIES with Documents & Relations

in KNOWLEDGE BASES)

• Ontology based&

• Linked Open Data (semantic Web)

• Object oriented• Native persistent

E

N

H

A

N

C

E

M

E

N

T

E

N

R

I

C

H

M

E

N

T

Semantic technologies:

What is a Media ?

PERFORMING

A R T S

Birth of the modern newspaper: early 17th century!

Le 31 Mai 1631, Théophraste Renaudot lance :

«La Gazette».

http://www.museerenaudot.com/imprim.htm

F

I

L

M

Birth of Radio: early 20th century!

N E T W O R K

Birth of the television: mid of the 20th century!

W.W.W.@ I

Birth of the internet: end of the 20th century!

Media :• Performing arts: a relationship to space / time, content and

audience• Newspaper (print Media): support• Cinema: formalization of writing, storage,

commercialization (cinema)• Radio: invention of “live”, radio news, games, .... and a

network!• Television: images ...• Internet: interactivity in all dimensions• Digital convergence (multimedia, transmedia, rich-media)

Since the very beginning, RTBF covered a wide range of events in Belgium, in any field (news, entertainment, music, History, cultural and scientific magazines, documentaries, sports, etc.) but also outside Belgium, and mainly in central Africa. It currently holds more than 100.000 hours of programming, and of course the catalogue is growing every day.

Å dîale li Hardwåre, Nanesse, avou

s’tchèrète di Betacam Gini !

Vîve li Vapeurwåre qui d’vint l’vrèye !

Semantic technologies:

What is a digital Media ?

MediaMap+ - modelling:

- media(s): audio – video – picture & text - information: the meaning of a data- process (transformation)- interfaces (View): roles/skills/entity

Before passing over the record head, a tape in a recorder passes over the erase head which applies a high amplitude, high frequency AC magnetic field to the tape to erase any previously recorded signal and to thoroughly randomize the magnetization of the magnetic emulsion. Typically, the tape passes over the erase head immediately before passing over the record head. A magnetic "image" of a sound signal can be stored on tape in the form of magnetized iron oxide or chromium dioxide granules in a magnetic emulsion. The tiny granules are fixed on a polyester film base, but the direction and extent of their magnetization can be changed to record an input signal from a tape head.When a magnetized tape passes under the playback head of a tape recorder, the ferromagnetic material in the tape head is magnetized and that magnetic field penetrates a coil of wire which is wrapped around it. Any change in magnetic field induces a voltage in the coil according to Faraday's law. This induced voltage forms an electrical image of the signal which is recorded on the tape.

Annotation

Content Description

Opus

Editorial Structure

MediaAsset EditorialObject

TemporalObject

Content Assemblage

SegmentSegmentation DigitalResource

Material Manifestation

AudiovisualMedia Track

© MediãMāp - All rights reserved

http://mediamapproject.org/

Content Annotation

33Brussels: 19/05/2010 MediaMap Final Review

VIEW calculation : F(R,E,S) = V

Script

Director

Pro

fess

iona

l

SKILL

Producer

Cameraman

Am

ate

ur

ENTITY

View[Opus]=F(Producer, professionnal, opus)

ROLE

Opus

© MediãMāp - All rights reserved

Semantic technologies:

How it can improve your audiovisual production processes

From pre-production to archivingarchiefnummer : ALG 20010813 1

fragmentnummer : 1

reeks : 1000 ZONNEN EN GARNALEN

bandnummer : E03024404

formaat : DBCM

fragmenttitel : 1000 ZONNEN & GARNALEN

beeld : KL/PALPLUS

fragmentduur : 18 20

tekst : 0'00" TOERISTISCH REPORTAGEMAGAZINE OVERZICHT

ONDERWERPEN GENERIEK TOERISTISCH REPORTAGEMAGAZINE,

OVERZICHT ONDERWERPEN

0'50" VANDAAG : KUNSTENAAR LUC HOFKENS ONTWIERP EEN OASE

OP ZIJN DAKTERRAS IN BORGERHOUT DIE DOET DENKEN AAN DE

GRAND CANYON INTERVIEW MET LUC EN ZIJN VROUW

MARILOU BUITENBEELD DAK MET OMGEVING BUITENKANT

ARBEIDERSWONING, PANO OVER ROTSWANDEN, KRATEN MET WATER,

BEPANTING, FOTOALBUM MET VERLOOP WERKEN

4'00" JUNIOR : KLAARTJE ALAERTS, 13 JAAR WIL ASTRONAUTEN

WORDEN ZE BEZOEKT HETEUROSPACE CENTER METRUIMTEVEREN,

RAKETTEN SIMULATIE IN RUIMTEVEER, INTERVIEW, HEEFT EEN

UFO GEZIEN MAAKT ZELF KLEIN RAKETJE, SCHIET HET AF

7'50" DE SCHEURKALENDER : ARCHIEF RECLAMEFILM IBM

INTERVIEW MAURICE DE WILDE, EERSTE PERSOONLIJKECOMPUTER

trefwoorden : BELGIE; BORGERHOUT; ARTIEST; OASE; KUNST; GRAND

CANYON (NATUURGEBIED); DAK; TERRAS; INTERVIEW; EURO

SPACE CENTER; RUIMTEVAART; PC; BOOTTOCHT; RIJKDOM;

PASSAGIER; GASTRONOMIE; RESTAURANT; PERSONEEL;

VAKANTIE; BINNENBEELD; SCHIP; BECKERS LEEN; VRT;

LOTTO; RADIOOMROEPSTER; KLANKSTUDIO; UITVINDING;

BARBECUE; BETONMOLEN; IBM; RECLAMESPOT

rechthebbende : VRT

From pre-production to archiving, proper information is essential to:• schedule the production• communicate with post-production• enable re-use of content• manage intellectual property • create interactivity and personalisation

In order to enable automation, document-oriented methods must be replaced by a model-driven approach that relies on the semantics in the documents(!)

A Model-driven ApproachScene

“Jeep”, INT

Character“Andi”

Action“ drives recklessly. sits next to him, ...”

Action“She smiles wearily.”

Character“Joe”

Dialogue

Dialogue“I’ll make it.”

Dialogue“Couple o’ hours. ...”

Action

“Suddenly, the motor .They look at each other, concerned.”

Instructiontype: “transition”“cut to”

sound “sputters”

char. “Joe”char. “Andi”

dialoguespan“How much longer?”

actionspan“shouting”

INT . JE EP – FAVO RING JOE CHAC O

JOE dri ves rec klessly. A NDI s its next to h im,an attr active g irl i n he r mid -t wenties.

AND I(sh outing)

How muc h lon ger?

JOECou ple o’ hour s. You okay ?

She smi les wea rily.

AND II’l l ma ke it .

Sud denly, the moto r SP UTTERS. The y loo k ateac h ot her, co ncerned.

CUT TO:

# 1

# 3

# 2

# 4

# 5

# 6

# 7

# 8

A single set of numbers

Pre-production

Ingest Automation

Objectives Faster than real-time ingest Computer Assisted Rough-cut Minimise time to post-production

Key Features Script Processing Scene recognition

Face recognition Speech recognition

Automatic Clip Identification

Spectrum of knowledge: representation and reasoning capabilities:

May 2012 : Google: towards a first big semantic move?

Enjoy ! Enrich !

- The picture of the object is not the object !

- The relation between signifier/signified is arbitrary

- The meaning is a process : data – syntax/grammar - culture

René Magritte

WARM SEMANTIC THANKS

We would like to express our semantic thanks to all the persons and organizations having contributed directly or indirectly to this Belgian Broadcast Days presentation:

The MEDIAMAP+ Consortium :

SGT, Radio France, Perfect Memory, Kane, Memnon, RTBF, Limecraft, Kendra, … for their contributions to the development of semantic technologies in the broadcast world.

EUREKA Celtic Cluster & IWT – INNOVIRIS – DGCIS (Fr)

EACEA: For the support of the development of the prototypes

TITAN: For the organization of the “European Media Wrapper Round Table”

EBU: In particular to Mr. Jean-Pierre Evain (the evangelist !)

And for their major contributions on standards: ISO (for OAIS); W3C; SNIA, IASA, SMPTE