interactive content authoring for a/153 atsc mobile digital television employing open source tools

1

• NAB 2010 came looking for information on how to author interactive content for the medium

• Other than standards papers there are few to no authoritative sources on how to use these standards in this environment.

2

• Interactive Broadcast Learning Lab • Digital Hydra --- TransMedia • 2M Funding 2yr Interactive Broadcast Ontario • Joined ATSC

• Hosted Canadian ATSC Rollout Conference • Tested datacast ideas using ATSC from signage to

robotic control applications

3

• ATSC M/H content group

4

• Mobile interactive television 10 years later

5

• A/153 Mobile Standard Adopted • A/153 Part 5 Application Framework

• Standard methodology for multimedia creation • Brings interactivity to TV medium • Puts standardized technology “in the band” or

“under the hood” to provide for interactivity • Significant enough to build upon for future

advances in transmission such as ATSC 2.0 and 3.0

• Just the start of “under the hood” technologies that have the potential to provide tremendous opportunity to the medium

6

• 2003 W3C Plenary Session • Initial Web --- Tomorrows Web

• XML, SOAP, SVG, XHTML common • Metadata Universe

7

• TV industry knows MPEG-4 as a video/audio compression format

• Way more that a compression format. • Multimedia object based coding format

8

• MPEG-4 now best thought of as a multimedia object coding standard

• Way more that a compression format. • 28 Parts MPEG-4 standard • Cross into areas shown in slide

• MPEG-4 now 28 parts from which the principles of the rich media environment for A/153 was derived

9

• Nutshell - Most important things learned • How to work with linked data • How to work with Multimedia objects • How to mold them into an A/153 Part 5 Rich

Media Application

10

• Agenda

11

• First • Presentation Format • Part that one really thinks of as the content

• Second • Packaging Format • Prepares content for distribution • Dependent on transportation format

• Third • Transportation Format • Medium of distribution

12

• A/153 Part 5 • Based on Open Mobile Alliance Rich Media Environment • OMA-RME

• Presentation Format • W3C SVG Tiny 1.2 • ECMAScript

• Mobile Profile • known as Javascript

• Packaging Format • 3rd GenerationPartnership Project (3GPP) • Dynamic Interactive Multimedia Scenes (DIMS)

• Transportation Format • ATSC Mobile DTV Format • A/153 • Uses ATSC transmitters to deliver content

• A/153 Part 5 • Umbrella standard • Contains elements

• Of application creation • Of Delivery • Of Control

• Our work • Prototype discovery driven • Involves presentation and packaging formats only • Did not include transmitting applications • No access to transmitter technology

13

• Classic academic paper or book on MPEG-4 objects

• Primitive multimedia objects are arranged and rendered into a multimedia scene

• MPEG-4 scenes were designed to pull primitive objects from any server on a network

• A scene stream server was devised to identify scene elements

14

• The scene stream server is responsible to deliver the Scene Description for a Rich Media Application that defines the scene layout

15

• The scene description can define each object, element or fragment that makes up a primitive multimedia object.

• The terms object and element are often used interchangeably when describing a scene of object.

16

• Scene Description • advises browser of scene objects • places (arranges) media objects on interface • transforms media objects attributes to alter

appearance o scale, transparency, color, animation etc.

• provides timing between media objects and streamed data

• provides for user interactivity • called (microDOM) uDOM in SVG Tiny 1.2

17

• Classic explanation of arranging primitive multimedia objects into a rendered scene on a browser never made much sense to me

• However when the primitive multimedia objects become traffic images, weather icons and information, news headlines, alerts and advertising this workflow starts making more practical sense for Rich Media Applications

18

• So in a nutshell ATSC A/153 Part 5 is all about creating and pushing multimedia objects through the Rich Media environment prescribed by the standard.

19

• push content model • push content in advance of consumption

model • return path not guaranteed in ATSC M/H • return path

• device dependent (out of band) • network availability dependent

• forward, store, use content design model • applications must be designed accordingly

20

• Weather Application • six “Multimedia Scenes” are used in the

weather app • they are transmitted all at one time • a small number of SVG fragments can be

reused over a number of scenes • judicious use of SVG fragments reduce

transmission requirements

21

• In this application ECMAScript is used to program the interactive buttons to switch between multimedia scenes

• SVG provides viewports and the buttons are programed to display 416*240 pixel slices

22

• In A/153 Part 5 W3C’s SVG Tiny 1.2 is used to create multimedia objects

23

• XML is the basis for metadata • XML-based file format is text

• editable text • human-readable, machine-readable • can be searched, indexed, and scripted

• XML markup is shown here in the creation of a raindrop object

• Note the use of the <g> </g> tags. • These are used to group a series of

elements • In this case they are used to identify the

elements that makeup the raindrop object

24

• JPEGS and PNG’s are binary based and are encoded into base 64 when placed in an SVG file.

• XML is comprised of text only • XML must be readable by people

• Example of a separate, specific or individual encoding type specified by the MPEG 4 Object Coding standard.

25

• SVG Vector Graphics Scale Well • 3X size: above rendered at 1262 * 727 – originally

designed for 416 * 240 • traffic object is originally a 293 * 220 bitmap • Bitmap does not scale with other SVG objects in

scene • SVG scales well into HDTV and beyond

26

• Multimedia scene creation is a three step project • First define and group objects • Secondly position them within the scene

27

• Scene Description • SVG simultaneously creates multimedia objects

and scene description as one authors content

28

• Third (if required) use XSLT (eXtensible Stylesheet Language Transform) to include desired data from linked data sources.

• Accomplished by adding appropriate eXtensible stylesheet language to the original SVG document.

• Appropriate objects will be added to the document and the XSLT processor will output a completed .svg file ready for the packaging format

29

• Freely available software for presentation format • any text editor will edit SVG and XSLT • integrated developer environment (IDE)

• Eclipse is most evolved

30

• SVG editors

31

• SVG Open Libraries

32

• Objects and their open source/open standards

33

• software used in weather app

34

• Originally MPEG-4 provided Scene Description and objects were pulled from multiple network locations

• Packaging came along for mobile devices that had limited on constrained network connections

35

• DIMS server for highly constrained networks and devices.

36

• Open source DIMS software • GPAC Project on Advanced Content • implementation of the MPEG-4 systems standard • Not broadcast product, distributed “as is” • Useful for prototyping only

37

• consists of • multimedia packager called MP4Box

• some server functions (experimental) • multimedia player called Osmo4 • MP4Box

• does 3GPP DIMS packaging from SVG files • Osmo4

• displays SVG Tiny 1.2 scenes packaged in 3GPP DIMS files

38

• DIMS is all about carousels • Two types

• Static and Dynamic • static carousel

• repeatedly sends scene description and objects

• repeats because devices tune in at different times

39

• dynamic carousel • sends updates to screen elements

40

• GPAC DIMS static carousel operation • Command line interface instructions for DIMS

Live streaming • DIMS generates an IP stream that A/153

transports to player

41

• GPAC OMOS 4 player • Command line interface instructions for

starting OSMOS4 • Opens IP stream that GPAC Dims server generates

42

• Traffic App uses the Static carousel to send the basic scene description only

43

• Traffic App then uses the Dynamic Carousel to update screen by sending all objects

44

• To send updates GPAC watches a text file • When new information is written to the text file

GPAC immediately sends its contents as an update

45

• To send updates GPAC watches a text file • When new information is written to the text file

GPAC immediately sends its contents as an update

46

• Linux BASH shell script to collect linked data and write the update file • BASH: wget --output-document=camera.jpg

http://www.toronto.ca/trafficimages/loc9.jpg • This line uses a BASH command that saves

the next traffic image to be sent • BASH/DIMS: a="<Replace ref=\"trafficImage\"

attributeName=\"xlink:href\" value=\"data:image/ jpeg;base64,"$(base64 camera.jpg)"\" />“ • This uses a DIMS replace command and

initiates the base64 encoding built into Linux and saves it as a string

• BASH: echo $a > updateFile.svg • This BASH line writes the string as a text file

to the update file. GPAC then sends the command.

47

• GPAC DIMS dynamic carousel operation • Command line interface instructions for DIMS

Live streaming • Name of update file added to command line

that initiates dynamic updates • DIMS generates an IP stream that A/153

transports to player

48

• Freely available software used for packaging format

49

• No freely available fully implemented SVG Tiny 1.2 standard viewer

• Makes one wonder what will find their way into A/153 Part 5 compatible devices

50

• Experimental nature of GPAC • dynamic carousel

• Prototype crashes when primary and update streams sent too close together

51

• So in a nutshell ATSC A/153 Part 5 is all about creating and pushing multimedia objects through the Rich Media environment prescribed by the standard.

52

• push content model • push content in advance of consumption

model • return path not guaranteed in ATSC M/H • return path

• device dependent (out of band) • network availability dependent

• forward, store, use content design model • applications must be designed accordingly

53

• Scene Description • SVG simultaneously creates multimedia objects

and scene description as one authors content

54

• Traffic App uses the Static carousel to send the basic scene description only

• Traffic App then uses the Dynamic Carousel to update screen by sending all objects

55

• free software used in prototype environment

56

• more information www.openmobiledtv.org

57

58