www.dharmeshmalam.com

M5EG Interactive TV in a new Era

Dharmesh Malam

14 Jun 2006

Supervisors

Arosha Bandara Ian Harries

M5EG - -

ii

Abstract

Existing Computers are now being used in new and exciting ways in convergence

devices as the centre of the digital lives of millions of people.

These ‘media PCs’ sorely lack applications that are specifically designed to utilise

this new form of experience. MHEG-5 is a standard that enables interactive ser-

vices to exist on standard digital TVs, where a vast amount of original content is

already available. By creating a MHEG-5 interpreter, it is hoped that these applica-

tions can be also made available for PC users. The richness of the PC platform

then can be utilised to add value to these experiences.

This report details the creation of a framework that facilitates such interactive

abilities.

M5EG - -

iii

Acknowledgements

I would to thank my project supervisors, Arosha Bandara and Ian Harries who both provided essential support throughout the project lifecycle. I would also like to thank my friends and family who had to deal with me during this gruelling time.

M5EG - -

iv

Contents

1. List of figures __________________________________________________ viii

2. Introduction _____________________________________________________ 1

2.1. Motivation _______________________________________________________ 1

2.2. Project Goals _____________________________________________________ 2

2.3. Report Structure __________________________________________________ 2

3. Background _____________________________________________________ 3

3.1. Interactive TV ____________________________________________________ 3 3.1.1. Introduction ___________________________________________________________ 3

3.2. Teletext __________________________________________________________ 3 3.2.1. Technical _____________________________________________________________ 3 3.2.2. Limitations ____________________________________________________________ 4 3.2.3. Availability ___________________________________________________________ 4

3.3. Digital Teletext ___________________________________________________ 4 3.3.1. Introduction ___________________________________________________________ 4 3.3.2. Digital TV ____________________________________________________________ 5 3.3.3. The UK ______________________________________________________________ 6

3.4. Interactive TV ____________________________________________________ 6 3.4.1. Introduction ___________________________________________________________ 6 3.4.2. The Systems ___________________________________________________________ 8 3.4.3. Systems used in the UK _________________________________________________ 11 3.4.4. Summary ____________________________________________________________ 13

3.5. Consumer Computer Convergence __________________________________ 14 3.5.1. Introduction __________________________________________________________ 14 3.5.2. 10 Foot UI vs 2 Foot UI _________________________________________________ 14 3.5.3. Relevance to MHEG-5 _________________________________________________ 15

3.6. Content _________________________________________________________ 17 3.6.1. Interactive advertising __________________________________________________ 17 3.6.2. Enhanced Television ___________________________________________________ 17 3.6.3. Multi-streams _________________________________________________________ 17 3.6.4. Static Information Services ______________________________________________ 17 3.6.5. Pay per View _________________________________________________________ 18 3.6.6. Radio _______________________________________________________________ 18 3.6.7. Gaming _____________________________________________________________ 18 3.6.8. EPG ________________________________________________________________ 19 3.6.9. Portals ______________________________________________________________ 19

3.7. Existing Implementations __________________________________________ 19 3.7.1. MHP _______________________________________________________________ 19 3.7.2. MHEG ______________________________________________________________ 20

4. MHEG-5 in detail________________________________________________ 24

4.1. Introduction _____________________________________________________ 24 4.1.1. The Reasoning behind MHEG-5 __________________________________________ 24 4.1.2. Why was MHEG-5 chosen for the UK _____________________________________ 24 4.1.3. MHEG-5 engine Requirements ___________________________________________ 25 4.1.4. Official Specifications __________________________________________________ 25

4.2. MHEG-5 Classes _________________________________________________ 25 4.2.1. Content Classes _______________________________________________________ 25

M5EG - -

v

4.2.2. Behaviour Classes _____________________________________________________ 25 4.2.3. Root ________________________________________________________________ 27 4.2.4. Group _______________________________________________________________ 27 4.2.5. Application and Scene __________________________________________________ 27 4.2.6. Ingredient ____________________________________________________________ 28 4.2.7. Action ______________________________________________________________ 28 4.2.8. Link ________________________________________________________________ 28 4.2.9. Program _____________________________________________________________ 28 4.2.10. Variable ___________________________________________________________ 29 4.2.11. Presentable ________________________________________________________ 30 4.2.12. TokenGroup _______________________________________________________ 30 4.2.13. Stream ____________________________________________________________ 30 4.2.14. Interactible ________________________________________________________ 31 4.2.15. Visible ____________________________________________________________ 31

4.3. The MHEG-5 Event model _________________________________________ 31

4.4. Format _________________________________________________________ 33 4.4.1. Textual ______________________________________________________________ 33 4.4.2. XMHEG ____________________________________________________________ 33

4.5. Delivery ________________________________________________________ 35 4.5.1. File structure _________________________________________________________ 35 4.5.2. Over the Air __________________________________________________________ 35 4.5.3. Application lifecycle ___________________________________________________ 36

4.6. The User Experience ______________________________________________ 37

4.7. MHEG-5 Graphical Model ________________________________________ 39 4.7.1. Graphics Plane ________________________________________________________ 39 4.7.2. Colour Palette ________________________________________________________ 40 4.7.3. Colour Space _________________________________________________________ 40 4.7.4. Font ________________________________________________________________ 40

5. Specification ____________________________________________________ 41

5.1. Overview _______________________________________________________ 41

5.2. General Requirements ____________________________________________ 41 5.2.1. Basic Functionality ____________________________________________________ 41 5.2.2. Advanced Functionality _________________________________________________ 42

5.3. Technology Requirements _________________________________________ 42 5.3.1. Basic Functionality ____________________________________________________ 42 5.3.2. Advanced Functionality _________________________________________________ 42

5.4. Amount of MHEG-5 specification to implement _______________________ 42 5.4.1. Basic Functionality ____________________________________________________ 42 5.4.2. Advanced Functionality _________________________________________________ 44

5.5. Usability Requirements ___________________________________________ 44 5.5.1. Basic Functionality ____________________________________________________ 44 5.5.2. Advanced Functionality _________________________________________________ 44

5.6. Accessibility Requirements ________________________________________ 44 5.6.1. Basic Functionality ____________________________________________________ 45 5.6.2. Advanced Functionality _________________________________________________ 45

5.7. Added Value Requirements ________________________________________ 45

6. Design _________________________________________________________ 46

6.1. Overview _______________________________________________________ 46 6.1.1. The Parser ___________________________________________________________ 48 6.1.2. The Engine ___________________________________________________________ 49 6.1.3. The Renderer _________________________________________________________ 50

M5EG - -

vi

6.1.4. The User program _____________________________________________________ 52

6.2. Technology Decision ______________________________________________ 54 6.2.1. Concerns ____________________________________________________________ 54 6.2.2. Sun’s Java ___________________________________________________________ 55 6.2.3. C# _________________________________________________________________ 55 6.2.4. MFC C++____________________________________________________________ 56 6.2.5. Reasons _____________________________________________________________ 56 6.2.6. DirectShow __________________________________________________________ 56 6.2.7. BDA - Broadcast Driver Architecture ______________________________________ 57 6.2.8. GDI+ - Graphical Device Interface Plus ____________________________________ 57 6.2.9. Direcxt3D ___________________________________________________________ 57 6.2.10. XSS4J - XML Security Suite for Java ___________________________________ 57

7. Implementation _________________________________________________ 59

7.1. Overview _______________________________________________________ 59

7.2. The Parser ______________________________________________________ 59 7.2.1. The XMHEG Parser ___________________________________________________ 61 7.2.2. The ASN1 Parser ______________________________________________________ 62 7.2.3. Optimisations _________________________________________________________ 63 7.2.4. Issues _______________________________________________________________ 64

7.3. Engine __________________________________________________________ 65 7.3.1. The Class Model ______________________________________________________ 65 7.3.2. Interpreter ___________________________________________________________ 68 7.3.3. Object Handling _______________________________________________________ 68 7.3.4. Content Reference _____________________________________________________ 70 7.3.5. Persistent Storage _____________________________________________________ 71 7.3.6. Event Processing ______________________________________________________ 71 7.3.7. Threading ____________________________________________________________ 72 7.3.8. Timers ______________________________________________________________ 74 7.3.9. Issues _______________________________________________________________ 74

7.4. The Renderer ____________________________________________________ 75 7.4.1. The IRenderers _______________________________________________________ 77 7.4.2. Issues _______________________________________________________________ 78 7.4.3. DC-DVB Source ______________________________________________________ 79

7.5. Logging framework_______________________________________________ 80

7.6. Unexpected Discoveries ___________________________________________ 80 7.6.1. Compressed XMHEG programs __________________________________________ 80

8. Evaluation _____________________________________________________ 82

8.1. Overview _______________________________________________________ 82

8.2. Specifications Adherence __________________________________________ 82 8.2.1. Class coverage ________________________________________________________ 82 8.2.2. Class Statistics ________________________________________________________ 83 8.2.3. Comparisons to existing systems __________________________________________ 83

8.3. Qualitative Testing _______________________________________________ 84 8.3.1. Rendering Quality _____________________________________________________ 84 8.3.2. Rendering Accuracy ___________________________________________________ 86 8.3.3. Coordinate Accuracy ___________________________________________________ 86 8.3.4. Usability of the M5EG application ________________________________________ 87 8.3.5. Accessibility _________________________________________________________ 90 8.3.6. User Testing __________________________________________________________ 90 8.3.7. Perceived Performance _________________________________________________ 91 8.3.8. Product Value ________________________________________________________ 91

8.4. Quantitative Testing ______________________________________________ 91

M5EG - -

vii

8.4.1. Speed of M5EG _______________________________________________________ 91 8.4.2. Speed comparisons ____________________________________________________ 93 8.4.3. Compressed XMHEG size reductions ______________________________________ 93 8.4.4. Resource Requirements _________________________________________________ 94

8.5. Overall Analysis of Project _________________________________________ 94 8.5.1. Compliance with Project Specification _____________________________________ 95

8.6. Project Statistics _________________________________________________ 96

9. Conclusion _____________________________________________________ 97

9.1. Overview _______________________________________________________ 97

9.2. Original Goals ___________________________________________________ 97

9.3. Original Contributions ____________________________________________ 98 9.3.1. The M5EG Framework _________________________________________________ 98 9.3.2. The M5EG Application _________________________________________________ 98 9.3.3. The compressed XMHEG format _________________________________________ 98

9.4. Future Work ____________________________________________________ 98 9.4.1. Finish Specification ____________________________________________________ 98 9.4.2. A plugin in for Windows Media Centre ____________________________________ 98 9.4.3. Additional Renderers and Parsers _________________________________________ 99 9.4.4. Optimisations _________________________________________________________ 99 9.4.5. Added Value _________________________________________________________ 99

9.5. Final Words _____________________________________________________ 99

10. Bibliography _________________________________________________ 100

11. Appendix A __________________________________________________ 101

M5EG - -

viii

1. List of figures

FIGURE 1 - DVB LOGO ................................................................................................................................... 5 FIGURE 2 - DVB-T ADOPTION MAP ................................................................................................................... 6 FIGURE 3 - MHP LOGO .................................................................................................................................. 8 FIGURE 4 - MHP ADOPTION MAP ..................................................................................................................... 8 FIGURE 5 - EUROMHEG LOGO........................................................................................................................ 11 FIGURE 6 - FREEVIEW LOGO .......................................................................................................................... 12 FIGURE 7 - 2 FOOT SOLITAIRE SCREENSHOT ...................................................................................................... 15 FIGURE 8 - 10 FOOT SOLITAIRE SCREENSHOT .................................................................................................... 15 FIGURE 9 - MYUK SCREENSHOT ..................................................................................................................... 16 FIGURE 10 - OPENMHP LOGO ...................................................................................................................... 20 FIGURE 11 - XLETTVVIEW LOGO ..................................................................................................................... 20 FIGURE 12 - CABOT LOGO ............................................................................................................................. 22 FIGURE 13 - REDKEY LOGO ............................................................................... ERROR! BOOKMARK NOT DEFINED. FIGURE 14 - MHEG-5 CLASS MODEL .............................................................................................................. 27 FIGURE 15 - INGREDIENT CLASS MODEL(13522-5 1997) ................................................................................... 28 FIGURE 16 - TOKENMANAGER CLASS MODEL ..................................................................................................... 29 FIGURE 17 - VISIVLE CLASS MODEL .................................................................................................................. 30 FIGURE 18 - MHEG-5 EVENT MODEL ............................................................................................................. 32 FIGURE 19 - MHEG-5 CLASS STATE MODEL ..................................................................................................... 32 FIGURE 20 - HELLOW WORLD IN TEXUAL MHEG-5 ............................................................................................ 33 FIGURE 21 - HELLOW WORLD IN XMHEG ........................................................................................................ 34 FIGURE 22 - MHEG-5 APP FILE STRUCTURE ..................................................................................................... 35 FIGURE 23 - MHEG-5 USER INTERFACE MODEL ................................................................................................ 37 FIGURE 24 - MHEG-5 STANDARD REMOTE MODEL ............................................................................................ 37 FIGURE 26 - TV CROPPING AREAS ................................................................................................................... 39 FIGURE 25 (BBC, DESIGNING FOR INTERACTIVE TELEVISION V 1.0 BBCI & INTERACTIVE TV PROGRAMMES 2006) ........ 39 FIGURE 27 - STANDARD 188 MHEG-5 COLOURS .............................................................................................. 40 FIGURE 28 - MHEG -5 COLOUR SPACE CONVERSION .......................................................................................... 40 FIGURE 29 - STANDARD MHEG-5 FONT .......................................................................................................... 40 FIGURE 30 - FRAMEWORK OVERVIEW .............................................................................................................. 46 FIGURE 31 - INTERNALS OVERVIEW ................................................................................................................. 47 FIGURE 32 - PARSER OVERVIEW ..................................................................................................................... 49 FIGURE 33 - ENGINE OVERVIEW ..................................................................................................................... 50 FIGURE 34 - RENDERER OVERVIEW ................................................................................................................. 51 FIGURE 35 - GUI MOCKUP ........................................................................................................................... 53 FIGURE 36 - GRAPHEDIT SCREENSHOT ............................................................................................................. 57 FIGURE 37 - PARSER DETAIL .......................................................................................................................... 59 FIGURE 38 - EXAMPLE XMHEG FOR A LINK OBJECT .......................................................................................... 61 FIGURE 39 - ASN1 PARSER DETAIL ................................................................................................................. 62 FIGURE 41 - MHEG CLASS MODEL (13522-5 1997) ........................................................................................ 66 FIGURE 40 - ENGINE DETAIL .......................................................................................................................... 65 FIGURE 42 - EXAMPLE C# CODE FOR INGREDIENT .............................................................................................. 67 FIGURE 43 - EXAMPLE C# CODE FOR UNLOAD .................................................................................................. 67 FIGURE 45 - FIND C# DEFINITION ................................................................................................................... 69 FIGURE 44 - EXAMPLE OBJECT REFERENCES ...................................................................................................... 69 FIGURE 46 - FIND C# IMPLEMENTATION .......................................................................................................... 70 FIGURE 47 - C# ACTION QS DEFINITION ........................................................................................................... 71 FIGURE 48 - C# ACTIVELINKS DEFINITION ......................................................................................................... 71 FIGURE 49 - C# THREADING DEFINITION .......................................................................................................... 73 FIGURE 50 - C# MAIN LOOP .......................................................................................................................... 73 FIGURE 51 - MAIN LOOP STATE MODEL ........................................................................................................... 73 FIGURE 52 - C# TIMER DEFINITION ................................................................................................................. 74

M5EG - -

ix

FIGURE 53 - RENDERER DETAIL ...................................................................................................................... 75 FIGURE 54 - RENDER OPTIMISER DETAIL ........................................................................................................... 76 FIGURE 55 - DRAWER CLASS MODEL................................................................................................................ 78 FIGURE 56 - VIDEO MIXER DETAIL .................................................................................................................. 80 FIGURE 57 - BBC TEXT RENDERING SAMPLE ...................................................................................................... 85 FIGURE 58 - BBC M5EG TEXT RENDERING ZOOM ............................................................................................. 85 FIGURE 59 - BBC 37WLT56 TEXT RENDERING ZOOM ........................................................................................ 85 FIGURE 60 - SCREEN GUI SCREENSHOT ........................................................................................................... 89 FIGURE 61 - REMOTE GUI SCREENSHOT .......................................................................................................... 89 FIGURE 63 - BOTTOM ZOOM SCREENSHOT ....................................................................................................... 90 FIGURE 62 - TOP ZOOM SCREENSHOT .............................................................................................................. 90 FIGURE 64 - GRAPH OF PARSING TIME V SIZE .................................................................................................... 92 FIGURE 65 - GRAPH OF RENDERING TIME V SIZE ................................................................................................ 92

M5EG - -

1

2. Introduction

2.1. Motivation

Interactivity with TV systems has had a long history. MHEG-5 is the latest in a long line of standards starting with Teletext that facilitates this. It has been success-fully deployed via Freeview in the UK, and much original content is available. Interactivity with computers has also had a defined history. Today many users are using computers as convergence devices to provide a ‘lean back’ experience simi-lar to consumer electronics but vastly different to the traditional ‘lean forward’ ‘WIMP’ (Window, Icon, Menu, Pointing device) user interface. There currently exists a rarity of these ‘lean back’ applications for the PC. Con-versely, a large amount of such content is available for free in the airwaves. This project will explore the possibility of providing this interactive MHEG-5 content to computers users. To facilitate this, a piece of software called an MHEG-5 Interpreter will be needed. While this will ‘run’ the MHEG-5 application careful thought needs to be taken on interaction with the user. The MHEG-5 application needs to be presented in a way that is true to what its authors indented - a ‘lean back’ experience. This project will detail the creation of a framework that will allow the execution of MHEG-5 sourced content. It is a framework in the sense that it could be used by any other application. A plugin for example could give MHEG-5 abilities to Win-dows Media Centre. The power of the computer platform stems from its ability to be extended easily and generically. This has made convergence possible. To make convergence de-sirable is the proposition of added value. This will be also the case with MHEG-5 and this project will try to explore any abilities to add value. Another motivation is that currently it is impossible for typical computer users to experience MHEG-5 content. A £1000 media centre PC cannot completely replace a £30 set top box. While certain implementations exist to allow interactive TV, they are tied down to specific hardware or do not provide an experience true to the original form. No usable interpreters exist for Windows1. It is hoped that a well designed framework will provide many tangible benefit to both to software developers and general users.

1 As of August 2006

M5EG - -

2

2.2. Project Goals

The goals are,

To implement a framework that can interpret MHEG-5 on Windows o The system should be a library. o The system should be designed with extensibility in mind. o It should stay true to goals of the standard.

To create an application that showcased this functionality: o The application should use the framework fully. o 10 foot (lean back) as well as 2 foot (lean forward) usage.

To explore any additional value: o Leverage the rich PC platform. o Explore ways to enrich the MHEG-5 user experience.

The suggested framework will be named M5EG.

2.3. Report Structure

This report begins by describing the history of interactive TV systems in Chapter 2. Time is spent not only describing the importance, but also the many different standards and implementations that exist. Chapter 3 explains in detail the inner workings of MHEG-5. It is hoped that a deep understanding of the standard will allow the implications of the created frame-work to be obvious. Chapters 4, 5 and 6 describe the journey of M5EG, from an idea to a working product. The specification, details and decisions of the design process, and any implementation issues and unexpected discoveries are thoroughly described. Chapter 7 tries to evaluate the framework by both qualitative and quantitative means. It also compares M5EG to existing solutions and summaries the good and bad of the project. Conclusions are made in Chapter 8 with summaries of original contributions and discoveries. Also suggestions are made on possible future work.

M5EG - -

3

3. Background

3.1. Interactive TV

3.1.1. Introduction

The use of TV outside its original purpose of displaying video has had a long his-tory.

3.2. Teletext

This was the first method of encoding additional information in the video signal. Devolved since 1970 by British broadcasters, it was first an exploration into em-bedding metadata into broadcast streams. Potential uses envisioned low band-width uses, for example monitoring transmitters, tracking network signals and subtitling. From further development, it became possible to encode significantly more data while not affecting the video signal. This gave a data transmission plat-form with far reaching applications. Discussion and creation of standards lead to the first trial in 1974, with further work allowed the introduction of the ‘teletext’ system to the general public in 1976.

3.2.1. Technical

In the 1970s, the BBC was researching possible ways for transmitting closed cap-tioning to audiences. Later work allowed whole pages of information to be pre-sented, by saving the text incrementally. The General Post Office (later BT) had also been developing a similar system since the 1960s called Viewdata, with the novelty of a return path via phone lines. This was obviously lucrative as additional phone charges would accompany usage. In 1972 the BBC showcased Ceefax, while a year later the ITA2 came up with another incompatible standard ORACLE3. Standards were needed and between 1974 and 1976 various technical points were specified, leading to CEPT1 and then World System Teletext (WST) - Teletext as we know it was born. Several other comparable systems were also developed, including Canadian NAPLPS, French Antiope, American NABTS (North American Broadcast Teletext Specification), American Electra, German Videotext. Most of these were not as popular as the WST system, with Antiope being replaced by WST in the mid 90s.

2 Independent Television Authority

3 Optional Reception of Announcements by Coded Line Electronics

M5EG - -

4

The US systems almost complexly failed to attract general interest as opposed to European broadcasters where text services were highly popular. Transmitted as part of the ordinarily analogue video signal, but hidden in the ver-tical blank interval. Teletext is digitally coded as 45 byte packets at the end of lines 6-22 and 318-335 (Wikipedia n.d.) resulting in 600bit s-1 of throughput. Each teletext page consists of one or more frames containing a screen full of text which are transmitted one after the other in a continual loop. The decoder in the TV waits for the particular selected page. This can be quite a while with many pages, so it is quite usual to transmit the most popular pages more than one per loop. Also modern decoder implement caching functions, to allow almost instant access to any page once a whole loop has been transmitted. Teletext was to be displayed on a monospaced 40 x 24 character grid with a choice of eight colours, with simple graphics possible with special graphics char-acters.

3.2.2. Limitations

There are many limitations to teletext, a system which is over 30 years old. The most pressing are the limited display options available to the developer like no images. It can be very slow and many possible use cases that newer systems offer are simply not possible.

3.2.3. Availability

Due to its simple nature, long history of quality content and huge installation base, Teletext has been a tremendous success. Even due to its limitations teletext at its height had over 22 Million users each week in the UK alone (ITC-Review n.d.). This was mainly due to easy access to a wide variety of very high quality informa-tion, especially important in the pre internet era. Analogue TV will not exist anymore in the near future as the digital switchover is planned to be implemented in phases from 2008 – 2012 (BBC-News n.d.). This will also mean the teletext system will not exist anymore and its service will have to be available via other means including the web and digital teletext.

3.3. Digital Teletext

3.3.1. Introduction

M5EG - -

5

A slightly misleading term, not only because the original teletext system was broadcasted in a digital manner, but also as it suggests it’s somewhat of an evolu-tionally successor to normal teletext. The following systems really have nothing in common except in that they are also used to display information on a TV screen. The digital really refers to the fact that these systems are always associated with digital TV transmission, where as teletext is usually found on analogue TV sys-tems. There are mechanisms to encode the old teletext data with digital TV sig-nals by emulating the Vertical blank Interval4.

3.3.2. Digital TV

Available throughout the world through various means, digital TV (DTV) is a para-digm shift in broadcasting technology compared with earlier analogue TV used since mid last century. It is a means of converting video information into digital signals and a system of transmitting and receiving them though various medians. By design digital TV provides a richer experience by providing better quality video, multiple higher quality sound channels including surround sound, support for HDTV, and more channels. This is all possible due to compression technologies to squeeze more information into the set bandwidth available for TV. As always, standards need to be created and again we have different competing but comparable systems. In the US the Advanced Television Systems Committee created the ATSC standard to replace NTSC (ATSC n.d.). It has since then been also used in Canada, Mexico, and South Korea and is used only for terrestrial broadcasts. Japan has been using ISDB (Integrated Services Digital Broadcasting) system (Wikipedia n.d.), and the rest of the world has settled on another incompatible system – DVB.

DVB (DVB n.d.) (Digital video broadcasting) comes in several open flavours and enjoys large acceptance in Europe and also in the US in cases where ATSC is not usable like cable and satellite. The DVB project is an industry-led consortium of

over 270 broadcasters, manufacturers, network op-erators, software developers, regulatory bodies and others in over 35 countries with 120 million DVB receivers deployed. There exists the DVB-S(2) standard for satellite transmission, DVB-T for terrestrial, and DVB-C for cable while standards like DVB-H exist also for TV on mobile phones. The DVB project tries to standardise as much as possible including the

4 DVB-TXT ,DVB-VBI do this

Figure 1 - DVB logo

M5EG - -

6

physical and data link layers, the modulation schemes, and compression tech-nologies. They also recommend using MHP for interactive systems, This diagram5 illustrates world wide acceptance of the DVB-T standard.

Figure 2 - DVB-T adoption map

3.3.3. The UK

In the UK, the DVB standards are almost exclusively used, and digital TV is avail-able via 3 means

- Satellite via Sky Digital - Cable via Cable operators (Telewest/ NTL) - Terrestrial via Freeview platform.

Also IPTV and mobiles applications can receive TV, but these are generally niche segments. Each transmission method specifies various parameters in ‘concrete’, for example the exact resolution of video, the exact compression system and compression op-tions and so on. They also specify what interactive TV systems will be used.

3.4. Interactive TV

3.4.1. Introduction

With digital TV information services have been thought of from the beginning and not ‘hacked’ on like teletext. Accordingly these services provide a significantly richer environment and hence they have been labelled interactive TV or iTV.

5 http://www.dvb.org/graphics/internal/Adoption-Map_DVB-T.jpg

M5EG - -

7

The term interactive television can be used to represent a number of different types of interactive scenarios. At its most basic level it refers to basic interaction primitives such as changing channels, adjusting volume, but also more advanced use cases like pausing live TV, rewinding / fast forwarding live or recorded TV and the like. What we are most accustomed to is interacting with additional content provided with the TV signal. This includes interacting with related TV content or ‘coactivity’, where we get more information that is contextual to what is on screen like re-views of the current movie showing, more details of an advertisement, etc. We also have interacting with non contextual data or experiences unique and self contained similar to most of the teletext era. In a more advanced sense, interactive TV could be used to interact with the actual program content. For example users could choose the ending of a drama, or news stories that are customised for the user. While this has not yet occurred in a mass scale, simpler forms exist like programs incorporating polls and viewer voting. The term ‘lean back’ is often used to describe interactive TV as users are generally experiencing it from living room environments with remotes controls being the primary interface. This is opposite to ‘lean forward’ experiences available on a PC or laptop. The terms 10 foot UI and 2 foot UI from the computer world are equiva-lent and are described later. iTV can be used in many original ways unthought-of in the teletext era. Viewers can interact with TV content, respond to a commercial, play games, take part in TV programmes, make purchases and do activities not imagined yet. Broadly speaking iTV allows broadcasters and advertisers to provide services that are not only highly targeted and relevant but also convenient and accessible at any time. Primary used are:

- Interactive advertising – more information for interested customers. - Enhanced television – voting in an a TV programme, extra contextual

information, extra video feed in a sports programme. - Textual services – pages of text information similar to teletext eg

stocks, weather, horoscopes, news but enhanced with high resolution graphics and video/audio.

- Entertainment - Gaming / Betting and so on. These services are possible because new standards for encoding interactive appli-cation have been created which provide a rich application model, quite similar in architecture to what is provided to ‘regular’ programmers by Windows/ Linux etc.

M5EG - -

8

Also a return channel is optionally specified allowing two way communication, and making some of the above use cases possible. This can be a connection to a phone line, mobile SMS, internet (ADSL) or cable and allows data to be sent from the user back to a service provider.

3.4.2. The Systems

The most popular systems deployed worldwide are discussed below. MHP Multimedia Home Platform (DVB-MHP) (MHP, Multimedia Home Platform Specification n.d.) is an advanced open mid-dleware standard for interactive TV as defined by the DVB pro-ject. Development started in 1997 and release 1.1.1 was ap-

proved in April 2003, with 1.1.2 being currently drafted. Its ar-chitecture specifies an extensive execution environment com-prising of a Java VM and some generic APIs. This open platform then allows ven-dors to highly customise deployments for specific country implementations. MHP defines a layer of interfaces that can sit on top of any operating system, mostly likely a set top box, and allow applications to run. MHP is part of a family member of the Globally Executable MHP (GEM) Standard which tries to define global adoption. MHP provides two application development models, firstly there is DVB-HTML which is based upon a modularised version of XHTML 1.1 including CSS 2.0, DOM 2.0, ECMAScript, and secondly DVB-J based on small java applets called Xlets that have access to the MHP API. These are considerably more popular due the late standardisation of DVB-HTML and the difficulties to author both DVB-HTML appli-cations and DVB-HTML engines. A return channel is optionally available which al-lows application to communicate back. MHP enjoys large scale deployment in Italy, Korea with smaller access in Germany, Finland, Spain and Australia. The cable OCAP middleware system in the US is also largely similar to MHP.

Figure 3 - MHP logo

Figure 4 - MHP adoption map

M5EG - -

9

OpenTV Core This is a complete set top box based middleware platform for a complete digital TV experience from OpenTV (OpenTV n.d.). In this respect it provides a generic software layer for all digital TV functions, such as decoding, scrambling, PVR6 functions etc. It has been deployed to great success by Sky Digital in the UK in their ‘digiboxes’. Sky customise the OpenTV middleware to provide a unique user experience (things like branding, help, program guides), this is then licensed to set top box manufactures, such as Grundig, Pace, and Amstrad. The underlying hardware may all be different, but because of the middleware hardware abstraction layer a con-sistent user experience is maintained. Of interest to us it what OpenTV Core provides for interactive TV applications, and in this respect it gives the developer a wide degree of potential choice. The pri-mary application model is based upon ANSI C which is compiled to what is called o-code. This is then interpreted by OpenTV Core, allowing different underlying ar-chitectures to run identical executable code. An extensive user library of routine functions is provided called, the OpenTV library, and works on a asynchronous message-based model. In addition to ANSI C, OpenTV provides optional ‘packages’ which can allow HTML, MHP or Flash based application to run. These are provided to give cross-compatibility to broadcasters, and also allowing then to use a potentially more familiar environment. MHEG An acronym for Multimedia and Hypermedia Experts Group and pronounced “emheg”, MHEG is a family of open standards used for coded representation of multimedia information. It grew out of the increasing convergence of broadcast and interactive technologies. The MHEG group has developed these standards over the years and has presented them for ISO standardisation. It has had several versions over the years. Standard Description

MHEG-1 - Object representa-tion-base International stan-dard (ISO/IEC13522 1997)

A generic standard for encoding multimedia ob-jects without explicit understaffing for the under-lying platform or hardware. In ASN1 (13522-1 1997)

MHEG -2 As MHEG-1 with alternative SGML notation. Has been withdrawn.

MHEG-3 Added a script interchanged language to MHEG-1

6 Personal Video Recorder

M5EG - -

10

MHEG-4 Standardised registration procedure for identify-ing objects

MHEG-5 - Support for base-level interaction (13522-5 1997)

A simplified version of MHEG-1 specifically de-signed for high performance on embedded TV applications with low resource requirements.

MHEG-6 - Support for en-hanced interactive applica-tions

As MHEG-5 but extending the declarative ap-proach with abilities to execute procedural code

MHEG-7 - Interoperability and conformance testing

Draft : As MHEG-6 but with tighter specifications to increase compatibility between implementa-tions

MHEG-5 is currently the only MHEG standard with wide use, and it is expected that future development in MHEG will be merged with MHP. I.e. MHEG-6 and 7 will probably never be used by themselves. MHP was created from the start to be an open worldwide standard for set top boxes, unlike MHEG which grew into that role. (MHEG can be used in many other scenarios apart for interactive TV, such as controlling AV equipment). It many ways MHP is an evolution of MHEG especially for the interactive TV market. We shall concentrate on MHEG-5 as this is what UK digital terrestrial transmission uses and also most importantly is what this project is about. A detailed description of MHEG-5 is included in a later chapter. MHEG-5 provides a lightweight means of handling multimedia content and con-trolling object presentation and user interactions. It is an object oriented declara-tive language that is usually transmitted via ANS1 (Abstract Notation Syntax) to pieces of software called MHEG engines which can interpret these programmes. While defined as lightweight, (this is relative to say Java or a computer based mul-timedia framework like QuickTime), it provides extensive support for nearly eve-rything you could want to do on a TV environment, including managing multiple video and audio streams. The specification also allows an optionally return chan-nel, but this has not been deployed in the UK implementations. The official MHEG-5 specification deliberately does not define many parameters; it is up to the regulation system in the country being deployed to make concrete all undefined behaviour. In the UK this is the DTG, and they release documents called UK Engine Profile (BBC, Digital Terrestrial Television MHEG-5 Specification Version 1.06 2003) that fully specify every last detail so that an interpreter can be created. The MHEG-5 programmer is equipped with a range of multimedia primitives, from audio / video, to line art, images and text, each of which is an object. These can be combined together in an event based model to respond to interaction. Some tools are available for aiding MHEG-5 development, but many are written in-house. (The BBC and Teletext Ldt do this).

M5EG - -

11

EuroMHEG - By DTG- and DigiTag (DTG, EuroMHEG DTG & Digitag 2001) While not a complete different version of MHEG, like say MHEG-6 or 7, EuroMHEG is more an advanced profile, like the UK engine profiles, but targeted for the entire European Market. Due time pressures the UK Engine Profiles left out unnecessary feature of MHEG-5, but allowed future extensions to be possible. EuroMHEG provides these. We have a number of new features

- A return channel provided by various means, ( Ethernet, modem etc) - A European font - Font downloading - Generic access to CA(ft) and SI(ft) for enhanced EPG functions - VCR control functions

EuroMHEG is being billed as a migration path from MHEG to MHP, and can be executed in MHP systems as a plugin. While version 1.0 of the specification has been released, no implementations have occurred. Other systems BD-J – A very new Java based system for interaction on Blu-Ray devices MediaHighway – Combination of MHEG and Java used by Canel+ BML – Used with ISBD in Japan Liberate - An Html based system by Liberate Technologies, used by Cable compa-nies.

3.4.3. Systems used in the UK

The UK has traditionally been very important market for interactive services, not only because of the creation and historical success of the teletext system but also of the early adoption of digital TV. A lot of quality content has also been provided on teletext by the BBC, and Teletext Ldt. With the digital switchover users expect at least the same benefits as before, but they also want enhanced experiences that give obviously perceivable benefits. This was especially difficult as DTV was launched very early in the UK, from 1997 onward and many of the above standards for providing iTV had not fully matured For example MHP was not available in a usable form until 2001 (MHP, MHP press release 2001). Also in the late 90s hardware was less powerful and more expen-

Figure 5 - EuroMheg logo

M5EG - -

12

sive than today. Both issues meant appropriate decisions had to be made on what standards to use. The following is a breakdown of interactive TV platforms used in the UK listed by transmission medium used Terrestrial DTT (Digital Terrestrial Television) was launched on 15 November 1998 with 6 groups of channels called multiplexes. These multi-plexes implement DVB-T and can carry pretty much any mix of video, audio and interactive steams. The DTG (Digital Terrestrial Group) standardises the way in which digital TV is transmitted over the air while the iTC (Inde-pendent Television Commission) split the multiplexes among the main broadcast-ers in the UK. Existing analogue broadcasters got half a multiplex each, and the remaining capacity was auctioned off to the highest bidder. ONdigital was formed, later ITVDigital, using the auctioned capacity to transmit pay TV. This proved unsuccessful for a variety of reasons and in 1 May 2002 it col-lapsed. Since then Freeview consortium consisting of the BBC, Crown Castle Interna-tional, and BSkyB, bid and won for the rights lost by ITVDigital. They launched on 12 October 2005 with just free TV channels. TopUp TV was launched in May 2004 to provide a limited pay TV service designed to supplement Freeview. Today over fifty free to air TV channels and twenty radio channels in addition to around 10 pay channels are available via DTT. DTT, especially Freeview is important because it is a direct replacement for ana-logue TV, and after the switch over (2008 -2012) DTT will be the only way to re-ceive TV for free over the air. The switchover will also provide additional band-width, which could be either sold, or used to provide additional services like HDTV or more channels. Due to an early launch, the DTT was quite limited on choices for an interactive TV system. One would expect MHP, the DVB recommendation to be used, as other-wise the DVB-T specification has been followed exactly. The reason for the use of MHEG-5 was that MHP was not standardised in 1998. MHEG-5 was recommended by DVB-T for early adopters, due to its resource friendly nature and wide availabil-ity. Benefits

Freely available.

Figure 6 - Freeview Logo

M5EG - -

13

Established open standards for transmission.

Low resource requirements.

Large user base.

Easy installation.

New TV have Freeview decoders, hence MHEG-5 built in (IDTVs).

Usable in mobiles application (Cars), and flats/apartments.

MHEG-5 is stable and many development tools exist. Limitations

Lack of bandwidth available, limits channels, interactively.

No return path, limits interactivity. Audience

75% of UK population can receive DTT (DTG, DTG press release 2006)

Will be close to 100% as digital switchover approaches.

Over 10m receivers sold, as of March 2006

Used in 6.4M UK homes (very fast growth, estimates suggest Freeview penetration will excess Sky digital by the end of 2006)

Other Systems System Features Benefits Weaknesses

Satellite Reaches 97% of homes, 7.89M subscribers

Return channel Proprietary closed platform

OpenTV High Bandwidth Not suitable for mo-bile use

Cable 3.3M homes receive

cable (Ofcom December 2005)

Other services can be included like phone, internet

Proprietary closed platform

Liberate system to transmit interactive ap-plications, XHTML based

An always on return path is available, with high bandwidth

Not suitable for mo-bile use

3.4.4. Summary

This project will be focussed on the MHEG system used by DTT because of its open nature and very high penetration in the UK. Sky’s system is encrypted and hence not usable from a PC; cable is also similar. DTT is freely available to anyone with a TV card and the interactive applications are not encrypted and so available to all. There is also a large amount of content available (see later chapter) on MHEG-5 that makes it desirable to have an interpreting engine for it.

M5EG - -

14

3.5. Consumer Computer Convergence

3.5.1. Introduction

We are moving closer to a world in which every electronic device will be inte-grated. This recently has resulted in the colliding of consumer electronics and IT industries and one of its main contributions has been the media hub concept.

This is a device that can be used to aggregate all digital media in one’s life, includ-ing video, audio, photos and anything else. Consumer electronics companies as well as computer companies have been creating products that service this pur-pose.

3.5.2. 10 Foot UI vs 2 Foot UI

Now many people having using standard computers to watch videos and play mu-sic for many years. However while this is part of the integration landscape it is only a sub set. This is called 2 Foot interaction or “lean forward”. It is generally seen as the traditional way of interacting with computers, sitting in an upright po-sition at a workstation using a mouse and keyboard. The user is approximately 2 feet away from the screen.

10 Foot or lean back is the interaction style most associated with TV viewing. It is sitting is a relaxed position, on a sofa perhaps, and interacting with a big screen, mostly likely using a remote. It is also likely to be not solitary, unlike the 2 Foot which almost always is. Now while each domain used to be completely separate, with TVs, VCRs and set top boxes almost completely providing the 10 foot experience, many users today are attaching their computers to TVs and using them as media hubs. Obviously the standard user interface is not suitable in these circumstances. Rea-sons include the low resolution of many TVs and the larger viewing distance mak-ing it much more difficult to discriminate small items. The remote also does not lend easily to the WIMP interaction typical of mice and keyboards. However specific software has been created for the purpose of giving a 10 foot UI to the computer world. These include Windows Media Centre, Apple’s FrontRow and MythTv available on Linux. Many also give SDKs allowing the third party de-velopers to easily create 10ft applications.

M5EG - -

15

This is Solitaire in a 2 foot form.

This is the same game, but in a 10 foot form for Windows Media Cen-ter (Salloway 2006).

3.5.3. Relevance to MHEG-5

What is important to note is that MHEG-5 was designed to be always used in a 10 foot environment. Hence all broadcast applications are made with appropriate considerations. To use MHEG-5 applications in their intended manner would also require a 10 Foot interface, a detail that most computer MHEG-5 implementations ignore. This pro-ject has taken this consideration from the start. Also a very nice consequence arises. Computer applications that want to provide a 10ft experience need to have the user interface completely re-architected, like the Solitaire game above. Due to the newness of developing 10 foot applications on the PC, such applications are quite rare as compared with 20 years of 2 Foot appli-cations.

Figure 7 - 2 Foot Solitaire Screenshot

Figure 8 - 10 Foot Solitaire Screenshot

M5EG - -

16

There exist many new 10 foot applications that try to provide information in a TV digestible form. For example the third party MyUK application for Media Centre aggregates the BBC news website amongst other things.

The full implications of a MHEG-5 engine are that it would instantly provide access to a large number of very high quality 10 Foot applications. Another important factor is that MHEG-5 can give unique experiences, like inter-action with video streams that are hard to achieve with current computer 10 foot frameworks. A PC based MHEG-5 interpreter is only really useful to the general public if it can be available in 10 foot form. There is no point using an MHEG-5 application to check weather when a web site would be quicker. It is only useful if you are in liv-ing room and want to check the weather. Many previous solutions have failed to grasp this distinction.

M5EG, by being a framework will allow both forms of usage.

Figure 9 - MyUK Screenshot

M5EG - -

17

3.6. Content

There are a variety of services available via the MHEG-5 DTT interactive TV plat-form. This section aims to explain what is available in the hope of convincing why an MHEG-5 engine would be useful on the PC platform.

3.6.1. Interactive advertising

These are auto boot applications that start when an ad is being played and allow the user to request extra information. For example they can take the user to a special MHEG-5 site with more detail about the product. Case studies suggest such advert can be more successful.

3.6.2. Enhanced Television

This encompasses use cases which provide contextual interactivity with respect to the current TV programme. For example with the TV programme “Test the Na-tion” the user can answer the questions along with the program, and then the in-teractive MHEG-5 application can tally the scores at the end. Voting application are not currently possible because of no return path, this may be rectified in the future versions of the UK Engine Profile.

3.6.3. Multi-streams

Multiple video streams are one of the killer applications of interactive TV. There are many popular use cases, for example News Multi-Screen, where users can se-lect what story they want presented. Also providing extra coverage at special events is popular, like providing extra angles in football matches, different sports at Olympics, and different courts at Wimbledon.

3.6.4. Static Information Services

This is the new version of the Teletext system, a kind of super teletext in that in-teraction is richer as full colour high resolution images are available, and elaborate animations and menu structures can be created. Also there is no limit on the amount of information available, and once the MHEG-5 application has started there should be no noticeable delay in accessing information. While MHEG inherently has a dynamic structure, you navigate to other pages via a menu system, the usability benefits of the old teletext page numbers proved too useful. Accordingly page numbers have been ‘added’ to most current MHEG-5 ap-plications.

M5EG - -

18

3.6.5. Pay per View

Pay per View is different from pay TV in that viewers pay once instantly per event instead of monthly subscriptions and contracts. It is implemented on the DTT plat-form in a rather ingenious way. While normal pay TV on DTT uses strong encryp-tion and users have to buy a card to decrypt the signal, this would be too cumber-some for pay TV applications as most users do not have a viewing card and hence the opportunity cost is too high. It would also stop impulse purchases as the ma-jority could not instantly view the event. This is in contrast to Sky and Cable where all users have subscriptions and hence they can implement Pay per view in the traditional way, encrypting the signal, and charging the user on their bill. The way it works is as follows. The video signal for the pay per view event is sent unencrypted over the air, but it PID (Program Id) is coded erroneously. The PID is what tells the receiver the type of a particular stream in the multiplex, be it video, audio or data. Because the PID is not encoded the video signal cannot be detected on receiver boxes. Potentially it is possible to manually tune into the frequency of the video stream and manually enter its PIDs, but these advanced tuning options are not available on most receivers. Now how does the system let users who have paid get access to the signal? The answer is an MHEG-5 application which prompts the user for a code and then manually tunes the box to the right frequency. The user obtains the code from phoning the broadcaster and paying. Payment systems have also been created by sending text messages. The cost of the text included payment for the pro-gramme, and the user is sent back the unlock code. The code is changed every day, and uses hashing techniques to make key genera-tion difficult. It is security through obscurity. An MHEG-5 engine for the PC would allow access to these otherwise unavailable programs.

3.6.6. Radio

Because of their low bandwidth demands many radio stations can be broadcasted on the DTT multiplex. Radio on Freeview now has an associated MHEG-5 applica-tion which shows extra information related to the radio stream, like station brand-ing, the current DJ, now and next information and the current artist / song. It is also possible to display any arbitrary information like competitions and contact phone numbers as with any MHEG-5 application.

3.6.7. Gaming

M5EG - -

19

Yooplay Games is the main provider of gaming on the DTT platform. They provide entire games written in MHEG-5, playable for a fee. Games include Tetris, Darts , Wheel of fortune and Solitaire. Each game is a complex MHEG application using advanced features line art. In this respect they may be the most complicated MHEG-5 application available.

3.6.8. EPG

A 7 day electronic program guide is broadcasted on DTT, and while not encoded in MHEG, it is another data stream which is delivered using the same means.

3.6.9. Portals

There are three portal like applications on DTT, which aim to provide as much in-formation as possibly including:

News headlines

Latest sports updates

Weather reports

Entertainment and cinema listings BBCi is a branding name for BBC flagship interactive service. It is available on DTT, Sky digital and Cable and so has been ported to all three underlying technologies. It is the replacement for Ceefax and includes the same information on the same page numbers SkyText is Sky’s interactive DTT service and very different to what Sky provides on satellite. It provides similar information as BBCi. Teletext LTD provides a new service available on ITV and associated channel. It is indented to be the replacement for the Teletext branded teletext service.

3.7. Existing Implementations

Existing implementations of interactivity systems are discussed below. Two groups are evident, closed source middleware solution for set top box manufac-tures and open source solutions for use with a PC.

3.7.1. MHP

Open Source OpenMHP (OpenMHP n.d.)

M5EG - -

20

A fully open source project for the PC aiming to “…produce an MHP (Multimedia Home Plat-form) compliant implementation of classes re-

quired by MHP specification.” It is supported by Axel Technologies and TUCS (Turku Centre for Computer Sci-ence). Primarily designed to aid development of MHP applications it can be used to create an interpreter to view Xlets. It has been quite successful and supports a substantial amount of the MHP spec.

XletView This is also an open source GNU project for emulating Xlets written in Java. De-signed primarily for viewing Xlets it im-plements about 60-70% of the MHP classes.

3.7.2. MHEG

Open Source: mhgegWWW (Euchner 1997) PRZ/TU Berlin released a project in 1997 called ‘Integrating of MHEG into the WWW’. Its aim

“... to write a MHEG5-Engine in pure Java and to evaluate the usability of MHEG as a framework for teaching applications ...”

They wanted to investigate the possibility of using MHEG-5 over the web, instead of say HTML as an interactive platform, and were not really concerned with de-coding actual interactive TV. The idea was by having an MHEG-5 system available on the web would allow TV applications to be ported easily. Against this goal it was successful, with demo programs which load up in a browser. However since the engine was not designed to be used to decode TV streams, it cannot be used to view transmitted MHEG applications. Furthermore since it only implements the pure MHEG standard and not any regional profiles, it cannot be used without significant modification to view over the air UK MHEG-5.

Mu (Barham 2005) This project was undertaken by Steve Barham (Imperial Computing) in 2005, and it main aim was to

“…to write, in Java, an interpreter for MHEG-5 applications…”

Figure 10 - OpenMHP logo

Figure 11 - xletTVview logo

M5EG - -

21

“… Mu will allow users to access interactive content, which will be extracted from a digi-tal television source…”

This project was successful in that dumped MHEG programs were able to be suc-cessfully interpreted under Linux. Video and audio were not implemented, dis-tracting from one of interactive TVs major user scenarios, enhanced TV. One of it significant breakthroughs was the creation of a XML notation for MHEG-5 (called XMHEG) in addition to the official ASN1 and textual representations. Techniques were found to translate from ASN1 to XMHEG, using a freely available library, and a DTD for XMHEG was also published. XMHEG is particularly useful as discussed later.

RedButton (Kilvington 2006) A GPL open source MHEG-5 engine usable on Linux written by Simon Kilvington It is structured as follows..

“RedButton consists of two parts. The first, rb-download, allows MHEG data to be downloaded from a DVB service. The second, rb-browser, allows the downloaded MHEG applications to be displayed.”

Rb-download serves as the DSM-CC used to demultiplex the data content from the transport streams, and rb-browser is the engine used to display the MHEG page. It is a client server architecture allowing each to be component to be split across a network. So in an example usage scenario, one computer could have rb-download running and serving MHEG application to any other computer running rb-browser. This is a nice feature even if not commonly used.

It has some limitations, notably

“It’ll only draw text, rectangles and bitmaps at the moment and some of the Elementary Actions are not yet implemented. …” “The main things missing are video and audio streams …”

It is written in C and is actively being developed - video/audio support should be added soon. However in its current form it has not been integrated into any 10 foot applications so is a standalone application and mainly used just for inquisitive purposes.

libfreeMHEG (libfreeMHEG n.d.)

MythTV is a 10 foot media centre application running on Linux and amongst other media orientated features has support for DTT TV. libfreeMHEG is a patch for MythTV that gives it MHEG interpreting abilities. Since it is a patch and not a

M5EG - -

22

standalone application, MythTV can provide a 10 foot UI and video presentation abilities staying true to the envisioned MHEG-5 usage. Written in C++ using the X-Server for rendering and supporting most of the UK Engine Profile, it can run most transmitted applications reliably. Currently it lacks video scaling amongst other minor issues. It is only available to users of MythTV on Linux and does not include any extra value features.

Commercial:

Hauppauge Interactive TV This was a plugin available for the WinTV application used to watch TV on Haup-pauge DVB-T TV cards. It runs on Windows and is only compliant with UK Engine profile 1.05, making it not fully compatible with currently broadcasting data streams. For this and other reasons it has been pulled from Hauppauge software offerings and is no longer available for download.

DigiTv by Nebula This is again specific software for Nebula TV card hardware. It is however compli-ant with version 1.06.

Mercator MHEG-5 Engine Developed by Cabot Communications Limited (Cabot 2006) Mercator is a complete SDK for an MHEG-5 implementation primarily for set top boxes. Set top box manufactures can licence Mercator and port it to their hardware. In return they get a robust MHEG engine that is

fast and gives reliable content rendering. Cabot claim that over 50% of Freeview receiver uses their engine and that Mercator is the fastest in its field.

Mercator is a closed source application and has not yet been deployed on the PC landscape, so its benefits are not available to all. RedKey by Digital Television Software (Redkey n.d.) This is another middleware solution aimed at set top box manu-facturers. It main claim to fame was that it was the first engine to fully support UK Engine Profile 1.06.

Figure 12 - Cabot logo

Figure 13 - Red-

Key logo

M5EG - -

23

They estimate they have shipped 2.5millions units and note improved portability and reduced system resources as leading features. Again no implementations have been seen on the PC. HyperPanel Lab and Ocean Blue Software also provided similar MHEG middleware and have the same problems as describe above .

M5EG - -

24

4. MHEG-5 in detail

4.1. Introduction

This sections aims to provide a thought working model of MHEG-5. This is pro-vided to help understanding of later sections.

MHEG-5 was created by the Multimedia and Hypermedia Experts Group to create a standard method of storage, exchange and display of multimedia presentations. Created in November 1994, it has been a DIS (Draft International Standard) since December 1995, and was promoted to a full international standard in July 1996[12].

Its major goals are to:

Provide a lightweight and user friendly multimedia framework.

Be open, standardised and hardware independent.

Use minimal resources (memory and CPU).

Define a final form of presentation optimised for interactive purposes.

Be extendable in an elegant way, maintaining backward compatibility.

Be used apart from iTV, like over the web and on CDs.

4.1.1. The Reasoning behind MHEG-5

For a framework to be successful it must appeal to the largest number of people possible. While there are many proprietary multimedia frameworks available these have restrictions and hence cannot serve the maximum number of people. MHEG was designed to an open standard that could work in a horizontal market. While there are many different MHEG-5 engines, and each is running on different hardware with different resources, any MHEG-5 application will work on them.

4.1.2. Why was MHEG-5 chosen for the UK

In 1996 in the UK the DTG (Digital TV Group) were trying to standardise digital ter-restrial TV for a late 1997 launch. They carried out an exhaustive evaluation of available APIs with MHEG-5 being technically superior to others for their horizon-tal domain, with low resource requirements especially important in 1996. Many platforms, like MHP described before did not exist at the time. The DTG created the UK Launch Profile (version 1) standardising the loose pa-rameters of MHEG-5, and applications development could begin. The latest UK Engine profile is 1.06 released in 2003. [13]

M5EG - -

25

4.1.3. MHEG-5 engine Requirements

MHEG-5 was designed to be highly resource efficient. The memory requirement of an engine can be as low as 300KB if implemented in a very low level way (ie in as-sembler or low level C). The processor requirements are similarly low, with a 40 Million Instructions per second (MIPS) combined with 4MB of RAM and 2MB Flash Memory being the minimum. This is compared with MHP which could need 200 MHz and as much as 32MB of Ram. These are the minimum requirements but many engines use a lot more proving faster interaction and other features such as caching giving a better experience. One of the goals of this project is to see what extra value can be added to MHEG viewing considering we have infinitely more resources and a richer programming environment to play with.

4.1.4. Official Specifications

There are two important specification documents for MHEG.

ISO Standard BS ISO/IEC 13522-5 Coding of multimedia and hypermedia informa-tion: Part 5. Support for base-level interactive applications [12]. This provides the essence of the MHEG-5 language and defines all important con-structs and behaviours. The UK Engine Profile: (Version 1.06 is latest) [13]

This is a closely specified profile of MHEG-5 developed by the DTG in the UK. It is based on ISO MEHG-5 and specifies a DSM-CC object carousel for data delivery. It gives detailed profiles of these two specifications and gives standards rules for all aspects.

4.2. MHEG-5 Classes

The MHEG-5 model is declarative and object orientated and the different types of classes are described below.

4.2.1. Content Classes

These encapsulate different pieces of multimedia data, like video, audio or im-ages. If the data is small, like a text string, it is stored in the object itself; other-wise a reference is included to a file stream object.

4.2.2. Behaviour Classes

These classes provide the interaction and control of multimedia primitives. For example they include classes to encapsulate different type of event, such as a

M5EG - -

26

user pressing a button or content being available. There are also classes to encap-sulate actions, like tuning to a new channel or selecting a piece of text, and classes to link the two together.

Each MHEG class is made up from the following properties o Exchanged Attributes

o This is data that is actuality transmitted representing properties on the class.

o Internal Attributes

o This is data that is available at run time and not transmitted. For exam-ple an object may have a property which details what state it is cur-rently in. These may also be constants that are defined in the specifica-tion and hard coded in the engine, for example default text colour if none is exchanged

o Inherited attributes

o These are either exchanged or internal attributes that a class inherits from its parent

o Events

o These events are generated from this class, and may have associated data

o Internal Behaviours

o These are analogous to private methods and describe semantic re-quirements for internal operations in the MHEG engine.

o Actions Behaviours

o These are like public methods, and describe behaviour targeted specifi-cally at this object.

(In the following diagrams7, Bold referees to concrete classes, whilst plain is Abstract)

7 ISO/IEC 13522-5 21

M5EG - -

27

Figure 14 - MHEG-5 Class model

4.2.3. Root

The super class of all other classes, it provides a mechanism for object identifica-tion and specifics the general semantic for the basic operations common to all classes, (preparation/ destruction / activation / deactivation).

4.2.4. Group

The Group class’s main purpose is to contain a collection of objects for exchange between entities, ie transmission. In that respect it is similar to a <<set>> in object oriented terminology. Group groups objects of class Ingredient, and each ingredi-ent is contained in exactly one group. Each object in the group is unique and can be reference from objects in other groups. Groups can also have timers, which can be set to trigger events.

4.2.5. Application and Scene

Application encapsulates the concept of a running application, while Scene allows spatially and temporally co-ordination of presentations. There is a semantic restriction that only one application can be active at one time. Each application has one or more Scenes associated with it, and again only one scene can be active at anytime. The Application object is the only entry point into a MHEG-5 application. Objects contained in the application are globally referable and visible. Scenes have the TransistionTo action which allows graphical transition between Scenes. An Application is started with the Launch action and destroyed with the Quit action. This also terminates the active Scene.

M5EG - -

28

Figure 15 - Ingredient class model(13522-5 1997)

4.2.6. Ingredient

Ingredient is an abstract base class representing items which Groups can con-tains. Its subclasses define all the various elements within a MHEG-5 program and the Ingredient class defines the main generic functionality of these objects.

4.2.7. Action

An Action object contains an ordered list of elementary actions that represent one consequence on one specific target object. For example one Elementary Ac-tion could change the colour of the string in a Text object. Since an Action object has many elementary actions, a sequence of consequences can be expressed. Each Elementary Action has a reference to what object it acts on. Executing an elementary action corresponds to calling a method of an object in procedural ob-ject oriented programming.

4.2.8. Link

Link objects are very important because they express the behaviour of MHEG-5 applications. Links have a condition and an Action object, and like any object in-heriting from Root can be active or not. The condition exchanged attribute de-scribes what Event on what object the Link attaches to. For example a Link may attach to the IsDeleted Event of Text object. When the Event occurs and the Link is active the Link is said to ‘fire’ and each of the Elementary Actions in its Action object is executed.

4.2.9. Program

The Abstract Program class encapsulates the functionality of calling procedural code from within the MHEG-5. It exchanges a list of parameters of the Variable class which are passed to the procedural program. Resident Program

M5EG - -

29

Resident Program is the only concrete class of Program allowed in the UK Profile. It encapsulates a predefined list of procedural methods that are stored on the en-gine. Examples include the GetDate function that returns the data in the supplied vari-ables. A full list of methods is in UKEngineProfile 1.06 [13]. Palette, Font, and CursorShape These classes are not available in the current UK Profiles.

4.2.10. Variable

The abstract Variable base class provides the possibility to store and retrieve data that can be referenced. The five concrete types are:

BooleanVariable – Stores a true or false

IntegerVariable – Stores a 32 bit signed integer

OctetStringVariable – Stores a string encoded in base64 (1 byte per char-acter)

ObjectRefVariable – Stores a reference to another object. It consists of a Group identifier, which describes which group the item is in, and an object identifier (an integer) which uniquely identifies the object within a Group.

ContentRefVariable – This references external data for example images or text files. It is a string which usually maps a file system location but can re-fer to video streams by a URL style notation.

Figure 16 - Tokenmanager class model

M5EG - -

30

4.2.11. Presentable

Presentable is an abstract base classes for Ingredients that be directly seen or heard. One of Presentable’s exchanged attributes is the content itself, either in-cluded or provided as a reference.

4.2.12. TokenGroup

TokenGroup encapsulated the functionality to navigate a logical token among a set of Visible objects. This can be used to give focus among a set of labels making a menu structure. ListGroup This extends TokenGroup to allow selecting objects in a long list for example a group of checkboxes or a scrollable list of items. Also items can be dynamically added or removed.

4.2.13. Stream

Stream defines a reference to a real multiplex of continuous media in synchroni-sation. Audio, Video and RTGraphics may be included inside a Stream and are pre-sented at the same time to the user. Audio The Audio class provides a reference to an audio stream and associated behav-iours such as changing volume and muting.

Figure 17 - Visivle class model

M5EG - -

31

4.2.14. Interactible

The Interactible class is an abstract mix-in class (not inheriting from root – more like an interface). Its use is to allow direct interaction with objects via a cursor or a keyboard. Only a very limited set of interactible objects are allowed in UK Engine Profile, as it is not concerned with mouse and keyboard interaction. So button is not needed as there is no facility to click it. Only Slider, HyperText and EntryField are permis-sible, with the restriction that Entryfield can only accept numeric data.

4.2.15. Visible

Visible concerns with the detail of rendering ingredients to particular places on the screen. These are items that can be seen. LineArt A LineArt object represents vector based graphics. DynamicLineArt This represents a graphics object that can be dynamically changed for example to draw lines or curves on the fly. Rectangle This represents a rectangle. Bitmap The Bitmap class is used to reference a real image. It can be either in PNG or M2V (a MPEG-2 I Frame). Effects such as offset and transparency can be specified. Video Video references a video stream in a real transport stream. RTGraphics This is Real Time Graphics and is not allowed in the UK Profile. Text The Text class is used to represent strings of characters. It contains various formatting options such as line spacing, justification and options like bold or italic. In the UK profile a default font is always used.

4.3. The MHEG-5 Event model

M5EG - -

32

!AvailiabilityStatus !RunningStatus

Prepare Activate

Deactivate Destroy

AvailiabilityStatus !RunningStatus

AvailiabilityStatus RunningStatus

The MHEG-5 engine is event driven as all actions are direct consequences of some sort of event. There are synchronous events that are the direct result of another event and asynchro-nous events like user input which drive the engine. The application is started with the first event, Launch which triggers other events until a pres-entation is available. The engine is then idle and then waits for asynchronous events to occur which may trigger more synchro-nous events. This is how interac-tion is constructed. Object life cycle This is a state diagram for any ob-ject inheriting from root. In its initial state the object is not us-

able. Preparation makes the

ject exist inside the engine from its exchanged representation, while Activate al-lows the object to interact within the engine.

Figure 18 - MHEG-5 event model

Figure 19 - MHEG-5 class state model

M5EG - -

33

4.4. Format

By the ISO standard the official representation of MHEG-5 programs is either ANS1 (Abstract Syntax Notation One) or a textual form. Obviously the ASN1 notation is machine readable and also highly efficient. For these reasons the UK Engine pro-file only allows ANS1 for transmission.

4.4.1. Textual

The textual form is used for written applications and is human readable. A sample is given below with the full version in the Appendix A.

// 'Hello world' written in MHEG-5 // Originally from http://www.digvid.info {:Application ( '/startup' 0 ) // Application content reference :Items ( {:Link 1 :EventSource 0 // Check this application... :EventType IsRunning // ... for the IsRunning event :LinkEffect ( // Load the scene :TransitionTo (( '~/hello.mhg' 0 ) ) ) } ) :BackgroundColour '=FF=FF=FF=00' // White :TextCHook 10 // Default text content hook :TextColour '=00=00=00=00' // Black :Font "rec://font/uk1" // Font to use for text rendering :FontAttributes "plain.26.32.0" // Default font attributes :BitmapCHook 4 // Default bitmap content hook }

Figure 20 - Hellow world in texual MHEG-5

4.4.2. XMHEG

Steve Barham created an alternative XML based representation. This is possible due to research carried out by T. Imamura and H. Maruyama (Barham 2005) Briefly, due to the fact that ASN1 is a framework for representing tree structured data it becomes possible to represent the same information in XML. This is bene-ficial because XML is human readable and there are excellent tool existing to ma-nipulate this data. The ITU-T with the ISO is developing similar technology The XML Security Suite project from IBM (IBM 2000), created a set of tools for Java called xsss4J that implement the ASN1 to XML translation and vice versa. If a DTD file was supplied the XML would be labelled as well. A tool was also provided that allowed the XML DTD to be auto generated from a grammar for ASN1 which was used by Barham for the ISO MHEG-5 ASN1 grammar. He then edited the DTD file to allow for the UK Profile and also relabelled the

M5EG - -

34

terms for better legibility. This DTD for a language he called XMHEG has been used by this project with changes detailed later.

Equivalent Hello World in XMHEG, full version is in Appendix A. <?xml version="1.0"?> <!DOCTYPE interchangedobject SYSTEM "xmheg.dtd"> <interchangedobject> <application> <root> <external-reference> <group-identifier encoding="UTF8">~/startup.xml</group-identifier> <object-number>0</object-number> </external-reference> </root> <items> <link> <root> <internal-reference>1</internal-reference> </root> <link-condition> <event-source> <internal-reference>0</internal-reference> </event-source> <event-type>4</event-type> </link-condition> <link-effect> <action> <transition-to> <target> <direct-reference> <external-reference> <group-identifier encoding="UTF8">~/hello.xml</group-identifier> <object-number>0</object-number> </external-reference> </direct-reference> </target> <connection-tag-or-null> <null></null> </connection-tag-or-null> </transition-to> </action> </link-effect> </link> </items> <default-attributes> <default-attribute> <text-content-hook>10</text-content-hook> </default-attribute> <default-attribute> <font> <direct-font encoding="base64">cmVjOi8vZm9udC91azE=</direct-font> </font> </default-attribute> <default-attribute> <font-attributes encoding="base64">ABgcAAA=</font-attributes> </default-attribute> <default-attribute> <text-colour> <absolute-colour encoding="base64">////AA==</absolute-colour> </text-colour> </default-attribute> <default-attribute> <bitmap-content-hook>4</bitmap-content-hook> </default-attribute> </default-attributes> </application> </interchangedobject>

Figure 21 - Hellow world in XMHEG

We can obviously see the XML is a lot less terse while the ANS1 is lot more efficien. (See testing for comparisons.) It should be noted that the XMHEG can be com-pressed quite well due to extensive repeatability .

M5EG - -

35

4.5. Delivery

4.5.1. File structure

Originally the MHEG-5 applications consist of a directory of ASN1 binary files rep-resenting the application and associated scenes and any resources like text or im-ages.

To the left is an example of the directory structure for “The Hits!” MHEG Ap-plication. “png” is used for images, and “dat” is for text resources. This directory structure needs to be encoded and delivered over the air.

4.5.2. Over the Air

In the UK the DTG has standardised on using the DSM-CC (Digital Storage Media Command and Control) (ISO/IEC-13818-6 1998) protocol as the method in which MHEG-5 applications are delivered to users. Digital terrestrial television is broadcasted on 6 multiplexes. Each multiplex can have any mix of video, audio and interaction services. There is a limit of 18Mbs or 24Mbs depending on whether 16QAM or 64QAM is used. For example here is the bit rate break down of multiplex

Mux 1 Video (kbs) Audio (kbs) Data (kbs)

BBC1 6000 256 BBC2 3000-6000 256 CBBC/BBC3 2800-6000 256 Radio 96 Radio 96 BBCi 1160

Capacity 16 QAM 18000 kbs

Figure 22 - MHEG-5 app file structure

M5EG - -

36

On multiplex B there are 2 video streams available for BBC interactive ap-plications. Other MHEG-5 streams also have a similar data speed.

DSM-CC was originally designed for use with networked video devices, ie to pro-vide VCR like control of MPEG stream (fast forward, rewind). It has then been ex-tended to allow many other services, most importantly for us the ability to encode arbitrary file structures into transport streams.

The transport streams used in DTT are unidirectional and like teletext interactive applications receivers are unable to request any particular file. As such a data car-ousel similar to teletext is used. Files are transmitted bit by bit and are recon-structed at the other end. An extension of DSM-CC turns the data carousel into an Object Carousel which broadcasts modules of up to 64KiB. Files larger than 64KiB must be the sole ob-ject in the module with the size of the module being increased. Modules can also contain directory structures allowing hierarchical file-systems to be reconstructed on the receiving end. This file-system is presented to MHEG-5 apps with the URL “DSM://,” and applica-tions sit in a directory named after the current channel name. Modules are broad-cast in sequence until they have all been transmitted, at which the whole proce-dure is repeated again. Caching mechanising can be used to store files which may be needed, reducing the wait for a particular file. MHEG provides caching priority hints to help facilitate efficient caching in memory limited environments. Smart design of the Object Carousel can increase efficiency, for example certain popular modules can be broadcasted multiples times per rotations reducing the latency for the highly requested file, at the expense of others. It should be noted that carousels are dynamic and the contexts may change at any time. This means caching mechanisms need to be careful to make sure they always have an up to date copy of data.

4.5.3. Application lifecycle

MHEG-5 application may be broadcast for varying lengths of time, from seconds for advert to continuous broadcasting for main services. When the user tunes into a particular channel, the associated DSM-CC object carousel is checked for any files. If an auto boot MHEG application exists it is lunched. Most channels imple-ment an auto boot application that just sits and waits for the user to press the text or the red button and may display logos indications that interactive services are available, ie “Press Red”.

M5EG - -

37

4.6. The User Experience

We will now explore the behaviour as seen the user of the MHEG application. These diagrams are from UKEngineProfile documentation [13].

Figure 23 - MHEG-5 user interface model

MHEG-5 allows video to be integrated into the interactive applications. This could be used for example to show the current channel while the user browses informa-tion, or the video could be contextual on the state of the interactive application.

The diagram on the left shows the standardised buttons that should be always available with any MHEG-5 implementation. While the shape and design of the remote may be different, all buttons excluding the receiver group will be available to MHEG developers. The “selected group” is exclu-sively for interactive applications and have no effect in usual TV programming, while the “En-tered” Group is dual use and has

Figure 24 - MHEG-5 standard remote model

M5EG - -

38

different meaning depending on context.

M5EG - -

39

4.7. MHEG-5 Graphical Model

4.7.1. Graphics Plane

The graphics plane is the canvas used for displaying all visibles except videos and

I-Frame bitmaps, which exist on a sepa-rate true colour buffer. The plane has size 720x576. This is in ratio 5:4, while DTT video is transmitted in either 4:3 or 16:9. This occurs because TVs have non square pixels, which are slightly wider that they are tall (1.067 times) making the 720x576 resolution available in a 4:3

aspect ratio.

TV also has a limited colour gamut compared with computers which use a full 8bits per sub colour. On TVs the colour range is reduces by 10%, with the ranges 16 – 240 available for each sub-pixel

Figure 26 - TV cropping areas

TV sets always over scan the video, so around 5% on each side is not visible. MHEG applications need to ensure that they are within the safe area. Also considerations need to be taken for 16:9 content shown on 4:3 non widescreen screens,

Figure 25 (BBC, Designing for interactive television v

1.0 BBCi & Interactive TV programmes 2006)

M5EG - -

40

4.7.2. Colour Palette

The UK profile requires at least 188 colours being available. The diagram on the right shows the opaque colours. Receivers are free to implement more colours.

4.7.3. Colour Space

The engine needs to convert between YUV and RGB colour spaces as appropri-ate.

4.7.4. Font

UKEngineProfile does not allow the downloading of fonts. All text is displayed with the Tiresias font which was spe-cifically designed for legibility on TV displays and is referred by “rec://font/uk1”. (Image [31])

Converting font metric to user display The font is provided in vector form, (TrueType), and need to be mapped to display pixels. Because of non-square pixels and two different aspect ra-tios the following rules are used: Vertical Resolution: The computer convention of

1 pixel = 1/72 inch is preserved. So 1 point = 1 pixel

Horizontal Resolution: Each of the 720 horizontal pixels in the graphics plane is considered to be 56/45 points wide. This approximates the ratio for 14:9, so all

text will be subject to some distortion. The advantage is the broadcasters do not have to author applications for each display type.

Figure 27 - Standard 188

MHEG-5 colours

Figure 28 - MHEG -5 colour space conversion

Figure 29 - Standard MHEG-5 font

M5EG - -

41

5. Specification

5.1. Overview

The main software deliverable of this project will be a framework capable of in-terpreting MHEG-5 applications compliant to UK Engine Profile 1.06 delivered over the air in ASN1. The framework will be called “M5EG”, and will at minimum try to provide similar functionality as that of a set top box. Advanced functionality that may be possible due to the extra resources and richer programming environments available on the computer platform will be also investigated. M5EG will be a modular system which will sit on top of any existing DSM-CC object carousel. Its parser and renderer will be plugin-able and separate from the main engine hence allowing easy integration of M5EG into other systems. While a user application will be developed, it will just be a container for the other modular software systems. The aim of the engine is to be a separate software black box that could easily be embedded in other media applications, for example Windows Media Player. The demo application will try to provide a 2 foot and 10 foot experience. The following is a detailed specification of the basic and advanced functionally for M5EG. Basic is considered the minimum to be implemented. Advanced require-ments will be investigated if time exists and may not be implemented.

5.2. General Requirements

5.2.1. Basic Functionality

G1. The system will be composed of three separate independent modules.

1. The Parser 2. The Engine 3. The Renderer

G2. There will be another module for capture, “The DSM-CC”, but this will not

be developed – instead an existing implementation, ‘DC-DVB Source’ will be used.

G3. A GUI based application will be created to show how the modules can be

combined.

M5EG - -

42

5.2.2. Advanced Functionality

AG1. The modules will be integrated into an existing media application to allow

the benefits of interactive TV in an environment more suited to it.

5.3. Technology Requirements

5.3.1. Basic Functionality

T1. The engine will be written in C# using the .Net Framework. T2. A parser capable of reading ANS1 will be developed. T3. IBM’s XSS4J will be used with Steve Barham’s XMHEG DTD to convert be-

tween the transmitted ASN1 and XMHEG. T4. A GDI+ based renderer will be developed.

5.3.2. Advanced Functionality

AT1. A XMHEG parser will be explored. AT2. A Textual parser will be explored. AT3. The possibility of translating between one MHEG-5 notation and another

will be explored. AT4. A DirectX renderer will be explored. AT5. A Windows Presentation Foundation renderer will be explored.

5.4. Amount of MHEG-5 specification to implement

5.4.1. Basic Functionality

M1. Regarding ISO/HEC 13522-5 4.1 Conformance of MHEG-5 objects:

M5EG to be called an MHEG-5 Engine will need to implement the following classes at minimum with all associated attributes, events and internal be-haviours

1. Applications Class

M5EG - -

43

2. Scene Class 3. Link Class 4. Action Class

M2. The corrigenda ISO/IEC 13522-5:1997/Cor.1:1999(E) MHEG-5 (13522-

5:1997/Cor.1:1999(E) 1999) will be followed. M3. The DTG’s UK Engine Profile version 1.06 will be used as the application domain. M4. ASN1 notations will be used for the application interchange format in cap-ture. M5. With respect to error handling the following conventions from the UK En-

gine Profile will be used.

1. The engine shall ignore unrecognised objects. 2. The engine shall ignore unrecognised fields. 3. The engine shall ignore unrecognised elementary actions. 4. The engine may ignore Ingredients that include an unrecognised Con-

tentHook. 5. The engine shall ignore the ObjectInformation exchanged attributes

where encoded. M4. The Extensions to ISO MHEG-5 in the UK Engine profile will be considered M5. UKEngineProfile 1.06 [13] provides a list of classes a compliant engine must

implement. M5EG will implement a subset of this. M6. Undefined classes in UKEngineProfile will not be implemented. These are.

1. Font 2. CursorShape 3. Remote Program 4. InterchangedProgram 5. Palette 6. Button 7. Hotspot 8. PushButton

M7. The following classes part of UKEngineProfile will not be implemented in

basic functionality.

1. The Interactive class and all subclasses 2. The ListGroup class 3. DynamicLineArt class 4. Stream, Audio and Video.

M5EG - -

44

M8. The following restriction will apply in basic functionality.

1. The M2V image format will not be supported 2. Some resident programs associated with streams will not be supported

5.4.2. Advanced Functionality

AM1. The set of features and classes not implemented but part of the UKEngi-

neProfile will be implemented including all items listed above. AM2. Extensions to MHEG-5 to allow a return channel will be explored. AM2. Extensions to MHEG-5 to allow XMHEG authoring will be explored.

5.5. Usability Requirements

5.5.1. Basic Functionality

U1. The M5EG application will allow MHEG-5 interactive application to be dis-

played is a 2 foot UI format, with interaction via a keyboard and mouse U2. Users will be able to open stored MHEG-5 applications and the engine will

interpret it in a faithful manner. U2. The engine should be reliable and deal with errors in the MHEG-5 applica-

tion in a resilient way.

5.5.2. Advanced Functionality

AU1. M5EG will provide a 10 foot experience via a full screen application with

interaction via a remote control. AU2. The engine will be able to support live execution of MHEG-5 applications AU3. Users will able to watch video streams and have auto-boot applications

start without any interaction. AU4. MHEG-5 applications will be able to change video streams.

5.6. Accessibility Requirements

M5EG - -

45

5.6.1. Basic Functionality

C1. The M5EG application will have a built in help system and keyboard short-

cuts similar to most Windows applications.

5.6.2. Advanced Functionality

AC1. Zooming of the MHEG-5 will occurs to help partially sighted users. AC2. A screen reader will be fed displayed text to help partially sighted users.

5.7. Added Value Requirements

V1. Quality of rendering especially text will be explored. V2. Extraction of structured content, for example EPG data will be explored. Throughout this project an eye will be kept on the home electronics / computer integration front and M5EGs position within it. Specifically it will be seen if any ad-vantages to either fields can be realised by this convergence.

M5EG - -

46

6. Design

6.1. Overview

As per the specification M5EG will be split into 3 modules, with each module spe-cialising in a particular task. The following diagram shows how M5EG will commu-nicate with external components.

Data is transmitted on a multiplex and this is received by the TV card hardware (here a Hauppauge Nova-T). The BDA driver presents a generic interface to the transport stream. DC-DVB Source is a DirectShow filter than can attach to a BDA interface and parse the data carousel encoded in the stream. It then dumps the MHEG-5 application which M5EG can use to decode. While data flows up the con-ceptual model, control goes down with M5EG controlling the DSM-CC, which in turn controls later parts. For example M5EG can tell DC-DVB to tune to a certain frequency and retrieve that data carousel associated. The data flow between DC-DVB Source is over the OS file system – DC-DVB dumps the MHEG-5 application to a temporally folder, which M5EG can use. The data flow of the transport stream until DC-DVB is over DirectShow pins which are in

Figure 30 - Framework overview

M5EG - -

47

memory. This is advantageous because the stream is 24 or 18Mbs and could seri-ously affect performance if written file system. Only the amount needed for the data carousel is dumped, minimising disk resource needs. The reason to separate the DSM-CC is be source independent. MHEG-5 applica-tions can come from disk, live capture via the DSM-CC or even over IP. Also a DSM-CC is strictly not part of the MHEG-5 Engine; it is a completely separate entity and has other uses outside MHEG-5. Another advantage is that an existing stable DSM-CC can be used. This is signifi-cant because a DSM-CC is not a trivial piece of code. The three parts of M5EG are the Parser, the Engine and the Renderer. Their rela-tions to each other are shown in the following diagram.

The various forms and translations of the MHEG-5 applications are illustrated by the arrows. The DSM–CC delivers the application in an ASN1 encoded representa-tion compliant with the UKEngineProfile. This is read from the file system by the Parser which parses it into an internal object model. Each of the encoded classes is instantiated and its parameters are set up.

Figure 31 - Internals overview

M5EG - -

48

The internal object model is what the MHEG Engine interprets. Behaviours speci-fied by the MHEG standards are executed and event processing begins – The in-teractive application is now alive. The sole visible output of the interactive application is a construct called the Dis-play Stack, which is an ordered list of Visible Objects that describes the position, size and content of every seen object. This is fed into the Renderer which turns this generic visual representation, using the rules of the MHEG-5 graphics model, into a raster image which is the final output. This can be then mixed with video and outputted to the display for the user to see.

6.1.1. The Parser

The parser is a generic module that implements the IParser interface. It encapsu-lates the process of taking an MHEG-5 application in one form and outputting the same application in the Engine own internal form. An IParser states which one of the three forms of MHEG-5 encoding it can accept, ASN1, textual or XMHEG. This allows the Engine to interpret any type of MHEG-5 application without any modifications, if the correct IParser exists. Each IParser also has the AttPlugin attribute allowing complete separation of the code from the Engine. This allows any third party to write a parser without having access to the source code of the Engine or Renderer. The IParser is complied into a DLL which is put in the ‘Parsers’ folder, which M5EG searches. Only an ASN1 parser is required as this is what is broadcast. The most obvious way to achieve this would be to parse the ASN1 binary manually. This is difficult and error prone even for small files, especially so since the MHEG-5 grammar is vast. A more robust method is using an ASN1 stub compiler which generates a parser from the ASN1 grammar. This is a reasonable way of implementing a parser as it is direct and the resulting parser is very efficient. The problem is that configuring a stub compiler is relatively difficult and that there are no open source stub compil-ers for our target language C#. These problems can be worked around as we could use asn1c (ASN1C n.d.)i a sub compiler that can output C++, that could be put into a managed C++ assembly. This would be accessed from our C# engine. While not a straightforward task it would work and would have been the preferred choice if it weren’t for a far more elegant method. Here is a conceptual diagram on how M5EG parses ASN1.

M5EG - -

49

We can see that the ASN1 application is first translated into XMHEG form by the component labelled ‘ASN1-2-XML’. This is a one to one mapping using the XMHEG DTD by IBM’s XSS4J conversion library. Now that the application is in XML, it is read in by .Nets XML reader into DOM. The Document Object Model is an object representation of the XML XMHEG data and is what is now parsed. Every node in the DOM representation is recursively read until an internal object model is built. We are left with an equivalent application to the original ASN1 but acceptable by the Engine. The advantages of this approach are firstly, we get two parsers for the price of one. While the above depicts an ASN1 parser, if the ASN-2-XML block is removed a XMHEG parser remains. While no XMHEG applications are transmitted, they can be easily written. Secondly at no stage does any of our code try to directly touch the ASN1 data. Those stages use proven standard tools – only the DOM is read by our code. This is important because we do not have to write any code that does the error prone process of reading bits in. It means our parser can be easily written and verified.

6.1.2. The Engine

The Engine is the heart of the M5EG system. It is what runs the MHEG-5 applica-tion and is split into two parts. There is the actual Interpreter which encapsulates all processing operations, and the object model which what is described in the ISO MHEG-5 standard.

ASN1 Parser

ASN1-2-XML

ASN1 MHEG-5 Application

XMHEG Parser

XMHEG Application

Internal MHEG-5 Object Model

From DSM-CC

XMHEG XML DOM

XML Reader

To Engine

Figure 32 - Parser overview

M5EG - -

50

The Engine applies the event driven model described in the MHEG-5 specifica-tions. It queues all asynchronous events, such as user input, and executes them one by one. If any synchronous events are created it handles them appropriately. It manages launching and quitting of applications and handles all tuning requests. The model is a C# version of the specified classes in the ISO MHEG-5 specifica-tions. It resembles the specifications as closely as possible and has all internal and exchanged attributes (fields in C#) and all internal behaviours and actions (meth-ods in C#). This model allows us to apply the MHEG-5 semantics as tightly as pos-sible. One of the internal constructs of the Interpreter is a Display Stack which is a model of what the user sees and is outputted to a Renderer. The Interpreter also manages timers and any other details specific to the running of an application like default values, and error handling.

6.1.3. The Renderer

The Renderer is also a generic plug-in that attaches to the Engine. Renderers im-plement IRenderable and have AttPlugin. This allows any external Renderer to be added to M5EG just like the parser. It also means the Renderers can target differ-ent types of displays and use different rendering techniques. The IRenderable in-terface ensures that all Renderers can communicate with the Engine in a stan-dardised manner.

Engine

To Tuner

To Display

User Interaction

Interpreter Model

From Parser

Figure 33 - Engine overview

M5EG - -

51

The Display Stack models exactly how the Visibles in the MHEG applications are displayed to the user. It is an ordered list of items with items lower down being drawn behind those above. The construct is specified in detail in the MHEG-5 specifications and Actions exist to directly manipulate it. A Video object can be contained, and this represents where the video should be decoded, other items like Rectangles, Bitmaps and Text can also be contained. The Display Stack is inputted to the Renderer from where it is further processed. The above diagram represents the GDI+ Renderer which must be implemented. GDI+ provides methods to draw strings, images and shapes to special canvases. Instead of drawing each item in the display stack manually, i.e. bit for bit, we try to match items to GDI+8 functions which are not only optimally implemented but also in many cases hardware accelerated. A naïve way to render would be to iterate though the display stack drawing each item one by one to the canvas. While the result would be accurate, it is not opti-

8 GDI+ generally wraps the older GDI functions making them easier to use.

Figure 34 - Renderer Overview

M5EG - -

52

mal because some items may be totally or partially obscuring others. Since draw-ing is an expensive process we want to minimize it at all costs. We do this by an Optimiser Stage which can work out via various algorithms what parts to render and what to discard. There are various complexities to the algo-rithm from simply disregarding completely obscured items, to keeping a memory of what has been drawn before and only changing what has been updated. Also adjacent shapes could be combined. The Layout stage concerns interpreting the Visible objects and working out a strategy to draw each shape. It has to work out all the colours, apply translucen-cies and then position and clip each item. For text this may be complicated as ad-vanced features like word wrapping and tabbing need to be manually worked out. Once this is done the items are passed to the Rasterer which draws them to a back buffer9. This is to implement double buffering a technique designed to minimise flicker. It also makes drawing faster as the main display is only accessed once per complete Renderer call.

Drawing to the back buffer means taking each item and working out exactly what colour to set the pixels on the 720 x 568 canvas. Once everything has been drawn to the buffer it is sent out to the display where either it is displayed immediately or alternatively mixed with a video stream.

6.1.4. The User program

The User program exists to show all the above components working in harmony. It will house the M5EG modules and hence allow the executing of MHEG-5 appli-cations. It has buttons which map to the standard remote controls ones specified in the UKEngineProfile. These could be connected to a real remote to allow a 10 UI. It should have a full screen mode to complete the experience. The program should at least run saved MHEG-5 application and possibly imple-ment direct capture by directly embedding DC-DVB as a DirectShow player. It needs to be easy to use and be able to showcase all the features of the core modules. Also it needs to present the advanced features is a usable form.

9 A portion of memory representing the display, pixel for pixel.

M5EG - -

53

A design mock-up is shown below.

Figure 35 - GUI Mockup

Full screen mode

Output of Renderer

Navigation

Status of engine

M5EG - -

54

6.2. Technology Decision

6.2.1. Concerns

There exists a range of technological solutions that can solve our design chal-lenges. Suitable logic needs to be used to fairly discriminate between similar choices. We need to use technology take makes the project easier. To help with this aim the minimum requirements of any platform used are given below. The implementation language must:

1. Have excellent support for XML . We need to parse MHEG-5 in XMHEG form.

2. Support a rapid applications development paradigm . Time is limited and a lot of functionality needs to be implemented.

3. Have solid development tools. Debugging will be a major part of the project.

4. Have excellent support for multimedia hardware. We need to easily deal with TV cards and video streams.

5. Have excellent support for 2D graphics. We need to implement a fairly complex renderer.

6. Have excellent interoperability. We may need stray components in completely different programming platforms. We need to use them easily.

7. Be easy for other developers to use. The modules need to be generic hence others need to be able to write and use them.

8. The platform must support architecture aims for the engine to be plugged into media centre applications.

9. Be easy for users to use.

10. The platform must be widely available, appealing to the largest user popu-lation.

The following software platforms are considered

M5EG - -

55

6.2.2. Sun’s Java

The Java platform provides a suite of development tools that target the Java Vir-tual Machine which should allow the creation of applications that are OS neutral. Advantages

1. Creation of cross platform applications.

2. Proven XML support.

3. Language is very powerful and clean. Disadvantages

1. Hard to access hardware, no way of communicating with TV Card in a ge-neric fashion.

2. Multimedia framework (JMF ) is poor, lack of industry support.

3. Graphics framework is mediocre, lack of hardware acceleration.

4. Not as fast as native implementations.

6.2.3. C#

C# is an object oriented language that is based on C++ but including many aspects from Delphi, Visual Basic and Java. It targets the Common language runtime (CLR), a Virtual Machine and can use components from any other .Net language. Currently in version 2.0, it has many advanced features that make it unique. Advantages

1. Excellent integration with Windows platform. a. Hence can use DirectShow as a multimedia framework.

2. Excellent support for hardware accelerated graphics via GDI+ or DirectX

3. Elegant Xml support.

4. Language is very powerful and has unique RAD10 features. Disadvantages

1. Not cross platform

10

Rapid Application Development

M5EG - -

56

2. Not as fast as native implementations

6.2.4. MFC C++

Microsoft Foundation Classes for C++ provide a direct API for Windows. It pre-dates .NET and much of Windows itself is written this way. Advantages

1. Excellent platform support, DirectShow is a C++ API as is GDI+

2. Very fast, as compiled code is native. Can use optimising compilers. Disadvantages,

1. Complex to use and error prone. 2. Poor web support (XML). 3. Not a Rapid Application Development environment.

Bearing in mind the above arguments, the main software deliverable will be writ-ten in C# using the .Net Framework.

6.2.5. Reasons

The reason for using C# is that it is a clean highly object oriented language that supports rapid application development. This can be said of Java, and as such C# as a language is very similar to Java but the real benefits of C# occur because of the tight integration with the Windows platform. This sits under .Net and supplies a plethora of extensive well documented well tested APIs for any conceivable purpose. We are concerned with multimedia and graphics programming and an extensive framework in the form of DirectShow, BDA and GDI+ are provided. Also as no other existing implementation is written in C# we will be creating a first for the world, and this becomes a unique feature of the project – A MHEG-5 inter-preter highly integrated into the Windows environment.

6.2.6. DirectShow

DirectShow is an API for windows that provides an extensive multimedia frame-work.

M5EG - -

57

It is based on COM and provides a common interface for media related functions. It is extensible via filters which implement sub functions and are connected via pins.

Figure 36 - Graphedit screenshot

DC-DVB Source, used for the DSM-CC Data Carousel, is implemented as a Direct-Show filter. It provides the functionally to tune a BDA compliant TV card and dis-play video. In many ways it is a whole TV card application in filter and any Direct-Show capable program can have TV functionality by using it. This is what M5EG could use to provide TV functionality in a simple manner. DirectShow is available in .NET by the DirectShow.Net project.

6.2.7. BDA - Broadcast Driver Architecture

BDA is a driver interface for TV cards. Any TV card with BDA compliant drivers can talk to any software written for BDA systems. This means that applications that deal with TV cards do not need to be hard coded, they just simply write against the BDA interface.

6.2.8. GDI+ - Graphical Device Interface Plus

GDI+ is the successor the default 2d graphics API on Windows and has support of advanced features like as alpha blending and gradient shading. The .Net frame-work proved a managed interface to these libraries allowing easy access to pow-erful graphics presentation. .Net 2.0 also has native support for optimised double buffering and is hardware accelerated on most Windows systems. Alternatives

6.2.9. Direcxt3D

Direct3D is a low level API for hardware accelerated 3D rendering. It can also be used for 2D applications. It was designed for game developers and hence is very fast. It is also highly complex and probably overkill for our needs. So it will not be used for our primary Renderer.

6.2.10. XSS4J - XML Security Suite for Java

This is a package of helper tools written by IBM. Of importance is a set of classes that implement ASN1 to XML translation. A sample command line application (Translator) is provided that given an ASN1 file and a DTD will output labelled XML equivalent to the input.

M5EG - -

58

While the library is in Java and would integrate nicely within a complete Java parser, using it from .Net is not as un-elegant as it seems. This is because Transla-tor is just like any command line application and can be executed in the back-ground. The ASN1 to XML is black boxed.

M5EG - -

59

7. Implementation

7.1. Overview

This section will try to describe in depth the implementation details of the final M5EG framework, and shall highlight some interesting cases and issues. Also un-expected discoveries and any optimisations will be explained.

7.2. The Parser

The parser is split into two parts. The first is a layer of logic which communicates which the rest of the system, mainly the Engine, and manages the detail of pars-ing. The second part is the IParser derived components that plugin to the first layer. These are responsible for the actual decoding of the data into the internal MHEG-5 Class model. The first layer is implemented as MHEGParser in our implantation and has the fol-lowing overloaded public static method definition. public static Group ParseMHEG ( String groupPath)

This, given the path of an encoded MHEG-5 file, will decode it to the internal class model. Group is the super class for Application and Scene, which are directly mapped to real file on the file system, no other specific MHEG-5 classes are con-tained in their own file - they are always inside a Group object.

Parser

To Engine Generic Parser Layer

ASN1 Parser

XMHEG Parser

Textual Parser

IParser

MHEG-5 file

Figure 37 - Parser detail

M5EG - -

60

The MHEG-5 file pointed to be groupPath may be in ASN1, XMHEG or textual form. MHEGParser has logic to work out which type it is and call the appropriate IParser. It can do this because of the unique traits of each file, the logic is explained below. Read head of file

1. if XMHEG identifier tags ( shown below) found then file is XMHEG 2. else search full text for following Textual identifier tags

a. if “{:Application” found then Application b. if “{:Scene” found then Scene

3. else open file in binary mode a. if first two bytes are equal to A0h 82h11 then Application b. if first two bytes are equal to A1h 82h12 then Group

4. else not MHEG-5 File

<?xml version="1.0"?> <!DOCTYPE interchangedobject SYSTEM "xmheg.dtd"[]> XMHEG identifier tags at the start of each XMHEG file

Xml Identifier Tags

After the above stage has completed the form of the file is known and an appro-priate IParser needs to found and invoked. The IParser interface specifies a few methods that allow plug in behaviour with MHEGParser. Firstly it has methods which return what form of MHEG-5 the IParser can accept. Secondly, it has the following method to do all the work, Group Parse ( Stream MHEGGroup );

The MHEGGroup parameter is a file stream to a file object, and was already opened by the notation checking code described above. Each IParser takes the stream, decoded it into whatever form it works with and parses it creating an ob-ject instantiation of the internal class model. This in turn gets returned to what-ever called MHEGParser. The plugin ability is made available by AttPlugin, which each IParser must also im-plement. It is also used by the Renderers as described later. AttPlugin is a C# at-tribute that encapsulated several functions that allow components to be built separately and be used at run time. Then there are a few properties which return identifier information about the plugin, including name, version etc. Then, there are abilities to describe what version of M5EG the plugin is compatible with.

11

ASN1 identifier for Application. 12

ASN1 identifier for Group.

M5EG - -

61

Each plugin in decorated with this attribute and is complied into a DLL. The plugin management code inside MHEGParser searches a set directory for all DLLs. Then using .Net reflection it searches each assembly for the AttPlugin attribute, if found it then checks if the assembly implements IParser. If both checks are ok, the plugin is loaded and stored for use. A similar system exists for Renderers as well.

7.2.1. The XMHEG Parser

The XMHEG Parser uses .NET XMLDocument class to load the file into a DOM13 ob-ject structure. The XMHEG.DTD file is used to verify the XMHEG is correct. This parser is structured into various builders, each of which is responsible for creating one specific class, for example a Link. Because the MHEG-5 model exhib-its a high use of inheritance, high code reuse is made in the parser. The builders are responsible for taking the exchanged attributes and instantiating an exact class model replica of the exchanged data. For example, the following XMHEG for a link is parsed, <link> <root> <internal-reference>1</internal-reference> </root> <link-condition> <event-source> <internal-reference>0</internal-reference> </event-source> <event-type>4</event-type> </link-condition> <link-effect> <action> <transition-to> <target> <direct-reference> <external-reference> <group-identifier encoding="UTF8">~/hello.xml</group-identifier> <object-number>0</object-number> </external-reference> </direct-reference> </target> <connection-tag-or-null> <null></null> </connection-tag-or-null> </transition-to> </action> </link-effect> </link>

Figure 38 - Example XMHEG for a Link Object

Firstly, Link builder instantiates a new link object. Then because Link extends In-gredient which extends Root, the Link builder calls the Ingredient builder. The Ingredient builder then calls the Root builder which stores the item identifi-ers, ‘1’ here, in the newly created link object (Note a capital Link refers to the MHEG object, while a small link refers to the C# internal class model instantiation). As Link does not contains any exchanged attributes of class Ingredient, the In-gredient builder does nothing and returns to the Link builder.

13

Document Object Model

M5EG - -

62

The Link builder then has to build the Link condition. It extracts the information from the XML node and sets up the condition on the link object. Event Source is a MHEG-5 Object Reference, so Link builder delegates the work to the Objec-tReference builder. Event type is coded as a number which represents one of the MHEG-5 events. This is coded by the builder into a C# enumeration and stored in the created link object. The next XML node is an Action. Remember a MHEG-5 Action object is a collection of Elementary Actions. The Link object aggregates Actions, like many other MHEG-5 objects. A separate Action builder is called, which builds each Elementary Action, only 1 in our example. This builder will call other builder as necessary, for example here it will again call the ObjectReference builder as TransitionTo targets a particular MHEG-5 object. Once the Action object is built it is stored in the link object Now the link object has been created, it is returned to whatever called it. Since only Group objects can contain Ingredients, this will be the Group Builder. The GroupBuilder is at the top of the hierarchy and is called first one the file has been decoded into DOM. It iteratively builds each ingredient inside of it and eventually returns a complete C# coded Application or Scene to the MHEGParser.

7.2.2. The ASN1 Parser

As described in the design section our ASN1 parser uses the MHEGParser. It first calls a block of code called ASN-2-XML that uses the XMHEG DTD to turn the ASN1 encoded MHEG-5 to identical XMHEG by a one to one mapping process. Once the data is in XMHEG, the ASN1 Parser calls the XMHEG Parser to decode the MHEG-5 program.

Figure 39 - ASN1 parser detail

M5EG - -

63

The ASN-2-XML block basically is a wrapper for the XSS4J library. It was the only library found that was free and could decode ASN1 to XML. While the library is free it is not open source and apparently development has stopped as of 2002. The XSS4J project has various libraries for security related purposes, but most in-teresting for us is the ASN to XML translation library. It contains various low level methods that can take translate between both forms. An example application called Translator is provided with source to demonstrate use. A problem for this project is that the libraries are developed in Java and hence cannot be directly access from C#. However there are many interop solutions which allow Java and C# to co-exist, and this was taken into consideration when C# was chosen as the implementation language. The Translator program could have been rewritten in C#, and then used some in-terop mechanism to call the XSS4J libraries. While relatively straight forward this was not the approach taken. This is mainly because of the small issues between Java and C#, like the way XML is handled - we would have to convert between the different DOM systems. Also we would have to waste time re coding the Transla-tor program. The method undertaken is quite simple in contrast. A Windows batch file is writ-ten to encapsulate calling the command line utility Translator with the correct pa-rameters and options. The batch file itself is given first the ASN1 file to decode, and secondly a filename to store the XMHEG version to. The ASN1-2-XML block in our ASN1 Parser, firstly asks the operating system for a temporary file. Then using shell execute, calls the batch file with the correct ar-guments, firstly the path of the ASN1 file to decode, and secondly the path of the newly created temporary file. The batch file does its work, and then the ASN1 Parser can continue calling the XMHEG Parser with the temporary file. Once that has finished the temporary file is deleted. While this scheme may not seem the most efficient it is sufficient for our needs due to the small files and infrequently use of the Parser, it only need to be called at most every time the Scene changes. Also if a C# version of XSS4J is ever created then it can be integrated fairly easily and remove this dependency on Java.

7.2.3. Optimisations

Caching From observing the execution of MHEG-5 applications it was found that the same Scenes are called again and again. Initially to launch and MHEG application, the start-up Application object is loaded. This will launch one Scene. Every time navi-gation occurs to a new segment, a new Scene is transitioned to. For example on

M5EG - -

64

the BBC, the initial Scene is called the bridge and is the gateway to other parts of the service. If a user selects weather the application transitions to a Scene for the weather. If the user wants to go back, the application transitions to the bridge again. Every time a transition occurs the Parser is called and has to load the Scene from the filesystem and decode it from scratch. However the same Scenes are called again and again and hence make an ideal candidate for caching. The complete parse process is quite computationally expensive as it contains IO. We could cache the objects just after decode in MHEGParser and if later requested we would save the decode time. This was experimented with a simple Least Recently Used buffer stored in MHEG-Paser. Experiments were made described later in the Testing section which show the benefits gained. Of note the caching mechanism needs to make sure that data is current, and this was implemented by checking timestamps of the files. A more complex system of hashes could be used in the future as sometimes the same file is rebroadcast.

7.2.4. Issues

Object Identifiers MHEG-5 specifically identifies each object by a group identifier and an object iden-tifier. The group identifier is the Group object that contains the Ingredient, and the Object identifier is a unique integer within that group. Sometime the group identifier is left out because it can be inferred, ie when an object referrers to an object in the same group. The MHEG-5 specification gives rules for this. A problem that stemmed from inferring in the Engine is that Ingredient objects could be used and its group identifier would be needed. Since it is not encoded in the object, resolving it would require some effort - We would need to find the Group that owns that particular Ingredient etc. To simplify the Engine, the Group identifier is always encoded in the instantiated class. The identifier is a string that is the name of the file that contains the Group. XMHEG.DTD The XMHEG DTD used from Steve Barham’s project Mu [20] did not contains the latest addition from UK Engine Profile 1.06. This consisted of additional actions, events, and changes to the some structures. These changed were incorporated and a new version of XMHEG.DTD was created.

M5EG - -

65

7.3. Engine

As discussed in the Design section the Engine is split into two sections, firstly the internal MHEG-5 Class model, and secondly the actual Interpreter itself.

7.3.1. The Class Model

The Class model implements the functionality of the MHEG-5 classes in our target language C#. It was designed to exactly replicate the specification ensuring se-mantically correct execution of MHEG-5 applications.

Engine

From GUI

Interpreter MHEG-5

Class Model

From Parser

MHEGTimer

MHEGDisplayStack

MHEGTuner

Figure 40 - Engine detail

M5EG - -

66

Figure 41 - MHEG class model (13522-5 1997)

Each of the MHEG-5 classes defined in ISO/IEC 13522-5 is realised as a C# class. Each of the defined exchanged and internal attributes are realised as public fields and each behaviour becomes a method. Below there is a code sample of the Ingredient class to demonstrate how the im-plementation works. Method bodies have been removed to avoid clutter.

M5EG - -

67

public class Ingredient : Root { #region Exchanged Attributes private int? contentHook = null; public int? ContentHook { get { return contentHook; } set { contentHook = value; } } private Content originalContent = null; public Content OriginalContent { get { return originalContent; } set { originalContent = value; } } #endregion #region Internal Attributes private Content content = null; public Content Content { get { return content; } set { content = value; } } public override void InittInternalAttributes () #endregion #region Internal Behaviours Private override void Destroy () #endregion #region Behaviours public virtual void SetData ( GenericOctetString newContent ) public virtual void SetData ( GenericContentRef newContent ) public void Preload () public void Unload () #endregion }

Figure 42 - Example C# code for Ingredient

The elementary actions defined in the specification are realised as lightweight ob-jects that target the behaviour in the appropriate class. public class Unload : ElementaryAction

{ … internal override void Execute () { ( ( Ingredient ) Target ).Unload(); }

Figure 43 - Example C# code for Unload

This is optional exchanged at-tribute ie it exists optionally in the coded MHEG. Note the use of C# 2.0 nullable types, if con-tentHook is not encoded, then it is set to null

This is an internal attribute and so is not encoded - it is set up when this class is prepared.

This is called from prepare and sets up all internal at-tributes.

An internal Behaviour which is private as can be only called from other be-haviours in this class.

These Behaviours are pub-lic methods and are called from Action objects.

Target is bound at run time by the ElementaryAction superclass

M5EG - -

68

}

The example above will, when executed, call the Unload behaviour on the correct Ingredient by using the Target. Every elementary action has an object reference encoded to which it targets. This is bound to the real Ingredient object by the ‘Find’ method in the Interpreter.

7.3.2. Interpreter

The interpreter houses all the required functionally to actually run the program, it is the heart of the engine. It is split over several classes in the Engine namespace. They are

1. MHEGEngine 2. MHEGDisplayStack 3. MHEGTimer 4. MHEGTuner 5. MHEGConstants

MHEGEngine is probably the most important class in the all of M5EG as it contains the main processing loop and all logic dealing with MHEG-5 objects. MHEGDisplayStack is responsible with implemented the MHEG display model. It holds all visible that are displayed and optimisation techniques to minimise draw-ing. MHEGTimer encapsulated the functionality of timers defined in the MHEG-5 speci-fications. Each timer run on a separate thread and creates an asynchronous event once fired. MHEGTuner hold all functionality required to interface with the TV card hardware. MHEGConstants have declarations for all constant values, such as default back-ground colour.

7.3.3. Object Handling

Each Ingredient in each Group is uniquely identified by a group id and an object id. An ObjectReference object servers as a pointer to any of these Ingredients and MHEGEngine is responsible for dereferencing them. A Group is identified by hav-ing an object id of 0. Below are some examples of a valid object references.

M5EG - -

69

An object is found by the MHEGEngine method, public Root Find ( ObjectRef reference ) Figure 45 - Find C# definition

This tries to resolve the reference. At anytime only one Scene and one Application are active; hence object reference to ingredients can only be inside either one of these Groups. If the object identifier is 0, then we are either referring to one of these groups or we need to load an external Group, for example when transition-ing between Scenes. MHEGEngine has references to the currently active Application and Scene. It uses the following logic,

1. If the object identifier is not equal to 0, then try to search these groups. a. Match the supplied GroupID to the Active Groups b. If no match - then Error, else c. Do a linear search though ‘Items’ collection of Ingredients inside

the Group d. If not found then error.

2. If object identifier is 0, then match GroupID to current Active Groups a. If match, return that Group b. Else need to load external file containing that Group

The code that does the grunt of searching the Groups is below. Root ret;

GroupId : ~a ObjectId: 0

GroupId : ~main.mhg ObjectId: 7

GroupId : ~a ObjectId: 1

Object References Refers to Application ‘a’

Refers to object 1 in Application ‘a’

Refers to ob-ject 7 in Scene main

Figure 44 - Example object references

M5EG - -

70

… ret = group.Items.Find( delegate( Ingredient item ) { item.ObjectIdentifier.ObjectNumber == reference.ObjectNumber; } );

… Figure 46 - Find C# implementation

Note the use of an anonymous method. The ‘group’ object is the Group that has been matched, and ‘Items’ is a List<Ingredient>. Searching uses the inbuilt method ‘Find’ that given a method returns the first object in the collection that the method returns true for. The parameter of Find can be either a delegate14 or like here an anonymous method which is declared inline and has no name. This search is linear wrt the number of items contained in the group, and hence not optimal. To reduce latency for further access a cache is use. After being re-trieved for the first time, each Ingredient is placed in a typed HashTable that hashes on its ObjectIdentifier. Future accesses can be retrieved in constant time. If we need to retrieve an external reference then we need to create a canonical path name for the group, eg ‘~a’ turns into ‘d:\TV\MHEG\BBC\a’. There are various rules given in the MHEG specifications that tell us how to do this. Once this is done and if the file actually exists, MHEGParser is given the path to decode.

7.3.4. Content Reference

Similarly content reference, i.e. references to images, text, video or audio are en-capsulated by a ContentReference object. Either these are files on the filesystem, like an image, or they refer to items in the multiplex like a video stream. The MHEG-5 specification gives various naming conventions that give a URL like interface to content references. For example Starts with

Example Notes

DSM:/ or ~/

~/a Refers to a file object relative to root of active ap-plication

DSM:// or //

//images/bitmap.png

Refers to a file relative to root of the filesystem

rec: or dvb:

Rec://abcd.123..456 Refers to streams in multiplex using dvb://<original_network_id>.[<transport_stream_id>].<service_id>

Rec://font/uk1 Refers to default font

14

Typed method pointer

M5EG - -

71

7.3.5. Persistent Storage

MHEG-5 allows application to store arbitrary data for later use. It is stored perma-nently so if the application is started later the data can be used again. The only requirement is that receivers have at minimum 1 KiB of storage, using a Least Re-cently Used Scheme. Hence applications have to deal with data being lost later. MHEGEngine uses an in memory store that applications can use with an arbitrary 1MiB limit. This is serialised to disk when the Engine is turned off and de-serialised back when the Engine is restarted. Importantly the same store is used for all ap-plications as different application often use data stored by others

7.3.6. Event Processing

Event processing is at the heart of the Interpreter. The complex semantics de-scribes in the MHEG-5 specification need to be followed exactly for applications to run correctly. Various structures are used to handle event processing. Firstly there are several queues to hold all the Events and Actions as they are created. The queues are de-clared as follows. private Queue<ElementaryAction> mainActionQ; private Queue<ElementaryAction> tempActionQ; private Queue<Event> asyncEventQ;

Figure 47 - C# action Qs definition

The asyncEventQ queues all synchronous events as they are created. As events are processed the resulting actions are put on the mainActionQ. The tempAc-tionQ is used for buffer actions that occur during the executing of other elemen-tary actions. These resulting actions need to be executed first. We need also to handle Links. As described earlier Links connect Events to Ac-tions. They can be either is an active or inactive state. When a Link is activated it is added to the following structure. private Dictionary<ObjectRef, Dictionary<enumEvents, List<Link>>> activeLinks;

Figure 48 - C# activelinks definition

It is a typed hash table of hash tables of Lists of Links. The first hash table hashes on an Object Identifier (ie for each MHEG-5 object). The second hash table is hashed on enumEvent (for each event). Both of these correspond to the Link-Condition stored in the link. The AddLink method adds the supplied link to the correct positions, while RemoveLink does the opposite. The structure is needed to allow very efficient lookups of links, because of its use in the main loop. There are two type events Asynchronous and Synchronous. The latter is handled by in the following way.

M5EG - -

72

1. Check the activeLinks structure for matching links. 2. For all matching links.

a. If it is a data event , check data condition in Link. b. If true, or no data, add resulting elementary actions to tempAc-

tionQ. Asynchronous events are added to the asyncEventQ and are handled by the main event processing loop after all pending actions have executed. Events are processed by ProcessEvents() which does the following,

1. For each event in asynchronousEventQ 2. Handle as if synchronous event. 3. Copy tempActionQ to mainActionQ 4. Clear tempActionQ 5. For each elementary action in mainActionQ

a. Execute. b. Handle all resulting Synchronous events (this will lead to adding

elementary actions to tempActionQ) c. Handle all resulting Asynchronous events by queuing (these will be

handled after all existing events have been). d. Pre-append tempActionQ to mainActionQ (ie execute resulting ac-

tions before any others). e. Clear tempActionQ.

6. Sleep thread as all asynchronous events are complete. Since all synchronous events are the result of asynchronous events we will have executed any pending actions. When asynchronous event occurs they also wake up the Engine thread.

7.3.7. Threading

A multithreaded system is imported not for performance but for responsiveness. The Engine needs to run on its own thread as so when it’s processing events it does not ties the GUI up and vice versa. The Engine should be running uncon-strained and the GUI should be displaying it. The GUI should run asynchronously to the Engine. We do this by splitting the Engine off from the GUI thread when an MHEG applica-tion has been selected. This is implements as the following Thread engineThread; … ThreadStart starter = delegate { MHEGEngine.Instance.Run( appPath); }; engineThread = new Thread( starter );

M5EG - -

73

engineThread.Name = "MHEGEngine"; Figure 49 - C# threading definition

engineThread.IsBackground = true; engineThread.Start(); The MHEGEngine is a singleton as only one is needed and hence it can be refer-enced in a global manner. The static Instance property on the MHEGEngine class returns the current instance of MHEGEngine - if one does not exist it creates one. The new thread starts executing ‘Run’ on MHEGEngine with the path to the MHEG-5 application. MHEGEngine then decodes it via the parser, and starts exe-cuting it using the event processing loop above. Its main loop is lock ( lockObj ) { while ( !quit ) { ProcessEvents(); UpdateScreen(); Monitor.Wait( lockObj ); } }

Figure 50 - C# main loop

We can see that the Engine thread Processes any asynchronous events. If any up-dates are made, the GUI thread is informed to redraw by ‘UpdateScreen’. Then it sleeps, waken up later by any new asynchronous event. Asynchronous events may come from the GUI thread, by user input, the Engine thread itself as a result of another event, or by timers.

Lockobj is a specific object to lock on

Figure 51 - Main loop state model

M5EG - -

74

7.3.8. Timers

Timers can be used by MHEG-5 application developers to trigger actions in the fu-ture. They are created from the SetTimer action on the Group class, and MHEGEngine takes take of handling them. The SetTimer action has parameters, time in milliseconds, a timer id and the Group to raise the Timerfired Event. After the set time has expired an asynchro-nous event with data time id is raised. Links are registered to react to TimerFired events, completing the cycle. Timers are implemented with the .Net Timer class. From the SetTimer action we will instantiate a MHEGTimer object which encapsulated all the associated data of the Timer. This is the code that sets the timer. Timer timer = new Timer( new TimerCallback( this.DoTimer ), MHEGtimer, MHEGtimer.FireTime, System.Threading.Timeout.Infinite );

Figure 52 - C# Timer definition

The TimerCallback delegate is the method to call when MHEGtimer.FireTime mil-liseconds have passed. MHEGtimer is an instance of MHEGTimer and hold all as-sociated data like timer id and what group to fire on. The Time.Infinite means the timer will only fire once. When the above timer is created is splits off onto another thread and sleeps the set amount of time. It then calls the DoTimer method which registers an Asyn-chronous event. As can been seen there can be many threads running concurrently, at least one for the GUI, one for the Engine, one for each Timer, and possibly more. Hence the code in MHEGEngine needs to be kept thread safe. This is not trivial as unneces-sary locking need to be avoided as well.

7.3.9. Issues

Problems reading text files MHEG-5 states that text files referenced by Text objects are to be encoded in Uni-code UTF8. This is fine as the .Net text reader uses Unicode natively. The MHEG-5 specification also states that text can contain markup, (to change colours), and these are specific byte values not used for text. For example 0x1B is the start of a markup, and these are littered in the text. Initially the text was read in using default settings, and markup was parsed. How-ever full parsing was not possible due to apparent erroneous data. On inspection it was found that some of the markup codes are greater than 127, but since they were being read in as UTF8 the system assumed they were multi-byte Unicode

M5EG - -

75

characters. Instead they should be interpreted as a single character as they are not text. The example below illustrates this. Real Markup in Hex What was read in

0x1B, 0x43, 0x4, 0xFF, 0x0, 0x0, 0x0 0x1B, 0x43, 0x4, 0xFF, 0x0, 0x0 The .Net text reader thought the 0xFF was the start of a multi byte UTF-8 charac-ter, and hence was combining two markup codes. A quick fix was to read the text in as ASCII, to give one to one mapping of bytes to characters. This is not ideal as the text surrounding the markup is encoding in UTF-8, and so higher Unicode characters like ‘£’ would not get interpreted correctly. The best solution was to read the bytes in as a binary stream and sort the text from the markup. The text is then turned into a Unicode character array, while the markup is interpreted as an ASCII character array.

7.4. The Renderer

The Renderer, like the Parser, is split into two layers, a generic layer that inter-faces with the Engine, and plugins that do the actual rendering.

The plugins implement IRederer and is decorated with the AttPlugin attribute. Each plugin takes a List of Visibles as input and returns a bitmap which represents the scene.

Figure 53 - Renderer detail

M5EG - -

76

In the Engine MHEGDisplayStack holds references to all Visibles currently dis-played in the application. It also contains helper methods related to handling Visi-bles. When an object that derives of Visible is prepared it registers itself in MHEGDisplayStack. In the main loop of MHEGEngine, after it has processed all events it calls a method to update the screen. The simple way would be to call the redraw method of IRenderer after every potentially display changing action. This is not efficient as we can combine these full screen redraws and only request it when all events have been processed. Another technique used is partial redraws. Normally each action that could change the display would trigger a redraw of the whole screen. However, most actions only affect a confined area of the screen. For example changing the colour of a rectangle only alters the areas of the screen in which the rectangle is present. Similarly moving a rectangle to a new position only alters the area where the rec-tangle was originally and the area where the rectangle has been moved to. If we request only the areas that have changed to be redrawn, the Renderer can reduce expensive redraws. To facilitate both optimisations, the MHEGDisplayStack class has a buffer to store requested partial repaints. Each display altering action makes a list of rectangles that need to be redrawn. For example move rectangle A to A’ as below.

This action will store in MHEGDisplayStack the two rectangles, as these areas need to be re-drawn. Similarly other actions will do the same, like resize, which stores the bigger size, and Unload which stores the old area, etc. After the main loop has processed all events it calls UpdateScreen which does the following.

Figure 54 - Render optimiser detail

M5EG - -

77

1. If the screen is unlocked continue (MHEG-5 applications can lock the

screen to programmatically stop updates). 2. Calls Flush on MHEGDisplayStack 3. This iterates though the redraw buffer compacting it. Combining rectan-

gles completely obscuring others. 4. Returns the minimal list of rectangles to be drawn. 5. Clears the buffer. 6. Calls the Visibles property on MHEGDisplayStack. 7. This iterates though the ordered list of visibles only returning Visible ob-

jects that can be seen. So ones that are completely obscuring others are discarded, this is not trivial as transparency need also to be considered.

8. Makes a clone of the object to return, as the Engine could start processing more events later that affect the Visibles and the Renderer is running asynchronously on another thread.

9. Calls the Update delegate on the GUI thread asynchronously with the above two items as parameters. This is done so the Engine does not block while redrawing, and also because of the .Net requirement that GUI con-trols are only accessed from the thread they are created on.

After the above process the Engine continues Event processing, and the Renderer now has to worry about displaying the results.

7.4.1. The IRenderers

Only a GDI+ IRenderer was developed not only, as this was in the minimal re-quirements, but also because it was sufficient for our needs. The IRenderer uses GDI+ to draw of GDI+ surfaces implementing double buffer-ing. This was achieved with having a back buffer which all drawing initially takes place, once this is complete the back buffer is blitted to display context. .Net 2.0 makes this simple by providing inbuilt support for multiple buffers. All drawing is down by classes implementing the IDrawer interface to a Buffer-edGraphicsContext object for the back buffer. Each IDrawer is responsible for rendering a particular visible. When the redraw request occurs, the areas needing redrawing are cleared on the back buffer with black, the default colour of the background. Then the Renderer iterates through each Visible in order working out if it is in an active areas. If so it is redrawn by the appropriate IDrawer. Anyone familiar with Windows graphics programming will know that GDI(+) con-texts need to be redrawn when the operating system invalidates the window of the applications. In .Net the OnPaint method is called on the control. This is over-ridden with code to just re-blit the back buffer saving an expensive full redraw.

M5EG - -

78

The GDI+ IDrawers use appropriate GDI+ methods to draw their respective Visi-bles. For example the Rectangle IDrawer uses the GDI+ method DrawRectangle and FillRectangle. The Drawer super class has generic functionality common to all Drawers.

Figure 55 - Drawer class model

7.4.2. Issues

Text Drawer Initially the Text Drawer just used GDI+ DrawString to print characters, however this was found to give not enough control of how text was places. For example it was not possible to specify line spacing. The MHEG-5 specification especially UK Engine Profile 1.06 supplies extensive documentation on how to place text so each implementation has identical text characteristics. One option was to completely code a text Renderer from scratch that would draw bit for bit on the GDI+ canvas. This was taken to be too drastic hence a simpler approach was taken. The string contained in the Visible Text objects was split into sections that could be drawn accurately by the GDI+ DrawString method. The main advantage of this over a complete rewrite is that functions that GDI+ does perfectly well can still be used, for example font hinting and subpixel anti-aliasing. These are complex traits that are benefited by GDI+ and hard to replicate. A text layout method is used to split up the Visible string into separate lines that can be drawn the correct distance apart. It also has to work out line wrapping by only breaking on the correct white space as defined in UKEngineProfile. Colour markup is also incorporated by decoding the colour codes, changing the colour and putting subsistent text on a new line. UKEngineProfile also defines how big tab stops are, 45 pixels, and this is implemented exactly. One area that caused problems was that UKEngineProfile required text to have non-square pixels. There is no functionality in GDI+ to support this, a work around could have been used. It would draw the text one to one to a separate image, and then this image would be rescaled and drawn to the back buffer. It was found that simply using a slightly smaller font size achieved the same effect.

M5EG - -

79

M2V images While PNG images are easy to deal with, M2V images present their own problem. There is no useable library from C# that allows them to be read in as images. This is mainly because a M2V image basically one I-Frame from an MPEG2 video in a video container. The only available method is to open the file as a video and then capture the first frame. This is quite easy to do using DirectShow.Net. Non Square Pixels Because a TV has non-square pixel, the entire canvas looks distorted on a PC screen. The solution to this is rescale the entire canvas by making a little thinner. Also over-scan was incorporated, as the outer 5% of the screen can sometimes contain junk that would not be seen on a TV screen.

7.4.3. DC-DVB Source

DC-DVB Source was the DirectShow filter used to encapsulate TV card functional-ity. DirectShow is a C++ API and to use it from .Net the DirectShow.Net library was used. This is a project that written definitions for each interface in Direct-Show for C#. This allows the original DirectShow API be to be called natively from C#. The video functionality in the M5EG container application is based on the PlayWND sample from the DirectShow SDK. It essentially is a simple video player that can play any media that has DirectShow compatible filters. Since DirectShow is the main multimedia framework on Windows (Windows Media Player, Media Centre, Zoom Player, etc are all DirectShow based) there is tremendous support for any type of video file. Only a .dvb file, the format for DC-DVB, needs to be ‘played’. DC-DVB initially tunes to the last selected channel, to change channels and more importantly get the root directory of the MHEG application we need to control DC-DVB. It implements the DirectShow interface IAMMediaSelect, which allows channel changing by pressing the next/ previous chapter buttons. But finer control is needed, ie to tune to specific frequencies. This is provided by IDVBSource, which DC-DVB implements. The problem is this is not a DirectShow interface but one specific to DC-DVB. Also it is written in Delphi, the implementa-tion language for DC-DVB. To use it from C# we need to port the interface over, like what DirectShow.Net does for C++ DirectShow. This was experimented to al-low full control over DC-DVB. Once we have video in our application, we need to mix it with MHEG-5 program. This can be done by the IVideoMixer9 API part of DirectX9. It is an advanced mixer that used the 3D pipe for blending allowing full alpha channels and high

M5EG - -

80

speed. The video and image are both treated as textures and are mixed onto the screen. One issue what that GDI+ contexts when used for the image input cannot have full alpha transparencies, only a simple one colour is fully transparent system. To get full alpha the GDI+ context needs to be transformed into a DirectX context. This is not trivial and was only experimented.

7.5. Logging framework

Log4Net [33] was used to provide complex logging support to every subsystem. Logging can be filtered by importance and a variety of views are available to see the results. This was essential when debugging the engine as following multiple threads is not easy.

7.6. Unexpected Discoveries

7.6.1. Compressed XMHEG programs

One of the limitations of the XMHEG formation for MHEG-5 programs is the in-credibly size difference compared to the ASN1 notation. In the course of imple-menting a XMHEG parser it was found that much of XMHEG programs repeat highly in the tags. This trait led to the experimentation of compression, and massive reductions in size were available, see Chapter 7 - Evaluation for more details.

Renderer DC-DVB

IAMVideoMixer9 M5EG

To Dis-play

Graphics Memory

Figure 56 - Video Mixer detail

M5EG - -

81

A nice way to implement compress for exchanged XMHEG program would be to compress the entire folder containing the application, rather like a JAR file. This would also compress any associated text files, and images, and would be trans-parent as the decompression would occur once in the initial parse. Also programs would the usability benefit of one file would could be double clicked to launch the program for example.

M5EG - -

82

8. Evaluation

8.1. Overview

This chapter will try to measure the success of this project against a number of qualitative and quantitative metrics. Success against original specification will be evaluated and comparisons to existing implementations will be made where ap-propriate.

8.2. Specifications Adherence

An important aspect for M5EG as with any MHEG-5 engine is how closely it follows the MHEG-5 specifications. These are documents ISO/IEC 13522-5[12] and version 1.06 of the UK Engine Profile[13].

8.2.1. Class coverage

Class coverage refers to how many of the classes defined in the two specifications are actually implemented. The following describes class coverage with respect to Uk Profile 1.06. Some classes should NOT be implemented to be compliant with 1.06, and these were totally ignored. Fully implemented Class name

Root : Abstract OctetStringVariable Group : Abstract ObjectRefVariable Application Presentable : Abstract Scene TokenManager : Abstract Ingredient : Abstract TokenGroup Link Visible : Abstract Program : Abstract Bitmap ResidentProgram LineArt Variable : Abstract Rectangle BooleanVariable Text IntegerVariable Action

Partially Implemented Class name Notes

ListGroup Audio Video DynamicLineArt Only Draw rectangle implemented

Not Implemented

M5EG - -

83

Class name

Interactive EntryField Slider HyperText

Classes that should NOT be implemented to be UKEngineProfile 1.06 compliant This requirement is due so that when they are specified, all implementation will work in a standard way. Class name

RemoteProgram RTGraphics InterchangedProgram Button Pallette HotSpot Font PushButton CursorShape SwitchButton

M5EG is fully compliant by ignoring these classes and has the ability to be easily extended to allow future access to these.

8.2.2. Class Statistics

Fully Implemented Partially Implemented Not Implemented Total

21 4 4 29 % Fully Implemented % Partially Implemented % Not Implemented Total

72% 14% 14% 100%

M5EG covers partially or fully 86% of the specification. The reason that M5EG does not support some classes it due the rarity of their use in actual broadcast applica-tions. Any subclasses of Interactible were never observed in the ‘wild’, also List-Group was only seen once and there was simply not enough time to fully imple-ment and more importantly test this class. A fairly complete stub was made that could be easily extended once more examples are broadcasted. In this light of comparing against what is actually broadcast, M5EG achieves near 100% coverage.

8.2.3. Comparisons to existing systems

RedButton “It'll only draw text, rectangles and PNG bitmaps at the moment and some of the ElementaryActions are not yet implemented. However, it is enough to be usable. The main things missing are video and audio streams, …” [34] M5EG is advantageous in that we support M2V bitmaps and all elementary actions associated with the classes we support. M5EG also has initial support for video

M5EG - -

84

and audio, however it should be noted that the author of RedButton is working to add video/audio support. Mu From Steve Barhams’ report , Mu does not implement M2V files, Audio, Video and Interactibles. [20] While M5EG also does not support Interactibles, it has at least partial support for everything else. Set top boxes New set top boxes must have 1.06 compliant MHEG-5 interpreters, however many old models are 1.05 or 1.0 compliant. Set top boxes are advantageous to M5EG in the area of class coverage. It is noted that many set top receives have bugs, which due to their deep penetra-tion of the market, MHEG-5 application authors work around. The UK Engine Profile also defines various details of the running engine. Two different event processing models are defined, with M5EG implementing the preferred one. Default error handling is also implemented is a compliant way.

8.3. Qualitative Testing

8.3.1. Rendering Quality

One of the major successes of this project was the rendering module. By using ex-isting Windows graphics APIs to their fullest extent, very high quality rendering was possible and techniques such as sub pixel anti-aliasing were able to be utilised without much effort. Text

M5EG - -

85

Figure 57 - BBC text rendering sample

Zoom in

Figure 58 - BBC M5EG text rendering zoom

Notice the sub-pixel anti-aliasing to enhance readability.

Figure 59 - BBC 37WLT56 text rendering zoom

M5EG - -

86

(Picture taken with camera, so colour is off and could not get a pixel for pixel comparison) It can be seen that text definition is good on M5EG. The set top box implementa-tion, believed to be Cabot’s, does no kerning or anti-aliasing. This is mainly due the limited resources available on embedded hardware. Image Quality Images and video can be scaled using highly sophisticated techniques, like bi-cubic, which are not available on set top boxes due to limited resources. Also PNG images are rendered in a true colour buffer, which is only optional in the UK Engine Profile.

8.3.2. Rendering Accuracy

Colour The UK Engine profile goes into detail about how to render colour and what col-ours should be available.

As explained in Chapter 4 a minimum of 188 colours must be available at mini-mum. M5EG supplements this with true colour (8bits per channel) support in all rendering.

Colour spaces are also mentioned with appropriate mixing. M5EG renders everything in an internal RGB full gamut buffer (0-255 allowed for each channel). DirectShow and DirectX are responsible for M2V and MPEG2 video rendering and handle all YUV to RGB conversions and gamut extending (ie Ex-panding 16-240 to 0-255).

8.3.3. Coordinate Accuracy

MHEG-5’s display model defines a 720x568 canvas on which Visibles are pre-sented. Each item need to be drawn to pixel accuracy and M5EG implements this. Text The UK Engine Profile is especially verbose upon text handling, as most MHEG-5 applications are primarily text based and also due to the difficulties of reading text on a low resolution TV screens.

M5EG tries to be as compliant as possible with as much as support as possible of the profile.

M5EG - -

87

The following is supported: Supported Features Not Supported Features

Left, Centre, Right justifications Variable character spacing Variable line spacing Full justification (Optional is 1.06) Colour Markup Other start corner (Optional in 1.06) Word wrapping Vertical Line orientation (Optional in 1.06) Top Left Start Corner Horizontal line orientation Tab stops of 45 pixels

Again the not supported features are rarely use, and only variable character spac-ing has been seen in the wild. Text rendering is highly accurate however there is a slight problem. UK Engine Profile specifies a specially created font Tiresias, but the version available on the PC (8) is slightly wider than the ones used on set top boxes (7.51). Also text aspect correction of 56/45 for horizontal font points is not implemented. M5EH uses a slightly smaller font size than specified to achieve the same effect.

8.3.4. Usability of the M5EG application

While M5EG original goal was to be a framework that would allow MHEG-5 abili-ties to be easily added to existing software, a demo application also called M5EG was created to showcases the abilities of the M5EG framework. This application was emulates a set top box and provides functionality to run MHEG-5 applications. A way to measure the usability of this application is Niel-sen’s 10 Usability Heuristics. The following is an analysis of the M5EG application. 1 Visibility of system status 2 Match between system and the real world 3 User control and freedom 4 Consistency and standards 5 Error Prevention 6 Recognition rather than recall 7 Flexibility and efficiency of use 8 Aesthetic and minimalist design 9 Help users recognise, diagnose and recover from errors

M5EG - -

88

10 Help and documentation

M5EG - -

89

Figure 60 - Screen GUI screenshot

4. GUI windows al-ways same

7/8. GUI is minimal and efficient to use

1. System status is always seen

2. Mimics

remote control

3. User can do what they want

9/10. Help always available

6. Match MHEG -5 application

5. System designed to prevent user errors

Figure 61 - Remote GUI Screen-

shot

M5EG - -

90

8.3.5. Accessibility

Some simple accessibility characteristics come from observing standard rules like providing keyboard shortcuts, and good design of UI. More profound abilities come from dynamically adapting MHEG-5 applications. One such approach that was in the project specification was zooming. This is available from a simple button, and is based on the fact we are using a rich API for rendering.

Figure 63 - Bottom Zoom screenshot

Also experiments were made with a screen reader, but not fully realised due to complexities.

8.3.6. User Testing

An experiment was devised were users were observed using the application. No goals were given, only that they should use it in a similar way to DTV. These were their comments: User A

“The GUI is nice and minimal. I like the fact that I can scale the application. The content looks exactly like a set top box and there is no noticeable delay. More information about the state of the program may be needed, like load-ing. Also error messages should be given if a incomplete file is used.”

User B

Figure 62 - Top zoom screenshot

M5EG - -

91

“The program is good, but could be faster. Some applications do not work but most are ok. Proper video support would be nice, but the included func-tionality is ok.”

8.3.7. Perceived Performance

Perceived performance is how responsive the application is perceived to be. Due to use of multithreading, the application never blocks, and is perceived to be run-ning fine. All decoding and processing is done in the background, so the user never has to see the application working. Also double buffering means half states are never seen. In this respect M5EG is perceived to be quite fast.

8.3.8. Product Value

One way to measure the usefulness of technology is to measure how much value is seen in it from people complexly unfamiliar with the technological details. A question was posed on a message board that if a MHEG-5 interpreter was avail-able as a plugin for Windows Media Centre, would you be interested in it, and how much would you be willing to pay if anything? Several comments were made; many would be interested if extra features were available that are not possible on set top boxes, like the accessibilities ones de-scribed above. Some would settle on just basic functionally. Overall the response was quite positive, with one saying “I use digital text a lot on my TV, so I do find it frustrating that you can't do it in MCE….” “…I would like to use it in a 10ft, just like you would on a TV…” And continued that he would be prepared to pay £20 for such functionality.

8.4. Quantitative Testing

8.4.1. Speed of M5EG

These are results from timing code added to M5EG for profiling. It shows exactly where time is spent when parsing and rendering several files. All tests were run on a 3.0 GHz P4 1GB Ram. All tests were repeated 20 times and averaged. A fairly representative range of MHEG-5 scenes were chosen, from the simplest and smallest to the most complex.

M5EG - -

92

0.00

20.00

40.00

60.00

80.00

100.00

120.00

140.00

160.00

180.00

0 2 4 6 8 10 12

MHEG-5 size / KiB

Ren

derin

g F

ram

es/s

MHEG-5 File Size of ASN1 (KiB)

Parsing (ms)

Parsing (KiB/s)

Rendering (ms)

Rendering (Frames/s)

enh_gateway.mhg 4.33 552.6 7.84 32.6 30.67 bridge_etv 10.7 1112.4 9.62 89.3 11.20 splash.mhg 2.74 320.9 8.54 11.8 84.75 tv.mhg 5.18 744.1 6.96 46.3 21.60 offair.mhg 0.79 120.3 6.57 6.1 163.93 ITV/a.xml 10.7 968 11.05 96.2 10.40

We can see that parsing takes a considerable amount of time, especially

This graph shows how the parsing process scales with file size. It can be seen that pars-ing rate in KB/s decreases line-arly with MHEG-5 complexity

This graph shows how rendering per-formance decreases increases with more complex scenes. It is assumed that larger MHEG-5 files are more complex.

To show the benefits of caching on the parser we measure how many times the file cache is accessed on various applications on a predefined set of user input. MHEG-5 File Non cache Cache Proportion from Cache

BBCi 32 38 54%

0

2

4

6

8

10

12

0 2 4 6 8 10 12

MHEG-5 size/ KiB

Pa

rsin

g t

ime

/s

Figure 64 - Graph of parsing time v size

Figure 65 - Graph of Rendering time v size

M5EG - -

93

ITV 18 9 33%

The Hits! 4 1 25%

We can see the benefit is most evident on large applications that inevitably keep accessing the same file again and again, usually the start page.

8.4.2. Speed comparisons

Of interest would be performance comparisons to set top box as this is what MHEG-5 is viewed on 95% of the time. An IDTV is used (Toshiba 37WLT58) and the software engine is believed to be Cabot’s. As a TV is a consumer device we cannot probe into the inner workings so we will test overall performance rather than in-dividual modules. The test,

1. Measure initial loading times to start application. Time should be given for a complete DSM-CC rotation so only testing interpreter not the DSM-CC implementation.

2. Time to load navigation between various scenes. Various scene to scene transitions are made inside the application and then averaged.

The whole process is repeated 20 times. The following table summarises the results. Identical MHEG applications were used for both Application Load M5EG IDTV

BBCi 3.41 2.61 Sky 2.14 2.66 The Hits! 2.69 1.3 Scene to Scene M5EG IDTV

BBCi 1.32 1.86 Sky 1.98 2.32 The Hits! 0.84 1.65 It can be seen that while M5EG is slower on the initial load, probably due to the translation between ASN1 and XML it is faster on internal transitions. This is likely to be because of the fact everything is in memory and cached which can save costly parsing on many occasions.

8.4.3. Compressed XMHEG size reductions

Compressing XMHEG was suggested earlier as a way to reduce space. The com-pression system used was zip.

M5EG - -

94

Here are the results of various comparisons MHEG-5 File ASN1 (KiB) XMHEG (KiB) Compressed XMHEG (KiB)

enh_gateway.mhg 4.33 121 4.38 bridge_etv 10.7 269 8.90 splash.mhg 2.74 58.4 2.97 tv.mhg 5.18 135 5.97 offair.mhg 0.79 18.9 1.43 ITV/a.xml 10.7 525 16.86

From the above data it can be seen at in all cases the compression gives big size reductions with the data approaching the size of the ASN1. Of note is that in some cases the compressed XMHEG is smaller than the ASN1. This can be attributed to the fact that included content, such as string cannot be compressed in ASN1. So MHEG-5 files with large numbers of string may be smaller in compressed XMHEG rather than ASN1 form. Note than compression in the above has only be applied to the actual MHEG-5 files, not any associated data. Further reductions are possible because a large proportion of other files are usually text.

8.4.4. Resource Requirements

One of the benefits of MHEG-5 has always been is frugal resource requirements, can the same be said of M5EG? Firstly M5EG has at least the same requirements of the software libraries used, for example the .Net virtual machine and DirectX. Various use showed that M5EG was using 84 MB and 40% of CPU time when run-ning at full steam on a 3.o GHz Pentium 4 machine with 1 GB of ram. While this is significantly more that the 50 MHz and 4MB ram minimum that an embedded MHEG-5 can run on, this is not a fare comparison as the embedded so-lution has hardware highly optimised for the purpose, and software written in low level languages like assembler. Similarly the benefits gained, like anti–aliasing, are deemed to outweigh their higher resource usage.

8.5. Overall Analysis of Project

This section tries to tie, what was achieved with the original project goals.

M5EG - -

95

8.5.1. Compliance with Project Specification

The specification outlined in chapter 4 detailed various functionalities that had to be implemented. These were either basic, that had to be completed, or advanced that time permitting, may be implemented. General Requirements All of G1, G2 and G3 were implemented by the M5EG. G1 required the use of three modules, while G2 specified the use of a separate DSM-CC. G3 was to create an application which demoed the M5EG framework. AG1 which required the integration into an existing media player was not done due to time constraints.

Technology Requirements T1(C#), T2 (ASN1 parser), T3 (XSS4J), T4 (GDI+ renderer) were all observed by the M5EG application. AT1 (XMHEG parser) was realised. The rest of the ATs were not implemented due to time constrains. The M sections defined standard compliancy, which is detailed above. Usability Requirements M5EG is compliant with all basic Us, U1 (2 foot UI), U2 (execution of stored appli-cations), and U3 (reliable execution). It also implements AU1 the 10 foot interface, and partially implements AU2, AU3 and Au4 functions relating to live capture and video. Accessibility Requirements C1 (basic accessibility) and AC1 (zooming) are implemented. AC2 (screen reader) was deemed to be too complex for time permitted. Using a text to speech engine is easy, but trying to get it to read the correct text is difficult as it requires the un-derstanding of a scene. Added Value Requirements V1 (Rendering quality) was met, as shown above. V2 (structured content) while explored was found to be problematic. EPG data, while not part of MHEG-5, is eas-ily readable and DC-DVB already does this. MHEG-5, however does not standardise the content, only how to present it. Hence, each application uses its own specific way of storing data, like news. Some have text files, while others hard code into the MHEG-5code, etc. Hence it is almost impossible to extract data in a systematic way.

M5EG - -

96

8.6. Project Statistics

In total there were 14870 lines of net code, with 1381 lines of comments. (…and counting.)

M5EG - -

97

9. Conclusion

9.1. Overview

This section will tie all previous chapters and conclude the project. Future direc-tions will also be considered.

9.2. Original Goals

The original goals were,

To implement a framework that can interpret MHEG-5 on Windows

To create a application that showcased this functionality

To explore any additional value This project proves that MHEG-5 content can be made available on the computer in an easy to use form. While the engine did not support the entire standard, it was enough to be highly useable in nearly all over the air real world applications. The example application showed how to put the modules of the framework to-gether. While TV decoding is not part of MHEG-5 (it is a separate function) the demo application has some basic video decoding ability. It main purpose was to show how video and MHEG-5 applications could be ‘blended’ together to arrive at an experience similar to a set-top box. An original goal was also to explore additional value. Several experiments were made but one of the most interesting was the accessibility aspects. Zooming is incredibly useful to people with poor vision and can be implemented in an elegant way. Another addition of value was the high quality rendering, especially text, and the highly responsive engine. These were possible due to significantly more resources being available as compared with a set top box. It makes facilities like extensive caching possible, and it also means high quality rendering can be used without noticeable delay. The project’s greatest success perhaps is that is gives a stable platform for decod-ing and running MHEG-5 which other developers could use in an easy way. Whilst there was no time to integrate with existing 10 foot application, this is no doubt possible. The added value combined with potentially embedding M5EG into an existing 10 foot interface would make a compelling argument for using a computer to pro-

M5EG - -

98

vide interactive TV at the expense of a set top box. Since no similar products are available, a market opportunity may exist.

9.3. Original Contributions

9.3.1. The M5EG Framework

The M5EG Framework is at heart of this project and provides an extremely useful set of libraries that can be used in many contexts. An entire MHEG-5 Engine is provided, as are renders and parsers. These can be combined and used in many situations including plugins for other applications. The system is also extendable for example a HTML render could be developed by a third party without access to the source code.

9.3.2. The M5EG Application

The M5EG applications not only provided a clear example on how to use the M5EG Framework, it is a usable application in its own right. It gives a 2 and 10 foot experience for executing MHEG-5 application as part of a TV stream. Basic TV con-trol functionally is also implemented.

9.3.3. The compressed XMHEG format

Compressing the folder of an XMHEG MHEG-5 application significantly reduces size making it comparable to ASN1 efficiency. This file format could be used to also allow double click semantics to MHEG-5 applications, rather like JAR files. It also makes them easier to distribute over the web say.

9.4. Future Work

9.4.1. Finish Specification

M5EG currently is not fully compliant with the UK Engine Profile 1.06. Extra work could easily achieve full compliance. M5EG could also be submitted to DTG for compliance testing and logo approval.

9.4.2. A plugin in for Windows Media Centre

One of the aims with creating a framework is that it will be useful in other applica-tions. One use with wide ranging benefit is a plugin for Windows Media Centre. This would provide the functionally of M5EG in a place where normal users would actually use.

M5EG - -

99

Also many other applications could be possible, such as a plugin for a Web browser or distributing MHEG-5 application over a network. Because M5EG is a framework that exposes the low level details of MHEG-5 application in a high level form, it empowers the third party developer with a powerful tool that they can use in ways not imaged yet.

9.4.3. Additional Renderers and Parsers

It would be nice if additional Renderer and Parser were implemented. The frame-work allows third parties to do this as well. For example a Windows Presentation Foundation based renderer would integrate very nicely with Windows Vista, and allow graphics to be accelerated by the 3D pipe. A textual parser would complete the suite, and a direct ASN1 parser would im-prove decoding performance.

9.4.4. Optimisations

Many areas of the framework could be improved in terms of performance and re-source usage. For example smarter algorithms could be used throughout, better caching techniques, and a smarter Engine could all improve M5EG. Extensive profiling would also be beneficial.

9.4.5. Added Value

Many advances features mentioned in the specifications were not implemented. This included things like bookmarks and a screen reader functions, and could be implemented in future. Also user consultation would be beneficial to improve M5EG in a user oriented way.

9.5. Final Words

This project has resulted in the creation of some very useful software. It has also been very interesting because of the exploration of many areas of the technologi-cal landscape. Some things were achieved that have never been done before. It is hoped that the creative contributions will be fully realised in the future.

M5EG - -

100

10. Bibliography 13522-1, ISO/IEC. “ISO/IEC 13522-1, `Coding of multimedia and hypermedia

information - Part 1:.” 1997.

13522-5, ISO/IEC. ISO/IEC 13522-5, `Coding of multimedia and hypermedia

information - Part 5:. ISO, 1997.

13522-5:1997/Cor.1:1999(E), ISO/IEC. “MHEG-5 Information technology - Coding

of multimedia and hypermedia information: Support for Base-Level Interactive

Applications. Technical Corrigendum 1.” 1999.

ASN1C. ASN1 sub compiler for C. http://lionet.info/asn1c/.

ATSC. “ATSC Official Specification.” http://www.atsc.org.

Barham, Steven. “Mu MHEG-5 interpreter project.” 2005.

BBC. “Designing for interactive television v 1.0 BBCi & Interactive TV

programmes.” 2006.

BBC. Digital Terrestrial Television MHEG-5 Specification Version 1.06. BBC, 2003.

BBC-News. http://news.bbc.co.uk/1/hi/entertainment/tv_and_radio/4247622.stm.

Cabot. Cabot Communications Limited Mercator Engine. 2006.

http://www.cabot.co.uk/products/mheg5/.

DTG. “DTG press release.” 2006.

http://www.dtg.org.uk/news/news.php?class=countries&subclass=193&id=1510.

—. “EuroMHEG DTG & Digitag.” 2001.

http://www.dtg.org.uk/reference/euromheg1.pdf.

DVB. “Official Digital Video Broadcasting Specification.” www.dvb.org.

Euchner, Joachim. “Integration of MHEG into the WWW an MHEG-Engine written

in Java.” 1997. http://www.prz.tu-berlin.de/~joe/mheg/mheg_engine.html.

IBM. “Mapping Between ASN.1 and XML, IBM Research Report RT0362.” 2000.

http://www.trl.ibm.com/projects/xml/xss4j/docs/RT0362.pdf.

ISO/IEC13522. ISO/IEC 13522 `Coding of multimedia and hypermedia information'.

ISO, 1997.

ISO/IEC-13818-6. “Generic coding of moving pictures and associated audio

information. Part 6 - Extensions for DSM-CC.” 1998.

ITC-Review. http://www.itc.org.uk/documents/upl_155.doc.

Kilvington, Simon. RedButton Engine. 2006. http://redbutton.sourceforge.net.

libfreeMHEG. A viewer for UK interactive television.

http://sourceforge.net/projects/freemheg.

MHP. “MHP press release.” 2001.

http://www.mhp.org/documents/presseleases/pr087.DVB%20IFA%20Press%20Relea

se.010824.pdf.

—. “Multimedia Home Platform Specification.” http://www.mhp.org.

OpenMHP. OpenMHP. http://www.openmhp.org/.

OpenTV. “OpenTV Specification.” http://www.opentv.com/products/mw_core.html.

Redkey. Digital Television Software. http://www.digitaltelevisionsoftware.co.uk/.

Salloway. “Media centre solitaire screenshot.” 2006.

http://www.salloway.org.uk/MediaCenter/2004/images/PowerToys/sol.JPG.

Wikipedia. http://en.wikipedia.org/wiki/Teletext.

Wikipedia. “ISBD System.” http://en.wikipedia.org/wiki/ISDB.

M5EG - -

101

11. Appendix A

These are the full listings of the example code.

// 'Hello world' written in MHEG-5 // Originally from http://www.digvid.info

{:Application ( '/startup' 0 ) // Application content reference :Items ( {:Link 1 :EventSource 0 // Check this application... :EventType IsRunning // ... for the IsRunning event :LinkEffect ( // Load the scene :TransitionTo (( '~/hello.mhg' 0 ) ) ) } ) :BackgroundColour '=FF=FF=FF=00' // White :TextCHook 10 // Default text content hook :TextColour '=00=00=00=00' // Black :Font "rec://font/uk1" // Font to use for text rendering :FontAttributes "plain.26.32.0" // Default font attributes :BitmapCHook 4 // Default bitmap content hook } {:Scene ( "~/hello.mhg" 0 ) :Items ( // Declare a background Rectangle that covers the screen. {:Rectangle 1 :OrigBoxSize 720 576 // Size of rectangle :OrigPosition 0 0 // Position at top left :OrigRefLineColour '=ff=ff=ff=00' // White :OrigRefFillColour '=ff=ff=ff=00' // White } // Place a Text box on the screen {:Text 2 :OrigContent "Hello World!" // Text to display :OrigBoxSize 300 50 // Size of text box :OrigPosition 200 100 // X,Y position :FontAttributes "plain.36.42.0" // Use large characters :TextColour '=ff=00=00=00' // Red } // Define a Link that triggers when the user presses the Blue key to // Quit the application. {:Link 3 :EventSource 0 // Source is this scene :EventType UserInput // Event type that we are looking for :EventData 103 // 103 for the blue key :LinkEffect ( :Quit ( ( '~/startup' 0 ) ) ) } ) :InputEventReg 3 :SceneCS 720 576 }

Xml Version

M5EG - -

102

<?xml version="1.0"?> <!DOCTYPE interchangedobject SYSTEM "xmheg.dtd"> <interchangedobject> <application> <root> <external-reference> <group-identifier encoding="UTF8">~/startup.xml</group-identifier> <object-number>0</object-number> </external-reference> </root> <items> <link> <root> <internal-reference>1</internal-reference> </root> <link-condition> <event-source> <internal-reference>0</internal-reference> </event-source> <event-type>4</event-type> </link-condition> <link-effect> <action> <transition-to> <target> <direct-reference> <external-reference> <group-identifier encoding="UTF8">~/hello.xml</group-identifier> <object-number>0</object-number> </external-reference> </direct-reference> </target> <connection-tag-or-null> <null></null> </connection-tag-or-null> </transition-to> </action> </link-effect> </link> </items> <default-attributes> <default-attribute> <text-content-hook>10</text-content-hook> </default-attribute> <default-attribute> <font> <direct-font encoding="base64">cmVjOi8vZm9udC91azE=</direct-font> </font> </default-attribute> <default-attribute> <font-attributes encoding="base64">ABgcAAA=</font-attributes> </default-attribute> <default-attribute> <text-colour> <absolute-colour encoding="base64">////AA==</absolute-colour> </text-colour> </default-attribute> <default-attribute> <bitmap-content-hook>4</bitmap-content-hook> </default-attribute> </default-attributes> </application> </interchangedobject>

Helllow World .Xml <?xml version="1.0"?> <!DOCTYPE interchangedobject SYSTEM "xmheg.dtd"> <interchangedobject> <scene> <root> <external-reference> <group-identifier encoding="UTF8">~/hello.xml</group-identifier> <object-number>0</object-number> </external-reference> </root> <items> <rectangle> <root>

M5EG - -

103

<internal-reference>1</internal-reference> </root> <original-box-size> <x-length>720</x-length> <y-length>576</y-length> </original-box-size> <original-position> <x-position>0</x-position> <y-position>0</y-position> </original-position> <original-ref-line-colour> <absolute-colour encoding="UTF8">white</absolute-colour> </original-ref-line-colour> <original-ref-fill-colour> <absolute-colour encoding="UTF8">white</absolute-colour> </original-ref-fill-colour> </rectangle> <text> <root> <internal-reference>2</internal-reference> </root> <original-content> <included-content encoding="UTF8">Hello World!</included-content> </original-content> <original-box-size> <x-length>300</x-length> <y-length>50</y-length> </original-box-size> <original-position> <x-position>200</x-position> <y-position>100</y-position> </original-position> <text-colour><absolute-colour encoding="UTF8">red</absolute-colour></text-colour> </text> <link> <root> <internal-reference>3</internal-reference> </root> <link-condition> <event-source> <internal-reference>0</internal-reference> </event-source> <event-type>UserInput</event-type> <event-data> <integer>103</integer> </event-data> </link-condition> <link-effect> <action> <quit> <target> <direct-reference> <external-reference> <group-identifier encoding="UTF-8">~/a</group-identifier> <object-number>0</object-number> </external-reference> </direct-reference> </target> </quit> </action> </link-effect> </link> </items> <input-event-register>3</input-event-register> <scene-coordinate-system> <SceneCoordinateSystem.x-scene>720</SceneCoordinateSystem.x-scene> <SceneCoordinateSystem.y-scene>576</SceneCoordinateSystem.y-scene> </scene-coordinate-system> </scene> </interchangedobject>