synchronization scheme in multimedia and …...synchronization scheme in multimedia and hypermedia...

Synchronization Scheme in Multimedia and Hypermedia

Application Development

R. N. Jugele1

Department of Computer Science, Science College, Congress Nagar, Nagpur.

Maharashtra

[email protected]

Dr. V. N. Chavan2

Head, Department of Computer Science, S. K. Porwal College, Kamptee,

Dist : Nagpur. Maharashtra

Abstract

Synchronization of multimedia is one of the important issue

in multimedia communication. The synchronization agency

triggers one of the three synchronization mechanisms, point

synchronization or real-time continuous or adaptive

synchronization in order to adapt to the run-time and life-

time presentation requirements of an application.

Multimedia communication deals with transfer of data over

the network among multimedia systems. These systems

include multiple sources of various media that are either

spatially or temporally related to create composite

multimedia documents. Spatial composition links various

multimedia streams into a single entity. Temporal

composition creates multimedia presentations by arranging

the multimedia streams according to their temporal

relationship, and the relationship is either loosely or tightly

coupled

Keywords

Multimedia, synchronization, object level, script, level,

MHEG, Cubic synchronization, Spatial, Temporal, Lip

synchronization, Live synchronization.

1. Introduction

Synchronization can be of two types:

i. Intra-object synchronization refers to the time relation between various presentation units of one

time dependent media object.

ii. Inter-object synchronization refers to the

synchronization between media objects.

Time dependent media objects usually consist of sequence

of information units, which is called logical data units

(LDU), it is classified as open and closed, closed LDU have

a predictable duration. The duration of open LDU is not

predictable before the execution of the presentation. For

delivering multimedia data correctly at the user interface

synchronization is essential.

In case of Live synchronization, the goal is to exactly reproduce at a presentation the temporal relation as they

existed during the capturing process. A typical application

of live synchronization is conversational services.

In case of Synthetic synchronization the temporal relations

are artificially specified. The emphasis of synthetic

synchronization is to support flexible synchronization

relation between media.

In case of Lip synchronization the temporal relationship

between an audio and video stream for the particular case

of human speaking. The time difference between related

audio and video units (e.g. frames) is known as the skew.

Continuous media (stream) such as audio and video are

characterized by well defined temporal relationship

between subsequent presentation units to be played, a

presentation unit is a logical data unit that is perceivable by

the user. Generally, the process of maintaining the temporal

order of one or more media streams is called multimedia synchronization. The problem of maintaining continuity

within a single stream is referred as intrastream or serial

synchronization, whereas the problem of maintaining

continuity among the streams is called as inter-stream or

parallel synchronization. The intrastream and inter-stream

synchronizations are necessary for both live stream(s) as

well as for stored media stream(s) presentations. This paper

describe the design method to the subject of

synchronization, which is properly sketched. It also

describe the topic with respect to the model and

architecture defined[15].

R N Jugele et al ,Int.J.Computer Technology & Applications,Vol 3 (5), 1818-1822

IJCTA | Sept-Oct 2012 Available [email protected]

1818

ISSN:2229-6093

2. Multimedia Synchronization

Synchronization in multimedia communications has been

studied and described extensively. The following carefully

selected literature has been used to describe

synchronization with respect to the designed process:

[1],[2],[3],[4],[5],[6],[7], [8],[9],[10],[11],[12],[13],[14].

When synthesizing the ideas presented in the above

mentioned literature, it seems there are four levels for

synchronization:

(a) the Script Level

(b) the MHEG Information Object Level

(c) the ACP level

(d) the OS level

The former two cover logical and the latter two treat

physical synchronization, concepts introduced by[9] in

1990. The physical marks synchronization between e.g.

sample packets in data flow while the logical identifies

synchronization between e.g. frames of a movie.

3. Elements of Synchronization

3.1. Synchronization Notion

Multimedia communications processes is also called as

synchronous, if the information entities keep together in

time. This condition is satisfied if the entities:

(a) arrive at the processor within acceptable delays.

(b) depart for presentation in the correct temporal sequence.

Two consecutive entities X and Y arrive within acceptable delay if their latency does not exceed a given tolerance.

Two consecutive entities X and Y depart in the correct

temporal sequence if Y=X^.next.

As may be clear, synchronization is done at the consumer,

whether information is synchronized at the producer is not

considered relevant.

3.2. Synchronization Tolerance

In case of synchronous synchronization, two entities X and

Y arrive within an acceptable delay if |X.time - Y.time| < ,

where is the allowed delay tolerance and X.time as well as Y.time are computed relative to the common timing

reference [3],[4],[5],[6],[7].

3.3. Synchronization Latency

Information entities which are transmitted to or from a

remote site, arrive at their destination with a certain latency

equivalent to: (a) for user-user interaction delay:

sample + encode + packet + transmit + buffer +

dcpacketize + decode + present

(b) for user-database interaction delay:

query + seek + access + packet + transmit + buffer +

depackctize + decode + present.

3.4. Synchronization Granularity A time-stamp is the grain of accuracy to which timing can be tuned: e.g. if this value is 1 ms then this means that

every millisecond, an operation can be performed.

The time-stamp is chosen relative to the finest granularity

needed, in general this will be the granularity of video

frames: 40 ms (PAL has 25 frames/ second), so every 40

ms it is possible to resynchronize information entities. At

the same time, it may be sensible to have a packet length of

40 ms, since resynchronization will then not be needing

extra buffering for packets which are disrupted during

processing and the developer may not have to think of how

to hold the entity presentation (e.g. holding a tone or

silence).

A transition point is a point where a specific kind of entity

begins or stops from being part of a multimedia

communications process.

On interrupt handling, it should be noted that when there

has been an interrupt of a process due to a demand for

synchronization, it must be clear what has to be done with conservation of the information which will be delayed and

how this may affect the presentation, this is the problem of

restricted blocking. According to[13],[14], while waiting

the presentation of an object may or may not be suspended

depending on the media type, also it is argued, the level of

synchronization is at stake: e.g. continuous synchronization

can be done between a text character and a voice sample,

but when there is blocking involved, this must be done at a

higher level like between the first written word of a

paragraph and its first spoken word.

3.5. Synchronization Strategy

An entity X to be played out at time t: information is

retrieved at time T, where T, such that T=t-R where R >= is a carefully chosen control value. The value of R

determines how large receiver buffers have to be. This

strategy is henceforth more suited to user-database

interaction.

3.6. Synchronization Cubic

Synchronization can be called synthetic or continuous involving discrete or continuous information (e.g. in

movies: subtitles or sound). At the same time,

synchronization can be intermedia (e.g. between voice and

text) or intramedium (e.g. between pieces of sound) and

interflow (e.g. between movies) or intraflow (e.g. within a

movie).



1819

ISSN:2229-6093

4. Script Level Synchronization

At this level[16] flows of information are treated and

operational pairs are accessible through some scripting

language[10] or ESTEREL [12]. The language is to provide

abilities like: delay specification, delay processing, input

acceptation, output transmission, interaction (serial/

parallel/ independent processes define serial/ concurrent/

arbitrary temporal relationships), repetition (sample

handling), anomaly recovery and interrupt handling.

Figure 1. Processes : Temporal Relationships

The developer/user may define (nonarbitrary) temporal relationships according to Object Composition Petri Nets

(OCPN) developed by Little and Ghafoor is shown in

Figure 1

5. Object Level Synchronization

There are various ways to deal with the synchronization of

several multimedia objects which have to fulfill certain

spatiotemporal user presentation requirements. However,

when synchronization at this logical level is at stake,

standardized procedures can be used. For structuring

reasons a special case will be treated first and then common

case will be explained.

5.1. Conditional Multimedia Synchronization

When certain events are generated by other objects than the

objects to be synchronized, by user-system interaction or by

using application's manipulation, sometimes, objects must

be conditionally synchronized. Based on this principle, a

so-called Conditional Action Set as shown in Figure 2 is

formed.

Figure 2. Structure of Conditional Action Set

This is a set of conditions and an action (operation) which

must be performed when the former are validated. The set

of conditions consists of one event i.e. a triple of event

type, object identifier and current state of the object and

zero or more additional conditions. For example, if the

elapsed time of a running object X equals 5000

milliseconds, then perform the action. Optionally, with

additional conditions, one can state precisely in which context the specified event may occur. When the associated

condition is true the objects transits to the next state and

actions are performed, thus performing a prescribed script.

5.2. Spatiotemporal Multimedia Synchronization

Because, within the framework of the standard, multimedia

synchronization pops up in relation to composite

objects[16]. With attributes in a composite object,

component objects can be positioned in space on a spatial

and in time on a temporal axis.

a) Spatial Positioning Spatial interobject relations can be shaped either in parallel

or in serial fashion, as shown in Figure 3. (note: MHEG

puts (0,0) in upper left corner). When parallel, all objects

are positioned with reference to one spatial base point in the

serial case, a latter object is placed according to the

former's spatial base point.

Figure 3. Spatial Interobject Relations

b) Temporal Positioning A division is made between parallel and serial temporal

interobject relations as illustrated in Figure 4. The situation

where positioning is done with reference to the composite

object's temporal base point covers the parallel case.

For the serial situation, the later object is activated after

zero or more delay with respect to its previous one's

temporal base point (typically, this will be a transition

point). This generic mechanism is included in the standard

to deal with general situations. However, to ensure more

compact coding, the standard also provides special tools to

optimize interaction for predefined subsets of time-relations

for situations that are likely to be frequently encountered.

These tools are atomic and elementary, as well as cyclic

and chained synchronization.



1820

ISSN:2229-6093

The maintenance of temporal relationships within a stream

or among the multimedia streams usually depends on the

following parameters[17],[18] :

(1) Network delays: The delays experienced by the

presentation units (PUs) in the network to reach its receiver,

which varies according to network load.

(2) Network jitters: Delay variations of inter-arrival of PUs

at the receiver due to varying network load.

(3) End-system jitters: Delay variations in presentation at the receiver due to varying CPU load and protocol

processing delays.

(4) Clock skew: The clock time difference between the

sender and the receiver.

(5) Clock drift: Rate of change of clock skew because of

temperature differences or imperfections in crystal clocks.

(6) Rate drift: Change in generation and presentation rates

due to server and receiver load variations.

(7) Network skew: Time difference in arrival of temporally

related PUs of streams, i.e., differential delay among the

streams.

(8) Presentation skew: Time interval in which the

temporally related PUs of the streams are presented.

Synchronization mechanisms are needed to cope-up with

these problems to ensure the temporal ordering of streams

and to maintain the presentation quality.

Figure 4. Temporal Interobject Relations

6. Application Control Program Level

Synchronization

The Application Control Program manages technical

heterogeneity of software, hardware and data. As a

consequence, this level deals with (synthetic or continuous

and intermedium or intramedium) intraflow

synchronization by managing timing relationships amongst

the elements of the integrated media.

From development viewpoint, synchronization at this

physical level is done between components: on interrupt

base, a component asks a composite for timing information

and if necessary, jumps to resynchronize.

An alternative solution for the synchronization problem is

to use separate virtual channels for each information type. Because the information types may be continuous or

discrete this has consequences for the grantability of the

data-rate and quality-of-service requirements.

Note as a reminder that continuous (discrete) information

requires for continuous (synthetic) synchronization: e.g.

video mail is a store-&-forward information service of

continuous media data thus the temporal relationships of

the communicators are synthetic but the synchronization of

the information entities is continuous.

7. Operating System Level Synchronization

Operating System manages geographical distributivity of

software, hardware and data. As a consequence this level

deals with (synthetic or continuous and intermedium or

intramedium) interflow synchronization by managing

timing relationships amongst integrated media.

From development viewpoint, synchronization at this

physical level is done between composites: on a regular

base, a composite asks a component for his tolerance and if

necessary, a jump of a component is issued to

resynchronize. The solution for the synchronization

problem is to use separate virtual connections for each

information stream[11].

Conclusion

The subject of multimedia synchronization has been linked

to the defined set of architectural components, which for this purpose has been split in a logical and a physical

section. Then, basic definitions and elements of

synchronization have been given in order to create a

common understanding. Finally, synchronization aspects at

the Script, Object, Application Control Program and

Operating System levels have been described.

We observe that synchronization scheme offers flexibility,

adaptability, reusability and maintainability. In this paper,

we analyzed the synchronization problems in multimedia

communications and proposed synchronization frame work

to handle synchronization mechanisms at application

service level depending on the life/ run-time presentation

requirements of the multimedia applications. Adaptive

synchronization mechanism adjusts playout times in

accordance with changes in network conditions and offers

better quality presentation by maintaining the sustainable

losses. This design provide flexible, adaptable and asynchronous mechanisms for multimedia communications.

The proposed technique can also be used for

synchronization of multimedia streams in mobile that

covers mobility features and resource scarcity of the nodes

that provides continuous and smooth playout at the mobile

devices.



1821

ISSN:2229-6093

References

[1] D.P. Anderson and G. Homsy A Continuous Media I/O Server

and its Synchronization Mechanism. IEEE Computer, Vol. 24 No.

10, October 1991, pp. 51-57.

[2] F. Kretz and F. Colailis Standardizing Hypermedia

Information Objects. IEEE Communications Magazine, Vol. 30 No.5, May 1992, pp. 60-70.

[3] T. D. C. Little and A. Ghafoor Multimedia Object Models for

Synchronization and Databases. Sixth International Conference on Data Engineering, 5-9 February 1990, Los Angeles (CA), USA.

[4] T. D. C. Little and A. Ghafoor Synchronization and Storage

Models for Multimedia Objects. IEEE Journal on Sel. A. in Communications Vol. 8 No.3, April 1990, pp. 413-427.

[5] T. D. C. Little and A. Ghafoor Network Considerations for

Distributed Multimedia Object Composition and Communication.

[6] T. D. C. Little and A. Ghafoor Spatio-Temporal Composition

of Distributed Multimedia Objects for Value-Added Networks.

IEEE Computer, Vol. 24 No. 10, October 1991, pp. 42-50.

[7] T. D. C. Little and A. Ghafoor Multimedia Synchronization

Protocols for Broadband Integrated Services. IEEE Journal on

Selected Areas in Communications, Vol. 9,No.9,December 1991. pp. 1368-1382.

[8] Various Coded Representation of Multimedia And

Hypermedia Information Objects, Part 1: Base Notation, Working Document S-6 ISO/IEC JTC1/SC29/WG12, June 1992.

[9] C. Nicolaou, An Architecture for Real-Time Multimedia

Communication Systems. IEEE Journal on Selected Areas in Communications, Vol. 8 (1990) No.3 (April), p.391-400.

[10] F. Oguet, C. Schwartz, F. Kretz, and M. Quere RAVI, a

Proposed Standard for the Interchange of Audio/Visual Interactive Applications. IEEE Journal on Selected Areas in

Communications, Vol. 8 No.3, April 1990.

[11] V. P. Rangan, H.M. Vin, and S. Ramanathan Designing an On-Demand Multimedia Service. IEEE Communications

Magazine, Vol. 30 No.7, July 1992, pp. 56-64.

[12] I. B. Stefani, L. Hazard, and F. Hom Computational Model for Distributed Multimedia Applications based on a Synchronous

Programming Language. Computer Communications, Vol. 15

No.2, March 1992, pp. 114-128.

[13] R. Steinmetz, Synchronization Properties in Multimedia Systems. IEEE Journal on Selected Areas in Communications,

Vol. 8 No.3, April 1990, pp. 401-412.

[14] R. Steinmetz, H. Schmutz, B. Schoner, and M. Wasmund Generic Support for Distributed Multimedia Applications.

SUPERCOMM ICC '90 (16-19 April 1990 Atlanta, GA, USA)

IEEE International Conference on Communications ICC '90 Vol. 4, IEEE (New York, NY, USA) 1990, p. 1451-1457.

[15] R. N. Jugele and Dr. V. N. Chavan, Model Designing for

Multimedia and Hypermedia Application. International Journal of Computer Science and Telecommunications, Vol. 3, issue 6, June

2012.pp. 51-54.

[16] R. N. Jugele and V. N. Chavan, Multimedia and Hypermedia Information Object : Object-Oriented MHEG Standard.

International Journal of Engineering Science & Advanced

Technology, Vol. 2, issue 5, Sep-Oct 2012.

[17] N. Cosmos, N., 1990. An architecture for real time

multimedia communication systems. IEEE JSAC 8,pp. 391–400.

[18] L. M. David, 1991. Internet time synchronization: network

time protocol. IEEE Transactions on Communications 39, pp.

1482–1493.

Books :

01. Principles of Multimedia

By. Ranjan Parekh Tata McGraw Hill Companies.

02. Hypertext and Hypermedia.

By. J. Nielsen

Academic Press.



1822

ISSN:2229-6093

synchronization scheme in multimedia and …...synchronization scheme in multimedia and hypermedia...

Documents