of 50 e gov universal access ahmed gomaa cimic rutgers university

Of 50 Of 50

EGOV Universal Access

Ahmed GomaaCIMICRutgers University

2Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

3Of 50 Of 50

Universal Access

Any MM Information Over Any Network

To Any One with varying expertise, capabilities, and preferences

To Any Device

4Of 50 Of 50

EGOV Universal Access Example

The Entrepreneur can view on the EGOV portal a set of information resources specific to his type of business on any type of internet appliance with customizable preferences based on his appliance and his own preferences and capabilities.

5Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

6Of 50 Of 50

A medical Digital Library Object

• The video must start immediately after the image has been displayed .

• The text must be displayed simultaneously with the image and the video.

[Adam et al,2001]

7Of 50 Of 50

• The digital library object is downloaded to the client machine.

8Of 50 Of 50

Implementation

9Of 50 Of 50

Implementation

10Of 50 Of 50

Object Plan as MOPN

Adam et al, 2001

11Of 50 Of 50

DL Object Plan for the above medical object

Adam et al, 2001

12Of 50 Of 50

The modified plan (with only audio and text capabilities)

Adam et al, 2001

13Of 50 Of 50

Parallel and Sequential

Adam et al, 2001

Adam et al, 2001

14Of 50 Of 50

An Adjusted plan

Adam et al, 2001

15Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

16Of 50 Of 50

Issues and Problems• Issue:

– Synchronization and User interactivity .

• Problems: *In the user interaction cases:Forward / Rewind / Pause.– How to make sure that the synchronization constraints are still

valid?– How to visualize and analyze the effect of the user interaction on

the temporal constraints?

17Of 50 Of 50

Issues and Problems

• Issue:– Automatic detection of desynchronization and minimize

the desynchronization periods automatically

• Problems:– Some of the multimedia object playbackperiod is determined on the runtime (Video). So even if there are constraints on the presentation,

desynchronization will appear. How can we automatically detect that desynchronization and put it to a minimum?

18Of 50 Of 50

Issues and Problems

• Issue:– We need to make the MM information accessible based on

user preferences, expertise and capabilities.

• Problems:– The user might need to convert some MM formats to

others based on his capabilities ( text to audio)– Personalize the Multimedia presentations based on user

preferences, expertise and capabilities. The broker server issue arise for converting to different formats.

– How to select the route for converting the MM presentation?

– What will be the cost function?

19Of 50 Of 50

Presentation outline

• Universal access definition and example• Issues and problems• Methodology• Previous work• Extensions proposed• SMIL and timing• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Conclusions

20Of 50 Of 50

Extended EGOV Universal Access

PETRI-NET Modeling for validation

and analysis

•Temporal Synchronization and User

interactivity

•Automatic detection of desynchronization

•Minimize the desynchronization periods

•Personalization and accessibility

•Temporal Synchronization and User

interactivity

•Automatic detection of desynchronization

•Minimize the desynchronization periods

•Personalization and accessibility

Implementation with

user/web friendly with flexibility

and interoperability (SMIL 2.0)•Temporal Synchronization with User

interactivity

•Automatic detection of desynchronization

•Minimize the desynchronization periods

•Personalization and accessibility

•Temporal Synchronization with User

interactivity

•Automatic detection of desynchronization

•Minimize the desynchronization periods

•Personalization and accessibility

21Of 50 Of 50

Why Use Petri-Nets?

• The Above mentioned problems can be implemented by hard coded solution. ( SMIL 2.0)

BUT• How can we identify if there is a temporal or spatial

conflict in case of normal playback or even in Forward or Rewind? How can we visualize the runtime of our presentation to prevent any deadlocks, resource constraints, or network constraints?

SO• We need a model to analyze and abstract the

problems mentioned. Petri- Net has been used in the academic field as well as the industrial field to model different systems.

22Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

23Of 50 Of 50

Server

ObletAdjusted

Object Plan

N/Wdelay

Synchronization:Fine and coarse-grain

Temporal Navigation

( User Interaction)

Client

Temporal Dynamic

presentation

ExtensionsInterpretable & easy to maintain

ObjectPlan(SMIL)

Automatic adjustment

of Desync.

AccessibilityCost

Function

24Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

25Of 50 Of 50

Why SMIL 2.01. XML Based.

2. Allows you to use clips in different locations . (Separate URL for each clip)

3. Time and control a presentation. (Different approaches for timing)

4. Lay out a presentation. When your presentation includes multiple clips, such as a Real Video clip playing simultaneously with text captions or banner graphics, you use SMIL to define the layout.

5. Content Control module provides alternate presentations ( multiple languages, or different bandwidths. )

26Of 50 Of 50

SMIL2.0 To Petri Net

Equals ( SYNC):<par dur="30s"> <img id="foo"

src="a.jpg"/> <text  src="text.html" />   <audio src="audio.au" /></par> |------------| image |------------| text |------------| Audio 30s

After ( Before):<seq> <img src="a.gif"

dur="6s" /> <img src="b.gif" dur="4s"

begin="1s" /> </seq>|----------| 6 sec |---------|

4sec|--------------------------|

11 seconds

27Of 50 Of 50

Temporal relations and corresponding OCPN

[Little et al, 1990]

28Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

29Of 50 Of 50

Data Retrieving Engine

[Yang, 2001]

30Of 50 Of 50

RTSM

[Yang, 2001]

31Of 50 Of 50 Of 50 Of 50

[Yang, 2001]

32Of 50 Of 50

Fast Forward operation

Of 50 Of 50 [Yang, 2001]

33Of 50 Of 50

Pre- Fetch table

Retrieval time =

(Playback period *play rate) / Estimated BW + RDdelay

Of 50 Of 50

[Yang, 2001]

34Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

35Of 50 Of 50

Model Global time

36Of 50 Of 50

User Interaction Modeling

37Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

38Of 50 Of 50

Absolute time line

• The main deficiency in the absolute time line is the indeterminism of continuous objects.

• It might take more or less than 10 seconds for the audio file to play. – How to insure synchronization? Indeterminism problem arise.

Of 50 Of 50

39Of 50 Of 50

Indeterminism and absolute time line

Of 50 Of 50 [Layaïda et al, 2002]

40Of 50 Of 50

Indeterminism Modeling

Of 50 Of 50

41Of 50 Of 50

42Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

43Of 50 Of 50

Accessibility• How to render a different

type of media to the client? For example, a blind

person do not need to download text files, instead he needs to convert it to audio format.

Three cases:• Converters on the Server.• Converters on the Client.• Converters with middleman

server

Cost

function?

Cost

function?

44Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

45Of 50 Of 50

Future Work• Issue:

– Incorporate Temporal and spatial constraints

• Problems: – What constraints shall be added to insure presentation integrity? ( Temporal and Spatial constraints )– How to synchronize between different components in a spatio-temporal fashion?

46Of 50 Of 50

Future Work

• Issue:– We need to make the different networks transparent to the

user, and making the user feel as if he is watching a local presentation. Which we call Fine and Coarse grain network delay manipulation.

• Problems:– How to estimate the bandwidth? (Known at runtime)– How to make the network delay seamless between

different networks (wireless – mobiles- cables …) can we develop a generic model that automatically adapt for changes between different networks?

47Of 50 Of 50

Future Work

• Issue:– Adaptation of Multimedia Presentations on different

hardware and software. ( OS / MM Boards / Devices ).

• Problems:– Resource constraints and device diversity. How can we

manage that the user device will be able to handle the Incoming presentation?

– How are we going to deal with the client resources? Buffer problem? Different board latency time?

48Of 50 Of 50

Server

ObletAdjusted

Object Plan

Multimedia Boards

Audio/Video

N/Wdelay

Client

Different Devices

Future WorkInterpretable & easy to maintain

ObjectPlan(SMIL)

Automatic adjustment

of Desync.

ResourceConstraints

N/WConstraints

Access Control for Timesheets ServerServer

Synchronization:Fine and coarse-grain

Temporal Navigation +

Spatial Constraints

Spatial&Temporal Dynamic

presentation

AccessibilityCost

Function

49Of 50 Of 50

Presentation outline• Universal access definition and example• Previous work• Issues and problems• Methodology• Extensions• SMIL• SMIL engine• PETRI-NET model for user interaction• PETRI-NET model for automatic detection and

minimization of desynchronization periods• Accessibility• Future Work• Conclusions

50Of 50 Of 50

Conclusion

• Converting SMIL 2.0 to Petri-net for interoperability purposes.

• Timesheets use in SMIL2.0.• User input into the Petri net model.• Forward / Rewind / Pause in a PN model.• Presenting Indeterminism and how to minimize

desynchronization for uncontrollable objects in a PN model.

• Presenting both Spatial and temporal constraints in one Petri net model ( In progress)

• Accessibility and cost function.

Of 50 Of 50