Download - Design and Implementation of a Wireless Handheld Multimodal Digital Video Library Client System
Design and Implementation of a Wireless Handheld Multimodal
Digital Video Library Client System
Sam K. S. Sze Henry K. P. ChoiFeb 27 2002Group Meeting
Abstract
We have developed a client system for accessing a multimodal digital video library (DVL) namely, iVIEW.
It provides a user interface that meets the challenge of rich multimodal information presentation on wireless handheld device. An XML schema is employed to organize the multimodal metadata.
Furthermore, we investigated a context awareness mechanism complementary to the XML schema to facilitate scalable degradation under restricted resources in wireless application environment.
Presentation Overview
Introduction iVIEW Overview – A Practical Perspective iVIEW Wireless Client Scalable Degradation Conclusion
Introduction
Evolving DVL technologies Common challenges for wireless handheld
device application development Challenges for implementing wireless digital
video library client system
Introduction:Evolving DVL Technologies
Multimodal information extraction and organization. The modals includes: speech, on-screen characters, geographical locations, names, faces, on-screen objects …
Internet access Wireless access at anytime and anywhere
Introduction:Common Challenges of Wireless Handheld Devices
Limitations Inherited from Hardware Screen Size Input Device (Pen device) CPU Power Memory Battery Life
Bandwidth Limitations
Introduction:Challenges for Implementing a Wireless Digital Video Library Client
Intelligent User Interface Content Selection Result Set Refinement Integrate Handheld Platform Operation
Manner Resources Management
iVIEW Overview
iVIEW Overview:If Someone Comes to Room 121 and Ask VIEW Technologies ….
I want my video content to be searchable over Internet and wireless tomorrow.
What kind of stuff can you guys provide? ?
iVIEW Overview:What’s your answer? My Answer is …
Demonstration
iVIEW Client Architecture
iVIEW Client: iVIEW Client System Development
The iVIEW Client has been developed in several approaches: As a Microsoft Windows Native Application using Visual
Basic as the frontend with a number of Visual C++ DLLs. (CMU version, Michael Christel) Further extended to support both English and Chinese. (April 2001, Sam)
As a Java Applet for internet access using internal frames (Jacky FYP). Extended to support Real and Quicktime streaming, XML message parsing (August 2001, Sunny, Sam).
As a web page for internet access using Dynamic HTML and ASP. (December 2001, Sam).
Wireless …
iVIEW Client: A History of iVIEW Wireless Client
April 2001 (1 month) Survey - Sam June 2001, (3 weeks) Wireless Web Version – Gordon, Sam August 2001, (2 weeks) Framework of Multi-windows native
application using Embedded Visual C++ - Sam October 2001 (2 weeks) Bluetooth and GSM compliance
tested – Sam, Mike November 2001 (3 weeks) A variant version with HTML
viewer control – Mike December 2001 (1 month) A multilingual search engine
using ASP – Sam, Gordon January 2002 (1 month) A full version of the multi-windows
application - Henry February 2002 (2 weeks) A performance monitor added -
Henry
iVIEW Wireless Client:Implementation
Platform: iPAQ PocketPC
Wireless connections: GSM HSCSD 2.5G CDMA 802.11b Bluetooth
Multi-windows User Interface
3 Phrases of Operation: Query Result Set Manipulation Presentation
iVIEW Wireless Client:Query
Chinese and English Query
iVIEW Wireless Client:Result Set Manipulations
Segment View, Tree View, Topics View
iVIEW Wireless Client:Presentation
<?xml version="1.0" encoding="big5" ?> <sequence path="/iview/video/"> <time start="0">
<script> GOVERNMENT HAS RESTORED FULL </script> <frame file="frame141_00.jpg" /> </time> <time start="2"> <script> DIPLOMATIC RELATIONS WITH LIBYA </script> </time> <time start="4"> <script> AFTER A 15 YEAR SUSPENSION. CNN'S </script> <frame file="frame141_01.jpg" /> </time> <time start="6"> <script> MARGARET LOWRIE HAS THE DETAILS FROM LONDON </script> <map>LONDON</map> </time> <time start="8"> <script> BRITISH FOREIGN SECRETARY ROBIN COOK </script> </time> <time start="11"> <script> ANNOUNCED DIPLOMATIC TIES WITH LIBYA WOULD BE RESTORED BECAUSE IT </script> </time> <time start="14"> <script>IT NOW ACCEPTS RESPONSIBILITY </script> <frame file="frame141_02.jpg" /> </time> . . . . . </sequence>
iVIEW Wireless Client:Gracefully Degradation at Multimodal Presentation
iVIEW Wireless Client:Gracefully Degradation at Multimodal Presentation
iVIEW Wireless Client:Gracefully Degradation at Multimodal Presentation
iVIEW Wireless Client:Embedded Resources Monitor
Resources CPU Usage Memory Utilization Bandwidth Consumption
iVIEW Wireless Client:Gracefully Degradation at Multimodal Presentation
A set modal significance order chain
(i.e. S(m1) > S(m2) > S(m3)> …)
should also be defined for different video types.
If system utilization saturates, the dispatch will be signaled to make a decision to stop dispatching a set of active modal of least significance. Mathematically,
iVIEW Wireless Client:Measurement – A Typical Client Operation Profile
iVIEW Wireless Client:Measurement – Degrade Transition
Conclusion
A wireless DVL client system, that meets the challenges of handling complex multimodal presentations in a handheld device, had been developed. We had shown a user interface design that allow flexible content selection, result set refinement in different views that fits the pen computing operation profile, and it also equips the capability of high complexity multimodal presentation.
We define a multimodal presentation XML with its corresponding context aware scalable degrade mechanism that optimizes the presentation content under resources limited and fluctuating environment.
Q & A