xisl: content-independent, device-independent mmi description language tsuneo nitta graduate school...
TRANSCRIPT
XISL: Content-Independent, Device-Independent MMI Description Language
Tsuneo Nitta
Graduate School of Tech., Toyohashi Univ. of Tech.
Issues on Interaction with Web
■ The coming of heterogeneous devices
and interaction manner
• access by : PC, Mobile phone, PDA, AutoPC, DTV, …
• interaction with: modality-rich terminals, mixed initiative, ...
Ex. On-line Shopping Devices - order goods at Kiosk ..... touch panel
- confirm the order with a mobile phone ..... voice- change the order at home ..... DTV remote terminal
XML Document Server(Contents ; View ; Interaction)
Our Purpose: Seamless Services over Internet
XML Document Server(Contents ; View ; Interaction)
How to do it?
<xhtml ........
<head>
VoiceXML
</head>
<body>
........
CSSCSS
<xhtml .......
Form
<salt: ......
CSS
XISL: Extensible Interaction-Sheet Language
CSS
<xhtml .......
Form
<xisl>
<dialog>
<exchange>
........
contents-
independent
<xhtml .......
Form
VoiceXML
........
remodels VoiceXML to meet MMI requirements
XISL: Device-Independent - 1/3
no modality-specific tag : input modality
<input type=“speech” event=“recog” target=“grammar.gxml”
match=“/grammar/goods” return=“goods_name”>
<param name=“mode”>data</param>
</input>
<input type=“dtmf” event=“push” target=“grammar.gxml”
match=“/grammar/goods_select”return=“goods_name”>
<param name=“mode”>digit</param>
</input>
...........
XISL: Device-Independent - 2/3
Front-end Module
<input ….. >type: speech
event:recognize recognizegrammar rule:
introductionreturn: word1
<input ….. >
type: pointing
:
ASR
Pointing
Keyboard
Hello!
Hellowor
d1: H
ellogr
amm
ar
type: speech
event: recognize
target: ***.xml
match: /page
return:
word1:Hello
Dialog Manager
XISL: Device-Independent - 3/3
no modality-specific tag : output modality
type event modewindow open, close, …speech play, stop, … TTS, rec_speech, …video play, stop, …audio play, stop, …agentmove, speak, …
MMI Description Language: XISL
- is content ( & view ) -independent.
- has no modality-specific tags that enable XISL device-independent.- has MMI control tags and attributes.
→ • realizes seamless services over internet and reuse of XML documents. • meets modality-rich services and avoids patent issues.
XISL enables combinatorial usage of modalities.
Sequential Parallel
adding a new modality
Alternative
<?xml version=“1.0” encoding=“Shift-JIS”?><!DOCTYPE xisl SYSTEM “xisl.dtd”><xisl version=“1.0”> <head>.........</head> <body> <dialogue id = “WAP_phone”> <exchange> <operation comb = “alt”> <input type=“dtmf” event=“push” target=“/d_gram.gxml” match=“/d_gram/goods_select” return=“number”> <param name=“mode”>digit</param> </input> <input type=“speech” event=“recognize” target=“/gram.gxml” match=“/gram/goods_select” return=“goods_name”> <param name=“mode”>data</param> </input> < /operation >
XISL: example -1/2
< action comb = “seq” > <output type=“speech” event=“play”/> <param name=“speaker”>nancy</param> <param name=“speech_text”> You selected No. One. < /output > < call dialogue_name=“2”/ > < /action > ： ： ：
XISL: example -2/2
Implementation -1/4
XISLXMLXSL
Inp
ut
Mod
alities
Ou
tpu
t M
odalitie
s
Document Server
Interpreter Interpreter
Dialogue
Manager
Dialogue
Manager
Input Integrator
Input Integrator
Response Controller
Response Controller
System Architecture
Dialogue Control
<output>
I/O profile
XML editing
transition
<action>
Op. Control
GLRparser
Input Integration
<input>
user operation
<operation>
Inte
rpre
ter
Start Dialog Dia
log M
anager
Action
Execution
Implementation -2/4 D
ocu
men
t Server
Do
cum
ent S
erver
Fro
nt - en
d
Fro
nt - en
d
Do
cum
ent B
uffer
Dialogue Control by Using GLR TBL
REDUCE-2５REDUCE-1４
SHIFT-5３SHIFT-4２
ACCEPT１SHIFT-2SHIFT-1SHIFT-3０EXCH1BUYGOODScompletetimeoutgoods_spbuy_btn
GOTO TBLACTION TBL INState
０ SHIFT-3
３ SHIFT-5
５ REDUCE-2
Implementation -3/4
①
②
③
Implementation -4/4
XISL Tree View ＸＩＳＬの構造をツリー形式･･･
で表示Ｉｎｔｅｒａｃｔｉｏｎ Ｗｉｎｄｏｗ
生成したＸＩＳＬを表･･･示
Interaction Builder - binding MMI -1/4
Interaction Builder - binding MMI -2/4
<input type="touch" event="click" match="/OLS/goods_list/category/goods[@id='23']/object/picture[1]"/>
Interaction Builder - binding MMI -3/4
<output type="speech" event="agent-speech"><![CDATA[<param name="agent-name">Merlin</param><param name="speech-text">このコートは着心地抜群です</param>]]>
Interaction Builder - binding MMI -4/4
Future Work
- XHTML + XISL
- Implementation of 3GPP applications
- MMI Requirements: more complicated usage of MMI
- Rapid-prototyping: GUI-based developer support system (IPA project)
http://www.vox.tutkie.tut.ac.jp/XISL/XISL-E.html