fname=“yevgen” lname=“borodin” zip=“11790” …
Post on 30-Dec-2015
22 Views
Preview:
DESCRIPTION
TRANSCRIPT
fname=“Yevgen”lname=“Borodin”zip=“11790”…
user=“Yevgen”session=“browse”…
title=“circuit city”mode=“formFill”…
Cached Files
main.vxmlallLinks.vxml history.vxmlfavorites.vxmlkeyList.vxml commands.vxml…
Processor
VariableSpace
Manager
InputManager
Event-HandlingManager
OutputManager
VXML SubmittedVariables
Input Queue
KeyboardInterface
ConsoleSUN Audio
PlayerSpeech
Recognition EngineText-To-Speech
Engine
KeyboardInput
VoiceInput
VoiceOutput
AudioOutput
Output Buffer
JSAPI
SGRSGrammarInterpreter
JavaScriptInterpreter
File Manager
VXMLSurfer in ActionVXMLSurfer in Action
A Flexible VXML (Voice XML) InterpreterYevgen Borodin
Computer Science Department, Stony Brook University
Back-End ProcessingBack-End Processing
HELP
…<rule id="phone_number"> <item repeat="0-1">d</item> <tag><![CDATA[$.phone="ddd-ddd-dddd";]]></tag> </item></rule> …
<?xml version='1.0'?><vxml> … <form id=‘shipping'> <field name="fname"> <prompt>Enter first name</prompt> <grammar src="customer.xml#fname“ type="application/grammar+xml"/> </field> <field name="lname"> <prompt>Enter last name</prompt> <grammar src="customer.xml#lname" type="application/grammar+xml"/> </field> … </form></vxml>
Interface Manager
Browser Object
Context Analyzer
Frame Tree Processor
VXML
Dialog Generator
Frame Tree
HTTPrequest
WEB
HTTPrequest
HTML
Frame Tree
Frame Tree
HTTPrequest
Big Picture: HearSay BrowserBig Picture: HearSay Browser
VXML
Interpreter
Mozilla
Engine
<catch event="Insert+F1 help"> <prompt> You are at Circuit City check out </prompt></catch>
<catch event=“Ctrl+S submit"> <submit namelist=“fname lname …”></catch> …
INS + F1
SGRS GrammarSGRS Grammar
Variables | Files | Return | EventsVariables | Files | Return | Events
fname=“Yevgen”lname=“Borodin”zip=“11790” …
Features of VXMLSurferFeatures of VXMLSurfer
Application of VXMLSurferApplication of VXMLSurfer
Future WorkFuture Work
Inadequacy of screen-readers in Web browsing Development of interface Manager for HearSay voice-browser Absence of fully-implemented open-source VXML interpreters Need for specialization in Web browsing applications Need for a multiplatform, extensible, modular, flexible system
Compliant with VoiceXML 2.0 specifications Geared to Web-browsing as opposed to telephony Modular, Extensible, Multi-Platform (Java) Extends VoiceXML 2.0 to give more control over dialog flow Loaded with add-ons: Spell Check, TTS, SR, etc.
VXMLSurfer is the interface of the HearSay voice-browser Users interact with VXMLSurfer through keyboard and mic HTTP request is forwarded to the Mozilla browser engine The loaded Web page is converted into a frame tree The frame tree is processed and converted to VXML dialogs
Complete VoiceXML 2.0 Specifications Messaging between VXMLSurfer and calling application Speech Recognition (CMU Sphinx) Java Script Interpreter and Grammar Interpreter Modules Multilingual TTS, etc.
VoiceXML dialog files are sent to the interpreter for processing Variables are returned to the calling application (HearSay) HearSay invokes form-filling module to fill and submit the form
MotivationMotivation
Uses of VoiceXMLUses of VoiceXML VXML is typically used in telephony applications Computer games use VXML to program interactive dialogs VXML dialogs disseminate information through public terminals VoiceXML can be used in voice browsing!
This material is based upon work supported by the National Science Foundation - Awards: IIS-0534419, CNS-0751083, IIS-0808678and National Institute on Disability and Rehabilitation Research (NIDRR) - Award: H133S090065.
top related