using voicexml, xhtml, and scxml to build multimodal applications james a. larson
TRANSCRIPT
Using VoiceXML, XHTML, and SCXML to Build Multimodal ApplicationsJames A. Larson
• SCXML controls the flow of an application
• SCXML controls modalities– VoiceXML– XHTML– Others, e.g., InkML, SVG
• SCXML controls Web services
Agenda
State Chart XML
• Represents a state transition system– States– Transitions
A
B
Possible Actions within a <state>
<if …> <start … /><else/> <log …/></if>
A
• Multimodal life cycle events<start … /><cancel … /><pause … /><resume … /><data … />
• Executable content<assign/><var><script/><if/>, <elseif/>, <else/><log/>
SCXML• Sequentially perform the actions in two
states
<prepare … />
<start … />
done
A
B
<state id="A">
<prepare … />
<transition event= "done" target="B"/>
</state>
<state id="B">
<start … />
</state>
SCXML
<state id="A">
<prepare … />
<transition event= "done"
cond="result='OK̍'" target="B"/>
<transition event= "done"
cond="result='Opps'" target="C"/>
</state>
<prepare … />
doneOK
doneOpps
A
B C
• Conditional branch
SCXML
<state id="A">
<transition …. target="B"/>
</state>
<state id="B"/>
<parallel>
<state id="C"/>
<state id="D"/>
</parallel>
</state>
A
C D
B
• Parallel States
• SCXML controls the flow of an application
• SCXML controls modalities– VoiceXML– XHTML– Others, e.g., InkML, SVG
• SCXML controls Web services
Agenda
SCXML + VoiceXML
<state id="A"> <start name= "getAirport.vxml" /> <transition event= "done" target="B"/></state>
state id="B"> <start … /></state>
done
A
B
<?xml version="1.0"?> <vxml version="3.0" xmlns =
"http://www.w3.org/2001/vxml"><form id = "destination" > <field name = "Airport"> <grammar src= "airport.grxml"/> <prompt>Say your destination airport </prompt> <grammar src= “Airport.grxml” </field> <filled> <send event= "done" namelist = "Airport" /> </filled> </form></vxml>
getAirport.vxml
SCXML + XHTML
<state id="A"> <start name= "getAirport.xhtml" /> <transition event= "done" target="B"/></state>
state id="B"> <start … /></state>
done
A
B
<html/>
<head>
<ev:listener ev:event="onchange" ev:observer="bodyId" ev:handler= "sendDone()" />
<script> {function sendDone…} </script>
</head>
<!--xhtml code -->
<body id="bodyId"/><p>enter month</p>
<input type="text" id="month"/>
</body>
</html>
getAirport.xhtml
SCXML + XHTML + VoiceXML
state id="B"> <start … /></state>
done
B
<state id="A1"> <start name= "getAirport.vxml" /> <transition event= "done" target="B"/></state>
<state id="A2"> <start name= "getAirport.xhtml" /> <transition event= "done" target="B"/></state>
A
A1 A2
done
B
SCXML + InkML + VoiceXML
state id="B"> <start … /></state>
B
<state id="A1"> <start name= "getStuff.vxml" /> <transition event= "done" target="B"/></state>
<state id="A2"> <start name= "getStuff.Inkml" /> <transition event= "done" target="B"/></state>
A
A1 A2
done
B
Another example of multiple modalities
SCXML + InkML + VoiceXML
state id="B"> <start … /></state>
B
<state id="A1"> <start name= "getStuff.vxml" /> <transition event= "done" target="B"/></state>
<state id="A2"> <start name= "getStuff.Inkml" /> <transition event= "done" target="B"/></state>
A
A1 A2
done
B
<interpretation mode = "speech"> <travel> <to hook="ink"/> <from hook="ink"/> <day> Tuesday </day> </travel></interpretation>
SCXML + InkML + VoiceXML
state id="B"> <start … /></state>
B
<state id="A1"> <start name= "getStuff.vxml" /> <transition event= "done" target="B"/></state>
<state id="A2"> <start name= "getStuff.inkml" /> <transition event= "done" target="B"/></state>
A
A1 A2
done
B
<interpretation mode = "speech"> <travel> <to hook="ink"/> <from hook="ink"/> <day> Tuesday </day> </travel></interpretation>
<interpretation mode = "ink"> <travel> <to>Las Vegas </to> <from>Portland </from> </travel></interpretation>
SCXML + InkML + VXMLB
<state id="A1"> <start name= "getStuff.vxml" /> <transition event= "done" target="B"/></state>
<state id="A2"> <start name= “getStuff.inkml" /> <transition event= "done" target="B"/></state>
A
A1 A2
done
B
<interpretation mode = "speech"> <travel> <to hook="ink"/> <from hook="ink"/> <day> Tuesday </day> </travel></interpretation>
<interpretation mode = "ink"> <travel> <to>Las Vegas </to> <from>Portland </from> </travel></interpretation>
<interpretation mode = "interp1"> <travel> <to> Las Vegas </to> <from> Portland </from> <day> Tuesday </day> </travel></interpretation>
state id="B"> <start … /></state>
• SCXML controls the flow of an application
• SCXML controls modalities– VoiceXML– XHTML– Others, e.g., InkML, SVG
• SCXML controls Web services
Agenda
SCXML Control More Than Just Modalities
• SQL
• Complex calculations
• Web services
<start … />
<start … />
<start … />
<start …/>
vxml
sql
complexCalculation
xhtml
References
• State Chart XML (SCXML): State Machine Notation for Control Abstraction– http://www.w3.org/TR/scxml/
• Multimodal Architecture and Interfaces– http://www.w3.org/TR/mmi-arch/
• Commons SCXML (Open Source)– http://jakarta.apache.org/commons/scxml/
Summary
• SCXML controls the flow of an application
• SCXML controls modalities– XHTML– VoiceXML– Others, e.g., inkML, SVG
• SCXML controls Web services
