Outline
• Grammar-based speech recognition• Statistical language model-based recognition• Speech Synthesis• Dialog Management• Natural Language Processing
© 2013 by Larson Technical Services 1
© 2013 by Larson Technical Services 2
Natural Language Processing—A Moving Target
• Command and control• System-directed dialogs• Continuous speech recognition• “How may I help you” and Statistical Language
Models• Dialogs with automatic error• Mixed-initiative dialogs• Multimodal dialogs• Talking avatar• Conversional dialogs• Intelligent virtual agents—SIRI• Advanced techniques
Dimensions of Natural Interaction
• Interaction style• Semantic Interpretation• Knowledge• Reasoning• Planning• Language complexity• Modalities• Modality synchronization
Interaction Style
• System-directed – Menu or form fill-in
• Mixed initiative– Combination of system-
directed and user-directed
• User-directed– Command and control
• Do something
– Web searches• Find something
– Question and answer systems• Who?, What?, When?,
Where?, How?
© 2013 by Larson Technical Services 4
Semantic Interpretation• Example grammar rule with Script Syntax:
<rule id = "action"> <one-of>
<item> small <tag> out.size = "small"; </tag> </item> <item> medium <tag> out.size = "medium"; </tag> </item> <item> large <tag> out.size = "large"; </tag> </item> </one-of> <one-of> <item> green <tag> out.color = "green"; </tag> </item> <item> blue <tag> out.color = "blue"; </tag> </item> <item> white <tag> out.color = "white"; </tag> </item> </one-of></rule>
• ECMAScript structure:
action: { size: "large" color: "white" }
© 2013 by Larson Technical Services 5
Large white
Knowledge about the Device
• Battery status API– Retrieve information about the battery status of a (mobile) device
(from HTML)– http://www.w3.org/TR/battery-status/
• Geolocation API– Get current geo location (longitude, latitude, altitude) from HTML– Independent from location provider (GPS, WiFi, Cell-Id, ...)– http://www.w3.org/TR/geolocation-API/
• Orientation API– Get current device orientation (e.g., tilt) from HTML– http://www.w3.org/TR/orientation-event/ – iPhone example: “Move the ball“ http://ad.ag/wjmtgt
© 2013 by Larson Technical Services 6
Knowledge about the User
• Biographical– Name, age, gender
• History– Visited websites– Recent purchases– Recent interactions
© 2013 by Larson Technical Services 7
Knowledge about the Domain of Discourse
• Web Ontology Language (OWL)• Resource Description Framework (RDF)
© 2013 by Larson Technical Services 8
Tree
MapleWhite Pine
Evergreen RootsDeciduous Trunk
IsA
PartOf
Reasoning
• Grocery stores sell breakfast cereals
• Corn flakes are a breakfast cereal
• Grocery stores sell corn flakes
© 2013 by Larson Technical Services 9
Planning
• Convert a large task to a series of smaller tasks– Example
origin: PDX; destination: NYC;
origin: PDX; destination: MSP;origin: MSP; destination: NYC;
© 2013 by Larson Technical Services 10
Natural Language Processing
• “Natural Language Processing” means different things to different people
• Applies may artificial intelligence techniques to dialog management
• When it works, it works well, when it fails, it fails badly
• It’s like the wild west: from anarchy and confusion good systems will arise
© 2013 by Larson Technical Services 11