Outline

• Grammar-based speech recognition• Statistical language model-based recognition• Speech Synthesis• Dialog Management• Natural Language Processing

© 2013 by Larson Technical Services 1

© 2013 by Larson Technical Services 2

Natural Language Processing—A Moving Target

• Command and control• System-directed dialogs• Continuous speech recognition• “How may I help you” and Statistical Language

Models• Dialogs with automatic error• Mixed-initiative dialogs• Multimodal dialogs• Talking avatar• Conversional dialogs• Intelligent virtual agents—SIRI• Advanced techniques

Dimensions of Natural Interaction

• Interaction style• Semantic Interpretation• Knowledge• Reasoning• Planning• Language complexity• Modalities• Modality synchronization

Interaction Style

• System-directed – Menu or form fill-in

• Mixed initiative– Combination of system-

directed and user-directed

• User-directed– Command and control

• Do something

– Web searches• Find something

– Question and answer systems• Who?, What?, When?,

Where?, How?

© 2013 by Larson Technical Services 4

Semantic Interpretation• Example grammar rule with Script Syntax:

<rule id = "action"> <one-of>

<item> small <tag> out.size = "small"; </tag> </item> <item> medium <tag> out.size = "medium"; </tag> </item> <item> large <tag> out.size = "large"; </tag> </item> </one-of> <one-of> <item> green <tag> out.color = "green"; </tag> </item> <item> blue <tag> out.color = "blue"; </tag> </item> <item> white <tag> out.color = "white"; </tag> </item> </one-of></rule>

• ECMAScript structure:

action: { size: "large" color: "white" }

© 2013 by Larson Technical Services 5

Large white

Knowledge about the Device

• Battery status API– Retrieve information about the battery status of a (mobile) device

(from HTML)– http://www.w3.org/TR/battery-status/

• Geolocation API– Get current geo location (longitude, latitude, altitude) from HTML– Independent from location provider (GPS, WiFi, Cell-Id, ...)– http://www.w3.org/TR/geolocation-API/

• Orientation API– Get current device orientation (e.g., tilt) from HTML– http://www.w3.org/TR/orientation-event/ – iPhone example: “Move the ball“ http://ad.ag/wjmtgt

© 2013 by Larson Technical Services 6

Knowledge about the User

• Biographical– Name, age, gender

• History– Visited websites– Recent purchases– Recent interactions

© 2013 by Larson Technical Services 7

Knowledge about the Domain of Discourse

• Web Ontology Language (OWL)• Resource Description Framework (RDF)

© 2013 by Larson Technical Services 8

Tree

MapleWhite Pine

Evergreen RootsDeciduous Trunk

IsA

PartOf

Reasoning

• Grocery stores sell breakfast cereals

• Corn flakes are a breakfast cereal

• Grocery stores sell corn flakes

© 2013 by Larson Technical Services 9

Planning

• Convert a large task to a series of smaller tasks– Example

origin: PDX; destination: NYC;

origin: PDX; destination: MSP;origin: MSP; destination: NYC;

© 2013 by Larson Technical Services 10

Natural Language Processing

• “Natural Language Processing” means different things to different people

• Applies may artificial intelligence techniques to dialog management

• When it works, it works well, when it fails, it fails badly

• It’s like the wild west: from anarchy and confusion good systems will arise

© 2013 by Larson Technical Services 11