ux scotland 2016 - designing voice and natural language experiences

Designing voice and natural language experiences

Dan Whaley

UX Scotland 2016

Why now?

User Acceptance

Core Speech Recognition/NLU

Data – fast mobile connections

Exposure – Siri, Google Now, Cortana

Conversational

Interfaces

User Expectation

Expectation for low effort, intelligent experiences…

Conversation is our most natural communication medium

Well design conversational and speech UIs are the next generation of low effort interface

Intuitive

•User doesn’t need to learn how to interact

Unobtrusive

•Technology or interface doesn’t get in the way

Personalised

•Tailor interaction to the specific user

Anticipatory

•Predicts users next action or intent

Cross Channel

•Switch between channels and time

Platform Explosion

Nina Web/Mobile

Design

Design Principles

Start with the user

Design with data

Get out of the way

Do the hard work to make it simple

Design for all

Understand context

Design for the interface

Deliver solutions, not technology

Be consistent, not uniform

Reuse and refine

Design

Analyse

Start with the user… design with data

Conversation as interface

Natural Language

Temporal

Linear

Repetetive

RedundantAmbiguous

Non-spatial

Transient

Humans are wired for

conversation

Part of evolutionary

development

Instinctively interpret both

words and dialogue acts

“The conscious knowledge that speech can have a non human origin

is not enough to overcome the historically appropriate activation of

social relationships by voice.” Nass and Brave 2005

Understand context

When is voice/nlu the right interface?

• Hands free environment

• Low background noise

• Number of questions to qualify task

• Very large number of options

• Controlled domain

When is voice/nlu the wrong interface?

• Lots of graphical information

• Selecting from a large number of defined options

• Noisy environment

• Difficult to say items (email address)

Understand context

Get out of the way

Conversational UI’s can be the least intrusive interface… truly invisible

Keep it that way

Conversation

Cooperative Principle

Paul Grice (1913-1988)

Philosopher of language

Conversation is built on cooperation towards mutual understanding by conversing parties

Participants must make every effort to listen and respond appropriately and at the right moment

"Make your contribution such as it is required, at the stage at which

it occurs, by the accepted purpose or direction of the talk exchange

in which you are engaged." (Grice, Studies in the Way of

Words1989: 26).

Maxims Of Conversation

Quantity

• Provide the appropriate amount of information

Quality

• Be truthful, do not say what you know to be false

Relevancy

• Make your contribution directly relevant, advance the conversation

Manner

• Choose words and concepts that can be understood by all participants to ensure clarity

Dialogue Entities

• People or systems engaged in conversation

Utterance

• Whatever is spoken or written including inherent semantics

Discourse makers

• Verbal or non verbal markers used to signal or mark a change in the conversation

• Period of time that a particular agent ‘has the floor’ or controls the conversation

Components of Conversation

Agent AgentAction

Action

Context

Signal

Me: “hello” (Action: initiate conversation. State: Waiting)

Friend: “hey Dan” (Action: reply, acknowledge. State: Conversing)

Me: “how are you finding the workshop?” (Action: Question, offer turn.

State: waiting response)

Prompts

Prompts are means to communicate or request information from the user or signal change of state or turn

Providing Information

• Order status, Cab ETA, Balance, etc…

Requesting Information

• Destination, date of birth, credit card number

Confirming Information

• Check understanding. Implicit or explicit.

Offering Choices

• Transfer funds, check account balance, etc…

Signalling an Error Condition

• Re-establish dialogue, get the user back on track

Tips for good prompting…

Speak like a human

Avoid jargon or slang

Start with the most

important information

Don’t explain the

interface. Avoid “speak”

Confirm explicitly only

where necessary

Be consistent, but not

uniform

Avoid mixing recorded

and synthesised audio

Don’t focus on errors

Avoid repeating “sorry”

and “please”

Use contractions, “I’m”

Keep choices simple and

distinct

Lost in time…

Mark where you are

• Implicit confirmation, e.g., “Card Payment…”

Use verbal/non-verbal discourse markers

• Silence to signal turn taking

Indicate any likely delay or latency

• Avoid unexpected silence, “one moment while I…”

Establish islands of safety

• Ok, I’ll take you back to the menu

Allow user to escape

• “start over”

Voice is a temporal interface: its easy to get lost

When it all goes wrong…

No Match

• When the system doesn’t understand the user

No Input

• When the user doesn’t respond to a question

No to confirm

• When the user rejects a confirmed response

Think about what drives each type of error in a

particular context. User is confused, weary,

distracted? Background noise?

Handling error

Take the blame

Give a couple of attempts

Tier prompting

Give examples of how to respond

Islands of safety

Don’t be repetitive

Switch modality

Persona

Function

Congruence

Persona

Coherence

Persona defines the overall

communication style

Diction (familiar vs formal)

Verbosity (concise vs wordy

Tone of voice (warm vs austere)

Characteristics (friendly, aloof,

warm, etc…)

Coherent matching of application function with

coherent persona creates congruent experience

Implementation

How Speech Works

• Phonetics

• /k/a//n//s/el/

• “cancel”

• “can sell”

Acoustic Model

• “cancel my account” {.8}

• “can sell my account” {.2}

Linguistic Model • NLU: Extract

meaning

• Action: ‘Cancel’

• Entity: ‘Account’

Semantic Model

I want to

cancel

account

Text only starts here

Architecture

Application

Controller

Prompts

Recog/

Action:

Entity

- Users response mapped

to action/entity model

- Dialogue crafted around

“Book a ticket to see

Batman tomorrow about

eight”

Book: CinemaTicket

(Film: Batman,

Date: Tomorrow,

Time: 20:00)

Training

Setup: Map example ‘utterances’ to semantic model

“Book a cinema ticket” -> Book:CinemaTicket

“make a reservation to see a film” -> Book:CinemaTicket

“get me a ticket to batman” -> Book:CinemaTicket

Craft dialogue based onOptimisation: Review and update regularly

“Need a pass to see batman” -> ??

“I wanna see a movie” -> ??

Data Sources

Contact Centre

Listen to calls –speech analytics

Web Chat

Web transcripts –

text analytics

Mobile or Web

Forum pages –site analytics

Artificial Intelligence

iStock

AI - Turing Test

AI Today

g ‘G

Specific Intelligence General Intelligence

AI Journey

DesignedIntelligence

Human Assisted

Intelligence

Full MachineIntelligence

Future - AI Driven Customer Service

Artific

ial In

tellig

Future drivers

Connected devices: no screen

• Wearables

• Smart homes

More data: personalisation/anticipation

• Biometrics

• Location

• Consumer preference

Health: cost of care

• Elder care

• Non-physician care

dwhaley@sabio.co.uk

@whaley_d

https://uk.linkedin.com/in/dwhaley-speech-ux

Contact

ux scotland 2016 - designing voice and natural language experiences

Technology

ux scotland 2014 19th june

proto-persona workshop ux scotland 2016

designing wearable experiences ux lisbon 2015

designing better-ux-workshop-5

mobile ux- mobile scotland 2016

designing better-ux-workshop-3

designing better-ux

ux lady-designing-for-error

designing an aesthetic learner experience: ux

ux success - designing for the web

designing for surprise in ux & cx

ux lessons from designing the storyteller

ux fist fighting - designing for usability

designing exceptional ux and cx

mapping the way forward ux scotland 2014

ux scotland - ux patterns case study

designing ux for a content blog

confessions of an ex unicorn - ux scotland 2015

out of context: ux scotland 2013 keynote

designing addicted products @ ux london 13