People & Speech Interfaces

CS 260Wednesday, October 4, 2006

People & Speech

• Born to be wild and understand speech

• Speech as an instrument of cooperative action

• What literacy has to do with speech

A video

Wired for Speech (Nass & Brave)

• “Humans are automatic experts at extracting the social aspects of speech.”

• Our brains associate voice with social relationships: so what happens when we communicate with technology through speech?

Our brains can’t tell the difference

• Gender stereotyping: the man (woman?) behind the curtain– pitch, pitch range

• Personality: opposites (don’t) attract– volume, pitch, pitch range, speed rate

Prosody vs. PragmaticsProsody vs. Pragmatics

Hello?Hey.Hey!Hey, what’s up?Not much. How are you?Okay!So…what’s going on?

“Our talk exchanges do not normally consist of a succession of disconnected remarks, and would not be rational if

they did. They are characteristically…cooperative efforts.”

~ H. Paul Grice

Conversational Maxims• Maxims of Quantity

– Make your contribution as informative as necessary.– Do not make your contribution more informative than is

necessary.

• Maxims of Quality– Do not say what you believe to be false.– Do not say that for which you lack adequate evidence.

• Maxims of Relevance– Be relevant.

• Maxims of Manner– Avoid obscurity.– Avoid ambiguity.– Be brief.– Be orderly.

How we use language

• The maxims describe how speech should be produced– Cooperative users

• But speech is often subordinate to some kind of action

• A sentence serves a syntactic function• A dialogue act serves a pragmatic function

Sentence Types

• Constituent interrogatives• Yes/no interrogatives• Imperatives• Assertions

• Each of these sentence types serve a different syntactic function

““Would you please pass the Would you please pass the salt?”salt?”

Dialogue Acts• Task Management Acts

– Constitutive acts• Expressives (complimenting someone)• Declaratives (sentencing someone to prison)

– Informative acts• Assertives (stating a fact)• Interrogatives (asking for information)

– Obligative acts• Directives (requests)• System directives (calling the help system)• Commissives (offering something)

• Dialogue Management Acts– Flow-regulating acts (beginning an exchange)– Grounds-keeping acts (clarifying a point)

Task /Dialogue Management Example

System: How may I help you?DM: exchange initiator TM: offer

Caller: I was trying to place a call and must have dialed the wrong numberDM: acknowledgment TM: acceptance, assertioncan I get credit for that?DM: turn assignment TM: request

System: Do you need me to give you credit?DM: acknowledgment, turn assignment TM: offer

Caller: Yes.DM: confirmation, turn release TM: acceptance

“The situation in which words are uttered can never be

passed over as irrelevant to the linguistic expression.”

~Bronislaw Malinowski

Prosody vs. PragmaticsProsody vs. Pragmatics

““Illiterate”Illiterate”

Illiterate people also…

• Live in poverty• Have not had formal schooling• Are marginalized• Feel inferior to educated people• May not have regular jobs

• Live in literate societies

Cognition & Literacy

#1All Kpelle men are rice farmers. Mr. Smith is not a rice farmer.Is he a Kpelle man?

Illiterate Subject: I don’t know the man in person. I have not laid eyes on the man myself…If you know a person, if a question comes up about him you are able to answer. But if you do not know the person, if a question comes up about him, it’s hard for you to answer it.

#2In the far north all bears are

white;Novaya Zemyla is in the far

north. What color are the bears there?

Illiterate Subject: You should ask the people who have been there and seen them…We always speak of only what we see; we don’t talk about what we haven’t seen.

Experiential, empirical, situational knowledge

Translating an interface into illiterate

(1) Replace all text with icons(2) Done!

It’s not that simple…

• Associating an identifier (icon, number) with a concept is a literate task

• Reluctance to learning how to do arbitrary tasks

2 apples

1. Oranges

2. Apples

3. Bananas

Please enter your selection:

Translating into illiterate, take two

(1) Turn it into a speech-based telephone application.

(2) Done!

Issues with speech only interfaces

• Can’t support browsing• Difficult to represent spatial and temporal

data• Requires user to remember a lot of

information• Forced through a deep decision tree

• Memorize command for abstract tasks

Our own experience

• Environmental factors– Reluctance– Attrition

• Memorization of commands– Clarifications– Dialect

An ideal interface for illiterate users