The HIGGINS domain

• The primary domain of HIGGINS is city navigation for pedestrians. • Secondarily, HIGGINS is intended to provide simple information

about the immediate surroundings.

This is a 3D test environment

HIGGINS research issues

• Which factors should govern the dialogue strategy regarding feedback and information detail?– user experience, success, errors, initiative...

• How do different levels of feedback and detail in system responses affect robustness and efficiency of the dialogue?

• How to find and categorise errors– e.g. confidence measures in different system modules

• How should the system act when it doesn't or only partially understands– clarification or "wait and see"

• How does the user's beliefs about the system’s understanding correlate to its actual understanding, depending on error handling strategy?

Error handling

• what should the system do when it understands nothing or just a fragment of the user’s utterance? (non-understanding)

• traditional method– signal non-understanding– encourage the user to repeat

• “I didn’t understand”• “I didn’t hear what you said”

– leads to• hyperarticulation• the user just repeats

Human handling of non-understanding

• Experiment with humans

• Task: navigate in a simulated campus

Vocoder

User Operator

Listens Speaks

ReadsSpeaks ASR

Results• Humans do not often signal non-

understanding (30% of the time).• Instead, they ask questions such as “can you

see a red building?”, confirming a hypothesis about the situation but not about what has been said.

• This leads to faster– error recovery– greater experience of task success

Skantze, "Exploring human error handling strategies: implications for spoken dialogue systems", 2003