A Study of Dialogue Management PrinciplesCorresponding to the Driver's Workload

• Mind-free concept (reducing mind distraction) is important in voice telematics systems.

• An experiment concerning a network-distributed voice-activated telematics system was conducted.

• Reference model, basic principles and requirements are also discussed.

• Standardizations of dialogue control, voice command and effect sound are effective.

• A standard could be developed by complementing the dialogue principles of ISO 15005.

T. Nishimoto (Univ. Tokyo), M. Shioya (Hitachi, Ltd.)J. Takahashi, H. Daigo (JARI)

HMI Reference Model for Telematics Systems

To meet ‘ Mind Distraction ’ - mind-free ( mind-on-the-driving )

To meet ‘ Visual Distraction ’ - hands-free ( hands-on-the-wheel ), - eyes-free (eyes-on-the-road )

Background and Objective

Network-Distributed Telematics Systems( Access Open-Specced Information Services )

Autonomous or Dedicated Telematics Car Navigation Systems( Access Closed-Specced Information Services )

- Priority Classes for Information- Prohibit to Provide Info. Not for Driving

Network-Distributed Telematics System to meetSafety & Situation ( Priority, Emergency, Preference )

- Situation-Aware Control- Introduce HMI Classes

Select Voice EngineDepending on Request

Reservation CenterWeb Server

Weather Info. CenterWeb Server

Restaurant Info. CenterWeb Server

Web Server

Voice-Activated Center System

Voice BrowserVoice Browser

Voice EngineVoice Engine

Internet

URL / Voice

Web Server

Web Server

Voice EngineVoice Engine

Voice / Recognized

Data

Text-To-Speech Data as Reply to Request

Network-DistributedVoice-Activated Telematics Service System

In-Vehicle System

Voice

VoiceEngineVoice

Engine

Voice EngineVoice Engine

Voice EngineVoice Engine

VehicleInfo.

LocationInfo.

ASP: Application Service ProviderASP

ASP

Center System

Driving & Environmental Conditions

In-Vehicle System

Request

HMI Server- Distraction Sensing- Dialogue Control considering Info. Priority & Driver’s Skill- Load Share btw Center / In-Vehicle

Driver’s Conditions

Info. Tags Contents

Request

Contents

Voice Telematics HMI System Architecture

SR

TTS

Voice/data

DriverCommunicationsNetworks Vehicle

SR

TTS

Ev

ent-

dri

ven

Mu

ltim

od

al In

terf

ace

(S

AL

T-l

ike

)

HMI Server- Sensing- Control - Share

Vo

IP <

=>

Vo

ice

Vo

IP <

=>

Vo

ice

VoiceXMLDocuments /Interpreter

VoiceXMLDocuments / Interpreter

pausedialogue

pausedialogue

An Example of Dialogue Control for Driver

Passing

Constant Cruise

Right Turn

Left Turn

Mixing

: Driving Load + Telematics Load

: Driving Load

TIME

Load

Driving Ability

Remained for A

Driver A’s SkillDriver B’s Skill

Driving Ability

Remained for B

Telematics Load

Resume Dialogueat Winker OFF

Pause Dialogueat Winker ON

May Resume Dialogueby Command when Signal is Red andVehicle Speed = 0

Dialogue Control in Case of Right Turn

Vehicle

Right Turn

(1) Control of Dialogues: - Most drivers agree that it feels much safe due to the control of dialogues. - Standardization of dialogues is also requested.(2) Pause of Dialogues: - Most drivers agree the necessity of the standardization of interruption and resumption method of dialogues.(3) Voice Commands: - Most drivers agree the necessity of preparing standard commands.(4) Effect Sounds and Announcements: - Most drivers agree the necessity and the effectiveness of the effect sounds and the announcements. - The standardization is also requested.

Evaluations of the Test

- Dialogues between a driver and telematics service systems while a vehicle is in motion.

- Use mainly voices as inputs by the driver, recognized mainly by voice recognition techniques.

- Use mainly voices or sounds as outputs by the systems.

- Treat services not only traffic information, POI information, news, and weather, but also various services, popular in the Internet world ( e.g. e-mail, electronic commerce, information retrieval, and entertainment ).

- The reference model aims to present basic principles, requirements and recommendations, for accessing the telematics services without mind distraction.

View Points of the Reference Model Scope

- In the dialogues for the telematics services, the important basic principle is that the skill for processing the driving load is always secured by the driver.

- That is, the driver should keep his 'mind-on-the-driving' until the driving load processing is completed, namely, should let his 'mind-free' from others but the driving.

- The dialogues for the telematics services should be executed within the driver’s remained skill besides for the driving load processing.

Basic Principle of the Reference Model

- The dialogues for the telematics services are to be managed or controlled so that the skill for processing the driving load is secured first.

- Then, within the driver’s remained skill the telematics load is processed.

- The driving load and the telematics load are to be pre-estimated or measured real-time.

- The driver’s skill or capability is to be pre-estimated or measured real-time.

Basic Requirements of the Reference Model (1)

- The management or control is to be done considering driver’s condition and driving situations ( e.g. road, traffic, vehicle and travel conditions ) of that time.

- The management or control is to be done considering the priorities of the services.

- The telematics service systems' state is to be informed to the driver properly.

- The final initiative of the dialogues is to be taken by the driver.

Basic Requirements of the Reference Model (2)

ISO 14813-1 ITS (TICS) Fundamental Services:32 Services in 8 Categories

ISO 15005 Dialogue Management :Ergonomic Design Principles for Dialogues between Driver and ITS Systems

Considering Driver’s Workload for Safe Vehicle Control

15006In-Vehicle Auditory Presentation:

Specifications for Presentation through Speech or Sounds

to Meet Auditory or Mental Overload

15008In-Vehicle Visual Presentation:

Specifications for Image Quality of Displays for ITS Systems used while Vehicle is in Motion

15007 Measurement of Driver’s Visual Behaviour:Key Terms and Parameters for Analysis

16951 Priority of On-Board Messages:2 Methods for Determining Priority of ITS and Non-ITS Messages

17287 Suitability Assessment:Procedure for Assessing whether Specific ITS Systems are Suitable for Use by Drivers while Driving

Typical HMI Related ISO Standards

Complementing the ISO Standards

A citation from ISO 15005:5.2.4.3.4 RecommendationTICS should provide timely visual information to the driver.

An example of a supplemental recommendation:5.2.4.3.4-1 RecommendationVoice-activated telematics systems will provide the driver with timely audible information.Example 1: In situations where the driving workload increases such as when turning right or left at an intersection, the driver will be advised sufficiently in advance of the maneuver so that the voice-based telematics service dialogue can be performed safely.Figure [Dialogue Control in Case of Right Turn] shows an example of dialogue control during the execution of a right turn, which is one situation involving a heavy driving workload. The dialogue is interrupted and resumed based on the On/Off status of the turn signal lever, respectively. "Interrupt" and "resume" commands are also provided so as to give precedence to the driver's judgment.

- Consensus for - Reference model of voice-activated telematcs systems - Workload measurement methods (Visual + Auditory)- Evaluations of proposing requirements/recommendations - Pause of dialog (driving situation & drivers' skill) - Standardization of voice commands - Standardization of effect sounds / announements

Future Works

Contacts

- Takuya Nishimotonishi@hil.t.u-tokyo.ac.jphttp://hil.t.u-tokyo.ac.jp/~nishi/