in-car speech user interfaces and their effects on driver cognitive load

59
In-Car Speech User Interfaces and their Effects on Driver Cognitive Load Andrew Kun

Upload: andrew-kun

Post on 13-Dec-2014

1.496 views

Category:

Technology


1 download

DESCRIPTION

Presentation given at FTW, June 2010.

TRANSCRIPT

Page 1: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Andrew Kun

Page 2: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Students and staff @ UNH

Jon Oppelaar

Oskar PalinkoZeljko Medenica Nemanja Memarovic

Alex Shyrokov Puneet Lakhanpal

Page 3: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Outline

• Ubicomp and cars• Studies:

– Police radio– SUI characteristics– Glove PTT– Navigation

• Conclusion

Page 4: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Navigation

Page 5: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Music, phones

Page 6: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Police cars

• Toys vs. tools• Vanguard of in-car

ubicomp

Page 7: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Project54

Page 8: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Studies

• Goal: – Evaluation tool– Improved HCI

Page 9: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Cognitive load

• Measures:– Performance

• Driving• Visual attention

– Physiological• Pupil diameter• Heart rate• Skin conductance

– Subjective • NASA TLX

Page 10: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Outline

• Ubicomp and cars• Studies:

– Police radio– SUI characteristics– Glove PTT– Navigation

• Conclusion

Page 11: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

DriveSafety driving simulator• tinyurl.com/p54sim

Page 12: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Driving performance

• Variance:– Lane position– Steering wheel angle– Velocity

• Collisions

Page 13: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Seeingmachines eye tracker• tinyurl.com/eyetracker

Page 14: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Visual attention

• Fixations (> 100 ms): number, timing, etc.• Percent dwell time: looking at road

Page 15: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Outline

• Ubicomp and cars• Studies:

– Police radio– SUI characteristics– Glove PTT– Navigation

• Conclusion

Page 16: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Police radio study: SUI vs. hardware

Page 17: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load
Page 18: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load
Page 19: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Outline

• Ubicomp and cars• Studies:

– Police radio– SUI characteristics– Glove PTT– Navigation

• Conclusion

Page 20: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

SUI characteristics study

• Secondary task: speech control of radio• 2 x 2 x 2 design:

– SR accuracy: high/low– PTT button: yes/no – ambient recognition– Dialog repair strategy: mis-/non-understanding

Page 21: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load
Page 22: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Rec. accuracy influences steering

Recognition AccuracyHighLow

Mea

n va

rianc

e of

ste

erin

g w

heel

ang

le

[deg

2]

500

400

300

200

100

0

Error bars: +/- 1 SD

Page 23: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Low acc. + PTT influences lane position

PTTYesNo

Mea

n la

ne p

ositi

on v

aria

nce

[m2]

0.3

0.2

0.1

0.0

Error bars: +/- 1 SD

Page 24: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Outline

• Ubicomp and cars• Studies:

– Police radio– SUI characteristics– Glove PTT– Navigation

• Conclusion

Page 25: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Glove PTT study• Motivation: PTT vs. driving performance• Secondary task: speech control of radio• 2 x 3 x 3 design:

– SR accuracy: high/low– PTT activation: push-hold-release/push-release/no push– PTT button: ambient/fixed/glove

Push-hold-release Push-release No-push

Ambient Fixed Glove Ambient Fixed Glove Ambient Fixed Glove

High

Low

Page 26: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Glove PTT study

Page 27: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Fixed PTT operation

Page 28: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Fixed PTT – glances!

Page 29: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Glove PTT operation

Page 30: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Glove PTT – few glances

Page 31: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Glove PTT transcription

Page 32: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Hand position on wheel [degrees]

Page 33: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Outline

• Ubicomp and cars• Studies:

– Police radio– SUI characteristics– Glove PTT– Navigation

• Conclusion

Page 34: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Navigation study

• Problems:– In-car devices vs.

driving performance and visual attention– Driving performance/visual attention vs.

probability of accident

Page 35: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Navigation study

• With vs. without map:– Time looking at the

road?– Effect on driving

performance?• Voice instruction

for both• 8 male subjects

Page 36: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Driving simulator

Page 37: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

1

2

2

3 3

1

Driving scenario

Page 38: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Visual attention

• Fixations (> 100 ms): number, timing, etc.• Percent dwell time: looking at road

Page 39: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Driving performance

• Variance:– Lane position– Steering wheel angle– Velocity

• Collisions

Page 40: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Effect of visual attention?

• Average variances (13 segments)• Cross-correlation peaks:

PDT decrease → variance increase?

Page 41: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load
Page 42: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Eye tracker camera Rear camera

Camera setup

Front camera

Page 43: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Standard PND

Page 44: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Voice-only PND

Page 45: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Questions

• Influence on visual attention?• If yes, effect on driving performance?

Page 46: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

PDT on the outside world

Series186%

88%

90%

92%

94%

96%

98%

90.4%

96.90%

Standard PND Spoken output only PND

PDT

[%]

Page 47: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

PDT at world vs. distance

60-80 80-100 100-120 120-140 140-16080

85

90

95

100standard spoken only

distance from previous intersection [m]

PDT

on o

utsi

de w

orld

[%]

Page 48: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

PDT at screen vs. distance

60-80 80-100 100-120 120-140 140-160

-5

0

5

10

15

20

distance from previous intersection [m]

PDT

on s

tand

ard

PND

[%]

Page 49: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Questions

• Influence on visual attention?• If yes, effect on driving performance?

Page 50: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Cross-correlation

0 1 2 3 4 5 6 7 80

0.001

0.002

0.003

0.004

0.005

0.006

0.007Lane position

standard

p = 0.05

spoken only

lag [seconds]

Rlp

[met

ers^

2 ]

0 1 2 3 4 5 6 7 80

5

10

15

20

25

30

35Steering wheel angle

standard

p = 0.05

spoken only

lag [seconds]

Rstw

[deg

rees

^2 ]

Page 51: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Cross-correlation, gazes > 200 ms

0 1 2 3 4 5 6 7 80

0.001

0.002

0.003

0.004

0.005

0.006

0.007

Lane position

standard

p = 0.05

spoken only

lag [seconds]

Rlp

[met

ers^

2 ]

0 1 2 3 4 5 6 7 80

5

10

15

20

25

30

35

Steering wheel angle

lag [seconds]

Rstw

[deg

rees

^2 ]

Page 52: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Fixations at standard PND screen

0.1-0.2 0.2-0.3 0.3-0.4 0.4-0.5 0.5-0.6 0.6-0.7 0.7-0.8 0.8-0.90

10

20

30

40

50

60

70

duration [sec]

num

ber o

f fixa

tions

[#]

Page 53: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Navigation study conclusions

• Display: (visual attention)↘

• (visual attention)↘ → (variances↗)

• (long gazes at PND) → (variances↗)• Display viewing: 60% of gazes last >200ms• Collision risk: need testing• But…

Page 54: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

I prefer to have a PND screen…

Strongly agree Agree Undecided Disagree Strongly disagree

0

6

1 1

0

Page 55: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Outline

• Ubicomp and cars• Studies:

– Police radio– SUI characteristics– Glove PTT– Navigation

• Conclusion

Page 56: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Design implications

• SUI may improve performance.• SUI details matter (SR accuracy, PTT).• Must earn users trust.• Complicated displays may reduce visual

attention even more.• Small screen? Voice-only may be just fine.

Page 57: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Next?

• “You’re on the right track.”• Complex environments?• Other possibilities e.g. augmented reality.• Safety?

Page 58: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load

Acknowledgements

• US DOJ (NIJ, COPS)• NSF• Microsoft Research• TellMe

Page 59: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load