in-car speech user interfaces and their effects on driver cognitive load
DESCRIPTION
Presentation given at FTW, June 2010.TRANSCRIPT
In-Car Speech User Interfaces and their Effects on Driver Cognitive Load
Andrew Kun
Students and staff @ UNH
Jon Oppelaar
Oskar PalinkoZeljko Medenica Nemanja Memarovic
Alex Shyrokov Puneet Lakhanpal
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
Navigation
Music, phones
Police cars
• Toys vs. tools• Vanguard of in-car
ubicomp
Project54
Studies
• Goal: – Evaluation tool– Improved HCI
Cognitive load
• Measures:– Performance
• Driving• Visual attention
– Physiological• Pupil diameter• Heart rate• Skin conductance
– Subjective • NASA TLX
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
DriveSafety driving simulator• tinyurl.com/p54sim
Driving performance
• Variance:– Lane position– Steering wheel angle– Velocity
• Collisions
Seeingmachines eye tracker• tinyurl.com/eyetracker
Visual attention
• Fixations (> 100 ms): number, timing, etc.• Percent dwell time: looking at road
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
Police radio study: SUI vs. hardware
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
SUI characteristics study
• Secondary task: speech control of radio• 2 x 2 x 2 design:
– SR accuracy: high/low– PTT button: yes/no – ambient recognition– Dialog repair strategy: mis-/non-understanding
Rec. accuracy influences steering
Recognition AccuracyHighLow
Mea
n va
rianc
e of
ste
erin
g w
heel
ang
le
[deg
2]
500
400
300
200
100
0
Error bars: +/- 1 SD
Low acc. + PTT influences lane position
PTTYesNo
Mea
n la
ne p
ositi
on v
aria
nce
[m2]
0.3
0.2
0.1
0.0
Error bars: +/- 1 SD
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
Glove PTT study• Motivation: PTT vs. driving performance• Secondary task: speech control of radio• 2 x 3 x 3 design:
– SR accuracy: high/low– PTT activation: push-hold-release/push-release/no push– PTT button: ambient/fixed/glove
Push-hold-release Push-release No-push
Ambient Fixed Glove Ambient Fixed Glove Ambient Fixed Glove
High
Low
Glove PTT study
Fixed PTT operation
Fixed PTT – glances!
Glove PTT operation
Glove PTT – few glances
Glove PTT transcription
Hand position on wheel [degrees]
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
Navigation study
• Problems:– In-car devices vs.
driving performance and visual attention– Driving performance/visual attention vs.
probability of accident
Navigation study
• With vs. without map:– Time looking at the
road?– Effect on driving
performance?• Voice instruction
for both• 8 male subjects
Driving simulator
1
2
2
3 3
1
Driving scenario
Visual attention
• Fixations (> 100 ms): number, timing, etc.• Percent dwell time: looking at road
Driving performance
• Variance:– Lane position– Steering wheel angle– Velocity
• Collisions
Effect of visual attention?
• Average variances (13 segments)• Cross-correlation peaks:
PDT decrease → variance increase?
Eye tracker camera Rear camera
Camera setup
Front camera
Standard PND
Voice-only PND
Questions
• Influence on visual attention?• If yes, effect on driving performance?
PDT on the outside world
Series186%
88%
90%
92%
94%
96%
98%
90.4%
96.90%
Standard PND Spoken output only PND
PDT
[%]
PDT at world vs. distance
60-80 80-100 100-120 120-140 140-16080
85
90
95
100standard spoken only
distance from previous intersection [m]
PDT
on o
utsi
de w
orld
[%]
PDT at screen vs. distance
60-80 80-100 100-120 120-140 140-160
-5
0
5
10
15
20
distance from previous intersection [m]
PDT
on s
tand
ard
PND
[%]
Questions
• Influence on visual attention?• If yes, effect on driving performance?
Cross-correlation
0 1 2 3 4 5 6 7 80
0.001
0.002
0.003
0.004
0.005
0.006
0.007Lane position
standard
p = 0.05
spoken only
lag [seconds]
Rlp
[met
ers^
2 ]
0 1 2 3 4 5 6 7 80
5
10
15
20
25
30
35Steering wheel angle
standard
p = 0.05
spoken only
lag [seconds]
Rstw
[deg
rees
^2 ]
Cross-correlation, gazes > 200 ms
0 1 2 3 4 5 6 7 80
0.001
0.002
0.003
0.004
0.005
0.006
0.007
Lane position
standard
p = 0.05
spoken only
lag [seconds]
Rlp
[met
ers^
2 ]
0 1 2 3 4 5 6 7 80
5
10
15
20
25
30
35
Steering wheel angle
lag [seconds]
Rstw
[deg
rees
^2 ]
Fixations at standard PND screen
0.1-0.2 0.2-0.3 0.3-0.4 0.4-0.5 0.5-0.6 0.6-0.7 0.7-0.8 0.8-0.90
10
20
30
40
50
60
70
duration [sec]
num
ber o
f fixa
tions
[#]
Navigation study conclusions
• Display: (visual attention)↘
• (visual attention)↘ → (variances↗)
• (long gazes at PND) → (variances↗)• Display viewing: 60% of gazes last >200ms• Collision risk: need testing• But…
I prefer to have a PND screen…
Strongly agree Agree Undecided Disagree Strongly disagree
0
6
1 1
0
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
Design implications
• SUI may improve performance.• SUI details matter (SR accuracy, PTT).• Must earn users trust.• Complicated displays may reduce visual
attention even more.• Small screen? Voice-only may be just fine.
Next?
• “You’re on the right track.”• Complex environments?• Other possibilities e.g. augmented reality.• Safety?
Acknowledgements
• US DOJ (NIJ, COPS)• NSF• Microsoft Research• TellMe