text entry nomadicity ambient awareness, handedness, and error adaptation ahmed sabbir arif york...
TRANSCRIPT
Text Entry NomadicityAmbient Awareness, Handedness, and Error Adaptation
Ahmed Sabbir ArifYork University, Toronto, Canada
2
Character-Based Text Entry
o One character at a time:» Non-ambiguous: Qwerty, ...» Ambiguous: Multi-tap, ...
o We also have:» Word-based text entry: handwriting, ...» Phrase-based text entry: predictive, ...
3
Techniques: Timeline17
14 M
echa
nica
l
Tran
scrib
ing
Mac
hine
1830
Sten
otyp
e M
achi
ne18
70s
Qw
erty
1880
sTy
pew
riter
s
1936
Dvor
ak
1970
sPe
rson
al
Com
pute
rs
1990
sM
obile
Key
pads
1993
han
dwriti
ng
Appl
e Ne
wto
n
4
Typewriters & Computers
o Qwertyo Dvorako Qwerty vs. Dvorak:» Path dependence?
5
Chording Keyboards
o Chording keyboards: Twiddler, ...o Chorded keyers: Septambic Keyer, ...o Was never widely accepted
6
12-Key Mobile Keypad
o Multi-tap:» Time-out, kill button
o T9 Text Entry» Predictive
o Other techniques:» TiltText, LetterWise, ...
o Other keypads:» Less-Tap, Reduced
Qwerty, ...
7
Reduced Sized Keyboards
o Mini-Qwerty or thumb keyboardso Virtual or soft keyboards and keypads:» Usually soft versions of physical keyboards» A few use different methods: RollPad, ...
8
Projection Keyboards
o The concept immerged from IBM in 1995o Failed to get its anticipated attentiono Business decision rather than usability issues
9
Touchscreens
o Touchscreen devices are in demand
o Many replace physical keyboards
o Difficult to input text with virtual keyboards:» No synthetic tactile feedback:
vibration, ...o More error prone:
» Error prevention techniques: character replacements, key-target resizing, ...
10
Nomadic Text Entry
o Non-stationary text entry» Walking, driving, or commuting
o Facts:» Slower and more erroneous [Hillman][Lin][Mustonen]» Perfect task-parallelism is not possible [Meyer]
Involves a limited peripheral resource – our eyes Creates competition for the attention between the device
and the ambient environment
from gettyimages.ca
11
Nomadic Text Entry Techniques
o Eyes-free:» Gesture – performs well only when guided by auditory
feedback [Brewster][Lumsden]» Voice – error prone, heavyweight, performance drops
when noisy, not realistic [Mankoff] [Brewster]o Other:» Chorded – takes time to master [Yatani]» Wearable – not convenient, erroneous [Chamberlain]» Synthetic tactile feedback improves touchscreen
performance [Hoggan]
12
AMBIENT AWARENESSNomadic Text Entry
13
Our Approach
o Reduce the competition for focus:» Increase users’ awareness of ambient environment
By providing real-time feedback on their surroundings Users already swap focus regularly between the text entry
area and the keyboard [Arif]
o Real-time, because» Users mostly occupied with instant spatial factors
Human navigation system is a dynamic, egocentric representation [Wang]
14
Four Feedback Techniques
a) Textualb) Visualc) Textual & visuald) Textual & visual
via translucent keyboard
15
Textual Feedback
o In textual or written formo Like turn-by-turn directional information by a
GPS
o We used the WOz method during the experiment» pre-set list containing messages, i.e. go straight,
left turn ahead, etc.
16
(Textual &) Visual Feedback
o Live video using the embedded camera
o Textual feedback:» Translucent (alpha = 0.5) in textual & visual
Visual feedback area is not compromised Background doesn’t obscure text
o Users hold devices in 10–40° angles:» Shows the next few metres of the path» Allows short-term navigation» Highly beneficial
17
Textual & Visual via Translucent KB
o A translucent virtual KB (alpha = 0.35) to show the visual feedback behind the keys» Less focus swap within the interface
o Solid textual feedback area» Background doesn’t obscure text
18
Similar Techniques
o No empirical studyo Video feedback obscures text input and the keyso Input background keeps changing» Causes confusion and irritation [Pilot study]
Road SMS Type n Walk Walk and Text
19
User Study
o Apple iPhone 4» Inputted the presented text phrases [Soukoreff]
o Textual feedback simulated by the Wizard» Sent directly to the iPhone using a web app
o Initial walking speedo Text entry» Stationary and nomadic
20
Obstacle Path
o Mimics realistic walking environments:» Forces users’ attention to the obstacles placed along
the path» Similar to [Barnard]
» Approx. 7.5×6 metres» One lap 24 metres» 13 turns, 3 intersections
21
Design
o 12 participants* 5 techniques* 15 phrases= 900 phrases
+ Initial text entry and walking performanceo Record:» WPM, Total ER, Tfix automatic» Lap time, total laps, wrong turns, bumps manual» Wizard’s mistakes manual
22
Results: WPM, Total ER
o No Significant effect
o Significant effecto Textual, textual &
translucent significantly faster
23
Discussion: WPM, Total ER
o Improved entry speed:» Textual 14%» Visual 8%» Textual & visual, 6%» Textual & translucent 11%
compared to the baseline
o Textual & Textual & translucent significantly faster
24
Results: Walking Speed
o 159% more time to finish a lap while nomadico No significant effect» Considered walking a secondary task
25
Results: Wrong Turns, Collisions
o No Significant effect
o Significant effect
26
User Feedback
o No significant effect» Most felt “neutral”» Wanted to use in challenging scenarios i.e. busy street» Wanted to acquire the textual feedback system
27
Overall Performance
o Textual and textual & visual via translucent keyboard had better overall performance» Improved entry speed by 14% and 11%» Reduced error rates by 13%
» Textual – fastest walking speed (51.10 sec. per lap) Collision count was high (8 in total)
» Translucent – Low collision count (4 in total) Highest lap time (57.11 sec. per lap)
28
HANDEDNESSNomadic Text Entry
29
Research Questions
o Do users input text while nomadic?» [YES] do they use
Both hands, The dominant hand, or The non-dominant hand?
o While nomadic & only one hand is available?» [YES] do they use:
The dominant hand, or The non-dominant hand?
30
Survey Design
o Online – forums, e-mailing lists, ...o Voluntary sampling method – users self-selectedo Screened for:» Adult – 18+» Owns a handheld device» Fluent in English» English is their primary mobile OS language
31
User Demographics
o 133 users after pre-screening» From 20 countries (4 continents)» 46% female» 71% touch-typists» Avg. usage – 4hrs/day» Avg. texts – 26/day» Handedness:
90% right-handed 7% left-handed 3% ambidextrous
32
Devices & Keyboards
o 89% owned a smartphone» All of them use physical or virtual Qwerty keyboard
o 11% use regular mobile devices» Use physical or virtual 12-key keypad
33
While Walking
o 48% input text [almost everyday]» Gender – no significant effect
Male 51% Female 49%
» Age – significant effect 18–25 84.2% 26–35 51.0% 36–45 29.2% 45+ 25.0%
34
While Walking: Handedness
o Mobile handedness» Handedness – no significant effect
Both 54.7% Dominant 36.0% Non-dominant 9.3%
» Gender – no significant effect» Age – no significant effect
35
While Walking: Hand Availability
o 88.7% input text» Gender – no significant effect» Age – significant effect
18-25 years old younger users are more committed
o Mobile handedness» Handedness – no significant effect
78.7% dominant & 21.3% non-dominant
» Gender – significant effect 65% male & 92% female users prefer using dominant hand
» Age – no significant effect
36
While Commuting
o 90% input text [significantly higher]» Gender – no significant effect» Age – no significant effect
o Mobile handedness» Handedness – no significant effect
53.5% both, 46.5% dominant, & 0% non-dominant
» Gender – no significant effect» Age – no significant effect
37
While Commuting: Hand Availability
o 85.8% input text [significantly higher]» Gender – no significant effect» Age – no significant effect
o Mobile handedness» Handedness – no significant effect
85.9% dominant & 14.1% non-dominant
» Gender – significant effect 77% male & 95% female users prefer using dominant hand
» Age – no significant effect
38
While Driving
o 75% drive: 58% input text [more than walking!]» Gender – no significant effect» Age – no significant effect
o Mobile handedness» Handedness – no significant effect
91.7% dominant, & 8.3% non-dominant Dominant hand use [significantly higher than
walking/commuting]
» Gender – no significant effect» Age – no significant effect
39
Key Findings
o A large number of users input text while nomadic» Commuting – significantly more» Walking & driving – similar
96% drivers texts while driving: while doing such is illegal!!
» Age and gender do not influence the decision of texting + commuting or driving
» Age influence the decision of texting + walking» No effect of handedness, age, or gender on mobile
handedness ~50% use both & ~50% the dominant hand
40
Key Findings
o Almost all users continue typing while while the other hand is occupied» [Commute] no effect of age or gender on this choice» [Walk] usually younger users
» Most female users prefer using the dominant hand» Most drivers prefer using the dominant hand
41
Conclusion & Recommendations
o Nomadic techniques must be properly investigated» In order to meet users’ need» Stationary handedness do not apply» Mobile handedness change based on:
Whether walking/commuting or driving Hand availability
• Gender
» Must develop/explore driver-safe techniques
42
ADAPTATIONAdapting to a Faulty Text Entry Technique
43
Research Questions
o Do users adapt to a faulty system?o Do system errors influence adaptation process?o Is there a threshold for identification as error-
prone?o How do users adapt to an error-prone system?
44
Selection of the 7 Letters
o To guarantee uniformity all letters must appear the same number of times
o Humans can remember 7 chunks ±2 in short-term memory tasks
o Require relatively similar effort to draw with Graffiti and Unistrokes
45
Procedure
o Input letters with pen on tablet» Primary method: Graffiti (large above)» Alternate method: Unistrokes (small above)» Suggested usage of alternate if unreliable
o $1 Recognizero Injected 10%, 30%, 50% errorso » Into three out of seven random Graffiti letters
» Different letters for each session
46
Results of the Pilot Studies
o Pilot Study 1I) No reliable switching behaviour for < 10% errors
o Pilot Study 2II) Error-prone letters not reliably identified
Instead: Global switch to alternate methodIII) Extra care if error-prone letters were identified
47
Final Study
o Within-subject, 12 participantsInitial session × 1 block × 280 letters +Final session × 3 blocks × 280 letters= 1120 letters/user, total 13440 letters
48
The Usage of Alternate Method
o Used Unistrokes instead of Graffiti - significant
49
Extra Care while Inputting
o More time than average to draw a letter - significant
50
For Both Measures
o Significant effect for extra care per letter with more time than average to draw a letter error rates
o Significant learning effect
51
Summary
o Users do gradually adapt to a faulty systemo Adaptation is proportional to error rateo Error rate has to be >10% to be perceived as
error-proneo Users learn to avoid frequently occurring errors
faster
52
Thank You!
Questions?