i. scott mackenzie york university · 2009-11-18 · 1 1 york university – department of computer...
Post on 20-Mar-2020
0 Views
Preview:
TRANSCRIPT
1
1
York University – Department of Computer Science and Engineering
ISO9241 – Part 9
I. Scott MacKenzieYork University
2
York University – Department of Computer Science and Engineering
What is ISO 9241-9
• ISO 9241• “Ergonomic design of visual display terminals
(VDTs) used for office work”
• Seventeen parts• Part 9: “Requirements for non-keyboard input
devices”• Part 9 - Annex B – “Performance testing”
2
3
York University – Department of Computer Science and Engineering
Annex B – Performance Tests
• Task #1 - One-directional tapping test• Task #2 - Multi-directional tapping test• Task #3 - Dragging test• Task #4 - Path following test• Task #5 - Tracing test• Task #6 - Free-hand test• Task #7 - Grasp and park (homing) test
4
York University – Department of Computer Science and Engineering
Task #1One-directional Tapping Test
3
5
York University – Department of Computer Science and Engineering
Task #2Multi-Directional Tapping Test
6
York University – Department of Computer Science and Engineering
Task #3Dragging Test
• Variation on• One-directional tapping test• Multi-directional tapping test
• An object is “dragged” and dropped in the destination target
• See MacKenzie, Sellen, & Buxton paper from CHI 1991
4
7
York University – Department of Computer Science and Engineering
Task #4Path Following Test
8
York University – Department of Computer Science and Engineering
Task #5Tracing Test
5
9
York University – Department of Computer Science and Engineering
Task #6Free-hand Input Test
etc.
10
York University – Department of Computer Science and Engineering
Task #7Grasp and Park Test
• “The subject performs a simple pointing task and operates a key on the keyboard between each pointing task with the same hand.”
6
11
York University – Department of Computer Science and Engineering
Device Assessment Questionnaire
• ISO 9241-9 also includes a comprehensive qualitative evaluation
• Uses 13 questions and a 5-point Likert response scale
• Assesses aspects of operation, fatigue, comfort, and overall usability
Next slide
12
York University – Department of Computer Science and Engineering
7
13
York University – Department of Computer Science and Engineering
14
York University – Department of Computer Science and Engineering
8
15
York University – Department of Computer Science and Engineering
ISO Metric for Performance Testing
Throughput
16
York University – Department of Computer Science and Engineering
Throughput (1)
• Combines speed and accuracy in a single measure
• Based on Fitts’ law (Fitts, 1954)• Endorsed by ISO as the metric for evaluating
pointing devices• Calculations and procedures are tricky
9
17
York University – Department of Computer Science and Engineering
Throughput (2)
• Benefits are substantial• Emerging use in industry• Aka “index of performance” or “bandwidth”• We’ll explain shortly, but first…
18
York University – Department of Computer Science and Engineering
Fitts’ Index of Difficulty (1)
10
19
York University – Department of Computer Science and Engineering
Fitts’ Index of Difficulty (2)
20
York University – Department of Computer Science and Engineering
ID Examples
11
21
York University – Department of Computer Science and Engineering
ID vs. Movement Time
22
York University – Department of Computer Science and Engineering
Throughput
12
23
York University – Department of Computer Science and Engineering
Some Data
Each score is the mean over repeated trials and multiple participants.
24
York University – Department of Computer Science and Engineering
Something is Missing
• The measure (2.66 bits/s) does not reflect the accuracy of participants’ responses
• This seriously weakens within-study and between-study comparisons
• A solution was proposed by Crossman (1956) and endorsed by Fitts (1964), Welford (1968), and others
13
25
York University – Department of Computer Science and Engineering
The Solution
• Transform W into We
• We is the “effective target width”• We = width of the distributions of “hits”• W reflects what participants were asked to do• We reflects what participants actually did
26
York University – Department of Computer Science and Engineering
Effective Target Width
• SDx – standard deviation in selection coordinates along axis of approach
• We is the central 96% of the spatial distribution• Assumption: distribution is normal (research shows that
the assumption holds reasonably well)• Nominal error rate is 4%• H = = 4.133 σ (Shannon, 1949)• See MacKenzie (1992a, 1992b)
We = 4.133 SDx
σπ e2
14
27
York University – Department of Computer Science and Engineering
4% Errors
28
York University – Department of Computer Science and Engineering
Less Than 4% Errors
15
29
York University – Department of Computer Science and Engineering
Greater Than 4% Errors
30
York University – Department of Computer Science and Engineering
Frequency Distribution Example1
1 From…
MacKenzie, I. S., Sellen, A., & Buxton, W. (1991). A comparison of input devices in elemental pointing and dragging tasks. Proceedings of the CHI `91 Conference on Human Factors in Computing Systems, pp. 161-166. New York: ACM. (Click here to view) (Google Scholar citations)
16
31
York University – Department of Computer Science and Engineering
“Effective” Index of Difficulty
• ID is the difficulty of the task presented to the participants
• IDe is the difficulty of the task achieved by the participants
• IDe reflects what participants actually did• May use De instead of D, if participants did not “go the
distance” (check data to determine this)
IDe = log2(D / We + 1)
32
York University – Department of Computer Science and Engineering
Some Data (Revisited)
17
33
York University – Department of Computer Science and Engineering
Compare
• Speed only:• Throughput = 2.66 bits/s
• Speed and accuracy:• Throughput = 1.81 bits
• Difference:• +47%
34
York University – Department of Computer Science and Engineering
So…
• We know how to calculate Throughput• Where do we begin?
18
35
York University – Department of Computer Science and Engineering
What is the Throughput for the Mouse?
• The question above is meaningless unless the test conditions are specified
• Conditions• Generic mouse• Standard mouse driver• Nominal gain setting• Serial task• Expert participants• Etc.
• Answer: TPMOUSE ≈ 4.5 bits/s• The above is a “baseline condition”
36
York University – Department of Computer Science and Engineering
Baseline Conditions
• Serve as a check on experimental methods; i.e., • Apparatus• Procedures• Data collection• Data analysis
• Obtaining a known and established result for a baseline condition means the methodology is sound
• Strengthens within-study comparisons• Strengthens between-study comparisons
19
37
York University – Department of Computer Science and Engineering
Clean up the Mess
Baseline condition (A) is the same in each study
38
York University – Department of Computer Science and Engineering
Clean up the Mess
Comparisons are valid, since baseline condition (A) is consistent between studies
20
39
York University – Department of Computer Science and Engineering
Example Study
• Four device conditions• Remote pointing
• Gyration GyroPoint (air)• Interlink RemotePoint
• Desktop pointing• Gyration GryoPoint (desk)• Microsoft Mouse 2.0
Reference:
MacKenzie, I. S., & Jusoh, S. (2001). An evaluation of two input devices for remote pointing. Proceedings of the Eighth IFIP Working Conference on Engineering for Human-Computer Interaction - EHCI 2001. pp. 235-249. Heidelberg, Germany: Springer-Verlag. (Click here to view) (Google Scholar citations)
40
York University – Department of Computer Science and Engineering
Gyration GyroPoint
21
41
York University – Department of Computer Science and Engineering
Interlink RemotePoint
42
York University – Department of Computer Science and Engineering
Results
22
43
York University – Department of Computer Science and Engineering
Questionnaire – “Ease of Use” (Q13)
5.0 5.0 3.1 1.9
44
York University – Department of Computer Science and Engineering
ISO 9241-9 Task #1 Demo
23
45
York University – Department of Computer Science and Engineering
ISO 9241-9 Task #2 Demo
46
York University – Department of Computer Science and Engineering
Thank you
Questions?
top related