Predicting Task Execution Time on Handheld Devices

Using the Keystroke Level Model

Annie Lu Luo and Bonnie E. John

School of Computer ScienceCarnegie Mellon University

CHI’05 – April 6, 2005

© Annie Luo, Carnegie Mellon University, 2005 slide 2

Motivation and goals

Keystroke Level Model (KLM) A priori prediction of expert user task time Intensively used on desktop computers Not yet been adapted to handheld devices

• Limited display size• Input device: stylus, touch-screen, hardware buttons• Interaction methods: tap, Graffiti, etc.

Investigate KLM on handheld UIs Applicability of model to novel interface modalities Accuracy of model predictions

© Annie Luo, Carnegie Mellon University, 2005 slide 3

KLM in brief

Describe a task by placing operators in a sequence K – keystroke (tap) P – point with mouse (stylus) H – homing (move hand from mouse to keyboard) (N/A) D (takes parameters) – drawing (N/A) R (takes parameters) – system response time M – mental preparation G – Graffiti stroke (580 ms – Fleetwood, et al 2002)

Five heuristic rules to insert candidate Ms into the sequence

Task execution time = Σ all operators involved

© Annie Luo, Carnegie Mellon University, 2005 slide 4

Start

Handheld task: Find information about the MET

1

City map

Museums list

2Soft keyboard

4

Scroll list

3Graffiti

Region map Street map

Query result

© Annie Luo, Carnegie Mellon University, 2005 slide 5

Create KLMs

One KLM for each of the four methods Used CogTool (John, et al 2004)

MacroMediaDreamWeaver

BehaviorRecorder

NetscapeHTML

event handler

ACT-REnvironment

Modeler mocks up interfaces as HTML

storyboard

Modeler demonstrates tasks

on the HTML storyboard

HTML mockups

Interface event

messagesvia

LiveConnect

ACT-Simplecode

based on KLM

KLM Trace

ACT-Simple complies code into ACT-R production

rules

© Annie Luo, Carnegie Mellon University, 2005 slide 6

Mozilla Firefox

Behavior Recorder

ACT-R

© Annie Luo, Carnegie Mellon University, 2005 slide 7

User study

10 expert PDA users (Female:Male = 3:7) At least one year experience using:

Palm series, pocket PC, or smart cell phone

Instructed to perform the task on a PalmVx Using four different methods (within subject design)

Training session before real session Repeating each method for 10 times

Data collection EventLogger: records system events to a log file Videotaped modeler’s behavior for verification

© Annie Luo, Carnegie Mellon University, 2005 slide 8

0.000

2.000

4.000

6.000

8.000

10.000

12.000

14.000

16.000

Map SoftKB Graffiti ScrollBar

Old Model Time

Old User Time

New Model Time

New User Time

New results since paper published

-9.3%

8.9% 5.8%

7.7%

Latest version of CogTool Better estimation of system response time Detailed analysis of model and user traces (140/400 removed)

2.3%

-1.4%-6.9%

-3.7%

© Annie Luo, Carnegie Mellon University, 2005 slide 9

Conclusion & Future work

KLMs produced with CogTool are effective for handheld user interfaces: Produces accurate execution time prediction Supports new input modalities: Graffiti

Future work: Detailed analysis of the user pauses (mental time) Use predictions of pauses to assist energy

management

© Annie Luo, Carnegie Mellon University, 2005 slide 10

Thank you!

Authors’ contact info:Bonnie John – bej@cs.cmu.eduAnnie Luo – luluo@cs.cmu.edu

The CogTool project:http://www.cs.cmu.edu/~bej/cogtool/

© Annie Luo, Carnegie Mellon University, 2005 slide 11

Participants information (backup)

User (gender) Device owned How long

1 (M) Palm Vx 5+ years

2 (M) Compaq iPAQ 3 years

3 (M) Palm IIIe 4 years

4 (M) Handspring Visor 3 years

5 (F) Handspring visor Pro 2 years

6 (F) Dell PDA 1 year

7 (M) iPAQ 3630 4 years

8 (M) Kyocera 7135 4+ years

9 (M) Handspring Visor Prism 3 years

10 (F) Palm VA 3 years

© Annie Luo, Carnegie Mellon University, 2005 slide 12

Results in paper (backup)Model Time vs. User Time

9.213

12.662 12.662

9.913

9.0055

12.8363

13.60328

10.29767

0

2

4

6

8

10

12

14

16

M1-MapNavigation

M2-Soft Keyboard M3-Graffiti M4-Scroll Bar

Tasks

Tim

e (s

ec)

Model TimeUser Time

(Average)

© Annie Luo, Carnegie Mellon University, 2005 slide 13

New results since paper published

0.000

2.000

4.000

6.000

8.000

10.000

12.000

Map SoftKB Graffiti ScrollBar

Model Time User Time

9.3%

-8.9% -5.8%

-7.7%

Better measurements of system response time Removed error trials (140 out of 400) Latest version of CogTool

© Annie Luo, Carnegie Mellon University, 2005 slide 14

Interface Widgets: - Buttons- Check boxes- Text fields- Pull-down lists- Links- Menus- Audio input- Audio output

© Annie Luo, Carnegie Mellon University, 2005 slide 15

Netscape

Behavior Recorder

ACT-Simple