1George Mason University

Human Factors & Applied Cognitive Program

This slide contains information in Note View. Switch to note view and print all of these out.

Be sure to set your printer to black and white.

You may need to change fonts if we have used some that you do not have (or if your printer or computer does funny things to our slides)

2George Mason University

Human Factors & Applied Cognitive Program

3George Mason University

Human Factors & Applied Cognitive Program

4George Mason University

Human Factors & Applied Cognitive Program

Cognitive Analysis of Dynamic Performance: Cognitive process analysis and modeling

Wayne D. Gray, Ph.D.

Deborah A. Boehm-Davis, Ph.D.

Workshop Presented at the HFES/IEA-2000 Conference in San Diego, CA

5George Mason University

Human Factors & Applied Cognitive Program

Outline of Workshop

Intro to the GOMS family of models

Keystroke Level GOMS

NGOMSL GOMS

CPM-GOMS

REVISED Tutorial Materials may be downloaded from:REVISED Tutorial Materials may be downloaded from:

http://hfac.gmu.edu/~graypubshttp://hfac.gmu.edu/~graypubs

6George Mason University

Human Factors & Applied Cognitive Program

Definition of GOMS

Characterization of a task in terms of The user's Goals

The Operators available to accomplish those goals

Methods (frequently used sequences of operators and sub-

goals) to accomplish those goals, and

If there is more than one method to accomplish a goal, the

Selection rules used to choose between methods.

7George Mason University

Human Factors & Applied Cognitive Program

What GOMS Is

GOMS is a task analysis technique Very similar to Hierarchical Task Analysis (Indeed, GOMS

is a hierarchical task analysis technique)

Hard part of task analysis is goal-subgoal decomposition

Hard part of GOMS is goal-subgoal decomposition

Different members of the GOMS family provide you with different sets of operators

With established parameters

But set is not complete (extensionable)

8George Mason University

Human Factors & Applied Cognitive Program

Where does GOMS go?

(When do you need to do a CTA versus a TA?)

Task analysis versus cognitive task analysis?

What distinguishes GOMS cognitive task analysis from other task analysis techniques?

E.g., Cognitive Task Analysis™

9George Mason University

Human Factors & Applied Cognitive Program

TASK ANALYSIS

10George Mason University

Human Factors & Applied Cognitive Program

Time Scale for GOMS

Scale(sec)

Time Units System Analysis Activities/Processes

World (theory)

105 days

104 hours Task Task analysis Subtasks Bounded

103 10 min Rationality

102 min Subtask Unit Task Analysis Unit Tasks

101 10 sec Unit task Cognitive task analysis Activities

100 1 sec Activity Embodied cognition Microstrategies Cognitive Band

10-1 100 ms Microstrategy Computational modelsof embodied cognition Elements

10-2 10 ms Elements Architectural Parameters Biological

10-3 1 ms Parameters Band

Levels of analysis (based on Newell’s Time Scale of Human Action)

11George Mason University

Human Factors & Applied Cognitive Program

Level 1: Task Analysis: Task Subtasks Subtasks

Scenario #1

SUBTASKsSUBTASKsHook Target ID#1

Hook Target ID#2

Hook Target ID#3

Hook Target ID#n

• • • •TASKComplete Argus Prime Experiment

Scenario #2

Complete Session #1Complete

Session #2Complete

Session #3SUBTASKsDuration

5 hr

1-2 hrs

12-30 min

30 sec to 3 min

Scenario #n

• • • •Scenario #3

12George Mason University

Human Factors & Applied Cognitive Program

Levels of analysis (based on Newell’s Time Scale of Human Action)

Scale(sec)

Time Units System Analysis Activities/Processes

World (theory)

105 days

104 hours Task Task analysis Subtasks Bounded

103 10 min Rationality

102 min Subtask Unit Task Analysis Unit Tasks

101 10 sec Unit task Cognitive task analysis Activities

100 1 sec Activity Embodied cognition Microstrategies Cognitive Band

10-1 100 ms Microstrategy Computational modelsof embodied cognition Elements

10-2 10 ms Elements Architectural Parameters Biological

10-3 1 ms Parameters Band

13George Mason University

Human Factors & Applied Cognitive Program

Level 2: Subtask Unit Tasks

TargetacquisitionClassified?

yes Targetclassification

noFeedbkcheck?

noFeedback checkyes

14George Mason University

Human Factors & Applied Cognitive Program

Levels of analysis (based on Newell’s Time Scale of Human Action)

Scale(sec)

Time Units System Analysis Activities/Processes

World (theory)

105 days

104 hours Task Task analysis Subtasks Bounded

103 10 min Rationality

102 min Subtask Unit Task Analysis Unit Tasks

101 10 sec Unit task Cognitive task analysis Activities

100 1 sec Activity Embodied cognition Microstrategies Cognitive Band

10-1 100 ms Microstrategy Computational modelsof embodied cognition Elements

10-2 10 ms Elements Architectural Parameters Biological

10-3 1 ms Parameters Band

15George Mason University

Human Factors & Applied Cognitive Program

Level 3: Cognitive Task Analysis: Unit Tasks Activities

TargetacquisitionClassified?

yes Targetclassification

noFeedbkcheck?

noFeedback checkyes

Classified?

Method for goal: Determine if target is classifedStep 1 Select target by mouse clickStep 2 Note target ID#

Step 3Determine if radio button in info window is black

Step 4Verify that info window id# is same as target id#

16George Mason University

Human Factors & Applied Cognitive Program

Levels of analysis (based on Newell’s Time Scale of Human Action)

Scale(sec)

Time Units System Analysis Activities/Processes

World (theory)

105 days

104 hours Task Task analysis Subtasks Bounded

103 10 min Rationality

102 min Subtask Unit Task Analysis Unit Tasks

101 10 sec Unit task Cognitive task analysis Activities

100 1 sec Activity Embodied cognition Microstrategies Cognitive Band

10-1 100 ms Microstrategy Computational modelsof embodied cognition Elements

10-2 10 ms Elements Architectural Parameters Biological

10-3 1 ms Parameters Band

17George Mason University

Human Factors & Applied Cognitive Program

Level 4: Embodied Cognition: Activity Microstrategy

Method for goal: Determine if target is classifedStep 1 Select target by mouse clickStep 2 Note target ID#

Step 3Determine if radio button in info window is black

Step 4Verify that info window id# is same as target id#

50verify crsr @ target

attend trgt ID#50retrieve trgt ID#50initiate mouse down

50mouseDn on target100attend visual50Information Window is updated

0initiate eye mvmt50eye mvmt30perceive black radio button100verifytrgt 50 verify trgt ID# 50 retrieve trgt ID# 50

http://hfac.gmu.edu/~mm

18George Mason University

Human Factors & Applied Cognitive Program

Levels of analysis (based on Newell’s Time Scale of Human Action)

Scale(sec)

Time Units System Analysis Activities/Processes

World (theory)

105 days

104 hours Task Task analysis Subtasks Bounded

103 10 min Rationality

102 min Subtask Unit Task Analysis Unit Tasks

101 10 sec Unit task Cognitive task analysis Activities

100 1 sec Activity Embodied cognition Microstrategies Cognitive Band

10-1 100 ms Microstrategy Computational modelsof embodied cognition Elements

10-2 10 ms Elements Architectural Parameters Biological

10-3 1 ms Parameters Band

19George Mason University

Human Factors & Applied Cognitive Program

Level 5: Microstrategies Elements(Where GOMS doesn’t go!)

Production Rules

Declarative memory elements Internal (created on RHS of production rules)

External (created by shifts of attention in the external environment)

20George Mason University

Human Factors & Applied Cognitive Program

Levels of analysis (based on Newell’s Time Scale of Human Action)

Scale(sec)

Time Units System Analysis Activities/Processes

World (theory)

105 days

104 hours Task Task analysis Subtasks Bounded

103 10 min Rationality

102 min Subtask Unit Task Analysis Unit Tasks

101 10 sec Unit task Cognitive task analysis Activities

100 1 sec Activity Embodied cognition Microstrategies Cognitive Band

10-1 100 ms Microstrategy Computational modelsof embodied cognition Elements

10-2 10 ms Elements Architectural Parameters Biological

10-3 1 ms Parameters Band

21George Mason University

Human Factors & Applied Cognitive Program

Level 6: Elements Parameters (GOMS doesn’t go here either!)

w -- amount of attentional capacity

d -- decay rate

s -- fluctuations in the strength of declarative memory elements

rt -- retrieval threshold

22George Mason University

Human Factors & Applied Cognitive Program

KR not KE

GOMS (as with most task analysis methods) focuses on

Knowledge representation, not on

Knowledge elicitation

There are many sources of knowledge elicitation techniques

23George Mason University

Human Factors & Applied Cognitive Program

Task Analysis versus Functionality

A task analysis does not guarantee functionality, seeKieras, D. E. (in press). Task analysis and the design of

functionality, CRC Handbook of Computer Science and Engineering.: CRC Press, Inc.

for a cogent discussion of this issue

24George Mason University

Human Factors & Applied Cognitive Program

Definition of GOMS

Characterization of a task in terms of The user's Goals

The Operators available to accomplish those goals

Methods (frequently used sequences of operators and sub-

goals) to accomplish those goals, and

If there is more than one method to accomplish a goal, the

Selection rules used to choose between methods.

25George Mason University

Human Factors & Applied Cognitive Program

Example of G-O-M-S

To carry out a GOMS analysis of the following task involving a digital clock:

Set the clock

Top level goal: SET CLOCK

26George Mason University

Human Factors & Applied Cognitive Program

Example of G-O-M-S: Goals

Goals and subgoals

Set Clock

Set Hour Set Min

27George Mason University

Human Factors & Applied Cognitive Program

Example of G-O-M-S: Operators

Operators are the most elementary steps in which you choose to analyze the task.

Reach <type> button

Hold <type> button

Release <type> button

ClickOn <type> button

Decide: if <x> then <y>

Verify

28George Mason University

Human Factors & Applied Cognitive Program

Example of G-O-M-S: Methods

Top-level user goalsSET-CLOCK

Method for goal: SET-CLOCKStep 1. Hold TIME button

Step 2. Accomplish goal: SET-HOUR

Step 3. Accomplish goal: SET-MIN

Step 4. Release TIME button

Step 5. Return with goal accomplished

Method for goal: SET-<digit>Step 1. ClickOn <digit> button

Step 2. Decide: If target <digit> = current <digit>, then return with goal accomplished

Step 3. Goto 1

29George Mason University

Human Factors & Applied Cognitive Program

Example of G-O-M-S: Selection rules

No selection rules in this example as this clock has only ONE method for accomplishing each goal, but . . .

Selection rule for goal: SET-HOUR

If target HOUR ≤ 4 hours from current HOUR, then Accomplish Goal: ClickOn HOUR

If target HOUR > 4 hours from current HOUR, then Accomplish Goal : Click&Hold HOUR

30George Mason University

Human Factors & Applied Cognitive Program

Applications of GOMS

Case 1. Design of mouse-driven text editor

Case 2. Directory assistance workstation

Case 3. Space operations database system (for orbital objects)

Case 4. Bank deposit reconciliation system.

Case 5. CAD system for mechanical design.

Case 6. Television control system.

Case 7. Nuclear power plant operator's associate.

Case 8. Intelligent tutoring system.

Case 9. Industrial scheduling system.

Case 10. CAD system for ergonomic design.

Case 11. Telephone operator workstation.

List compiled by John & Kieras (1997a).

31George Mason University

Human Factors & Applied Cognitive Program

Where does GOMS go?

Organization Issues [anthropology, social Ψ, Ι/Ο]

Conceptual Ιssues, Mental Models [cognitive science]

Procedural Ιssues (methods) [GΟMS]

Perceptual Ιssues [tradtional HF, cognitive

engineering, Tullis, Wickens]

Usability writ large

32George Mason University

Human Factors & Applied Cognitive Program

Development process

without analytic modeling

Development process

with analytic modeling

34George Mason University

Human Factors & Applied Cognitive Program

GOMS as Analytic Modeling

GOMS analysis produces a model of behavior

Given a task, the model predicts the methods, or sequences of operators, that a person will perform to accomplish that task

Can look at the GOMS model in different ways to qualitatively and quantitatively assess different types of performance

35George Mason University

Human Factors & Applied Cognitive Program

Scope of GOMS: What it can do

Predict the sequence of operators an expert will perform

Predict performance time of expert users - even in real-world situations

Predict learning time in relatively simple domains

Predict savings due to previous learning

Help design on-line help and manuals

36George Mason University

Human Factors & Applied Cognitive Program

Scope of GOMS: What it can do, con’t

GOMS has been applied to both: User-driven interaction

“Situated” or event-driven interaction

37George Mason University

Human Factors & Applied Cognitive Program

Scope of GOMS:What it might be able to do Research has made progress on

Predicting the number of some types of errors

see discussion in: Gray, W. D. (2000). The nature and processing of errors in interactive behavior. Cognitive Science, 24(2), 205-248.

Predicting the effects of display layout on performance time

38George Mason University

Human Factors & Applied Cognitive Program

Scope of GOMS: What it can't do

Predict problem-solving behavior

Predict how GOMS structure grows from user experience

Predict behavior of casual users, individual differences...

Predict the effects of fatigue, user preference, organizational impact...

39George Mason University

Human Factors & Applied Cognitive Program

General Factors to Considerin GOMS Models

When deciding what type of GOMS model you need, you must consider...

what control structure

what level of analysis

whether to approximate behavior with serial or parallel processes

...different uses of GOMS models lead to different values of these factors

These factors will be a recurring theme

40George Mason University

Human Factors & Applied Cognitive Program

GOMS Family of Analysis Methods

Keystroke-Level Model

CMN-GOMS

NGOMSL

CMN-GOMS for Highly Interactive Tasks

CPM-GOMS

= “worked example” provided in this HFES workshop

41George Mason University

Human Factors & Applied Cognitive Program

Break time

42George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Intro

The simplest of all GOMS models: OM only!!! No explicit goals or selection rules

Operators and Methods (in a limited sense) only

“Useful where it is possible to specify the user’s interaction sequence in detail” (CMN83, p. 259).

Control structure: Flat

Serial or Parallel: Serial

Level of Analysis: Keystroke-level operators

43George Mason University

Human Factors & Applied Cognitive Program

area code exchange line pin1 2 3 4 5 6 7 8 9 10 11 12 13 14

num 7 0 3 9 9 3 1 3 5 7 1 2 3 4

Keystroke-Level Model: Example

TAO example

44George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Overview

Step 1: Lay out assumptions

Step 2: Write out the basic action sequence (list the keystroke-level physical operators involved in doing the task)

Step 3: Select the operators and durations that will be used

Step 4: List the times next to the physical operators for the task

Step 4a: If necessary, include system response time operators for when the user must wait for the system to respond

Step 5: Next add the mental operators and their times

Step 6: Sum the times of the operators

45George Mason University

Human Factors & Applied Cognitive Program

Keystroke-level Model: Operators

K: Keystroke

T(n): Type a sequence of n characters on a keyboard

P: Point with mouse to a target on a display

B: Press or release mouse button

BB: Click mouse button

H: Home hands to keyboard or mouse

M: Mental act of routine thinking

W(t): Waiting time for system to respond

46George Mason University

Human Factors & Applied Cognitive Program

Card, Moran, and Newell on “Mentals”

“M operations represent acts of mental preparation for applying physical operations. Their occurrence does not follow directly from the physical encoding, but from the specific knowledge and skill of the user” p. 267

“The rules for placing M’s embody psychological assumptions about the user and are necessarily heuristic, especially given the simplicity of the model” p. 267.

47George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

Basic psychological principle: physical operations in methods are chunked into submethods.

RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit.

•Pointing to a cell on a spreadsheet is pointing to an argument -- no M•Pointing to a word in a manuscript is pointing to an argument -- no M•Pointing to a icon on a toolbar is pointing to a command -- M•Pointing to the label of a drop-down menu is pointing to a command -- M

48George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

Rules 1-4 are heuristics (rules of thumb) for deleting mentals

“A single psychological principle lies behind all the deletion heuristics . . . physical operations in methods are chunked into submethods” p. 268

49George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

Basic psychological principle: physical operations in methods are chunked into submethods.

RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit.

RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).

•That is, the “M” drops out because the “P” and “BB” belong together in a chunk -- mental unit.•The button press “BB” is fully anticipated as the cursor is being moved to the target.

50George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

Basic psychological principle: physical operations in methods are chunked into submethods.

RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit.

RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).

RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first.

•Works with command names -- but what is a command name in a GUI interface?

•Physical actions: P(File)+ B + P(Save) + B•RULE 0: MP + MB + MP + MB•RULE 1: MPB + MPB•Does rule 2 apply to eliminate the middle mental? MPBPB ?

51George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

Basic psychological principle: physical operations in methods are chunked into submethods.

RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit.

RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).

RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first

RULE 3: If a K is a redundant terminator (e.g., the terminator of a command immediately following the terminator of its argument), then delete the M in front of it.

•Applies to clicking OKAY in dialog buttons after you select a command; e.g., in Powerpoint, you have selected text, gone to the FORMAT:FONT palette, clicked on bold, and now point and click on OKAY -- pointing to and clicking on OKAY is PBB, not MPBB

52George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

Basic psychological principle: physical operations in methods are chunked into submethods.

RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit.

RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).

RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first

RULE 3: If a K is a redundant terminator (e.g., the terminator of a command immediately following the terminator of its argument), then delete the M in front of it.

RULE 4: If a K terminates a constant string (e.g., a command name), then delete the M in front of it; but if the K terminates a variable string (e.g., an argument string), then keep the M in front of it.

53George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

The four heuristics do NOT capture the notion of method chunks precisely -- these are only approximations

Ambiguities: Is something “fully anticipated” or is something else a “cognitive unit”?

Much of this ambiguity stems from variations in expertise of the users we are modeling

54George Mason University

Human Factors & Applied Cognitive Program

Heuristics for inserting mental operators

Basic psychological principle: physical operations in methods are chunked into submethods.

RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit.

RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB).

RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first

RULE 3: If a K is a redundant terminator (e.g., the terminator of a command immediately following the terminator of its argument), then delete the M in front of it.

RULE 4: If a K terminates a constant string (e.g., a command name), then delete the M in front of it; but if the K terminates a variable string (e.g., an argument string), then keep the M in front of it.

55George Mason University

Human Factors & Applied Cognitive Program

KLM--mentals: example 1. Example: SET COLUMN WIDTH 5<cr>

List the keystroke level physical operators involved in doing the task KKKKKKKKKKKKKKKKKKK (19 K’s)

RULE 0 M+KKKK+M+KKKKKKK+M+KKKKKK+K+M+K or M+4K(set_)+M+7K(column_)+M+6K(width_)+1K(5)+M+1K(<cr>)

RULE 1 no change in this example

RULE 2 M+17K(set_column_width_)+1K(5)+M+1K(<cr>)

RULE 3 No change in this example

Rule 4 No change in this example

56George Mason University

Human Factors & Applied Cognitive Program

KLM--mentals: example 2

Example: spellcheck “catelog” List the keystroke level physical operators involved in doing the task

P+BBBB+P+BB (where BB is a mousedown + mouseup, and BBBB is a doubleclick) RULE 0

P+M+BBBB+M+P+BB RULE 1

n/a RULE 2

n/a (“catelog” + spellcheck do not form a cognitive unit) RULE 3

n/a

RULE 4 n/a

57George Mason University

Human Factors & Applied Cognitive Program

KLM--mentals: example 3

Example: save a file on a Mac using menus List the keystroke level physical operators involved in doing the

task P+B+P+B

RULE 0 M+P+M+B+M+P+M+B

RULE 1 M+P+B+M+P+B

RULE 2 n/a or M+P+B+P+B ???

Issue: Is this FILE-->SAVE menu selection a single cognitive unit or two?

58George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m1 current

Step 1: Lay out your assumptions There are several fields on the display, first thing that any error

recovery method must do is to identify the field to be changed. In this case the field is the calling-card field (CCN).

For purposes of this exercise, we assume the error is made in the second number of the exchange.

TAO’s hands are on the keyboard

Step 2: Write out the basic action sequence (the physical operators)

ƒkey(ccn) + digit(14) + enterKey

59George Mason University

Human Factors & Applied Cognitive Program

operator abbrev durationmental M 1200mseckeystroke-<key> K 280msecmouseDown or Up B 100msecclick (mouseD & Up) BB 200msechoming H 400mecpointing w/mouse P 1100msecdoubleClick BBBB 400msec

Keystroke-Level Model: m1 current

Step 3: select the operators and durations that will be used

We will use the ones from Kieras (1993).

60George Mason University

Human Factors & Applied Cognitive Program

Method 1 of TAO task NUM op type timepress reset function keyfKEY(ccn) 1 K 0.28type digits digit 14 K 3.92outpulse new number to dbaseenter 1 K 0.28

total time 4.48

Keystroke-Level Model: m1 current

Step 4: List the times next to the physical operators for the task.

61George Mason University

Human Factors & Applied Cognitive Program

Method 1 of TAO task NUM op type timeM before cmd 1 M 1.20press reset function key fCCN 1 K 0.28type digits digit 14 K 3.92M terminates argument string 1 M 1.20outpulse new number to dbase enter 1 K 0.28

total time 6.88

Keystroke-Level Model: m1 current

Step 5: Next add the mental operators and their times

Step 6: Sum the times of the operators Predicted time for current method is 6.88 sec

(note: this time is the same regardless of “where” the error is made)

62George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m2 bs/delete

Step 1: Lay out your assumptions s/a model 1 except;

delete key backs up and deletes each digit

Step 2: Write out the basic action sequence (the physical operators)

ƒkey(ccn) + delKey(10) + digit(10) + enterKey

Step 3: Same operators as for model 1.

63George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m2 bs/delete

Step 4: List the times next to the physical operators for the task.

Method 2: bs/delete NUM op type timepress reset function key fCCN 1 K 0.28bs/delete to digit delKey 10 K 2.80digits to retype digit 10 K 2.80outpulse new num to dbaseenter 1 K 0.28total time 6.16

64George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m2 bs/delete

Step 5: Next add the mental operators and their times

Step 6: Sum the times of the operators

Method 2: bs/delete NUM op type timeM before cmd 1 M 1.20press reset function key fCCN 1 K 0.28bs/delete to digit delKey 10 K 2.80digits to retype digit 10 K 2.80verify done 1 M 1.20outpulse new num to dbase enter 1 K 0.28total time 8.56

65George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model:m3 bkup/delete

Step 1: Lay out your assumptions s/a model 1 except;

backup key backs up without deleting. Delete key backs up and deletes

Step 2: Write out the basic action sequence (the physical operators)

ƒkey(ccn) + bkupKey(9) + delKey(1) + digit(1) + enterKey

Step 3: Same operators as for model 1.

66George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m3 bkup/delete

Step 4: List the times next to the physical operators for the task.

Method 3: bkup-delete NUM op type timepress reset function key fCCN 1 K 0.28backup to digit bkup 9 K 2.52delete digit del 1 K 0.28digits to retype digit 1 K 0.28outpulse new num to dbase enter 1 K 0.28total time 3.64

67George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m3 bkup/delete

Step 5: Next add the mental operators and their times (Your turn!!! Our answer are on the next page, no peeking!!!)

Method 3: bkup-delete NUM op type time

press reset function key fCCN 1 K 0.28

backup to digit bkup 9 K 2.52

delete digit del 1 K 0.28

digits to retype digit 1 K 0.28

outpulse new num to dbase enter 1 K 0.28

total time

68George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m3 bkup/delete

Step 5: KLM w/mentals.

Method 3: bkup-delete NUM op type timeset up workstation to retype number 1 M 1.20press reset function key fCCN 1 K 0.28backup to digit bkup 9 K 2.52delete digit del 1 K 0.28digits to retype digit 1 K 0.28verify done 1 M 1.20outpulse new num to dbase enter 1 K 0.28total time 6.04

69George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m4 zap-gp

Step 1: Lay out your assumptions s/a model 1 except;

Four separate function keys, zaps (deletes) either area code, exchange, line, or pin number. Retyping need only retype the zapped numbers.

Step 2: Write out the basic action sequence (the physical operators)

ƒkey(ccn) + zapExch(1) + digit(3) + enterKey

Step 3: Same operators as for model 1.

70George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m4 zap-gp

Step 4: List the times next to the physical operators for the task. Your turn!! (Our answer are on the next page, no peeking!!!)

Method 4: zap-gp NUM op type time

total time

71George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m4 zap-gp

Step 5: Next add the mental operators and their times (Your turn!!! Our answer are on the next page, no peeking!!!)

Method 4: zap-gp NUM op type time

press reset function key fCCN 1 K 0.28

zap-group zap 1 K 0.28

digits to retype digit 3 K 0.84

outpulse new num to dbase enter 1 K 0.28total time

72George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: m4 zap-gp

Step 5: KLM model w/mentals

Method 4: zap-gp NUM op type timeset up workstation to retype number 1 M 1.20press reset function key fCCN 1 K 0.28zap-group zap 1 K 0.28digits to retype digit 3 K 0.84verify done 1 M 1.20outpulse new num to dbase enter 1 K 0.28total time 4.08

73George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model:

Summary

Of the three new methods, only one seems likely to be fast enough to justify expense of redesign

predicted timeMethod 1: Current 6.88Method 2: bs/delete 8.56Method 3: bkup-delete 6.04Method 4: zap-gp 4.08

74George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface

Fi le Edi t Databases

123- 4567234- 5678345- 6789345- 7899765- 4321

The task: To issue a query on telephone numbers to one out of several databases. The users will be working in a window system, and the information that is to be looked up in the databases can be assumed to be visible on the screen.

Thanks to Bonnie John for allowing us to use this material!!!

Fig 1

The screen as it looks before the menu choices. The telephone numbers to be looked up are in the window at the bottom of the screen.

75George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface (2)

To use this interface, the user first pulls down a menu from the menubar at the top of the screen. This menu contains the names of the databases and the user positions the mouse cursor over the name of the relevant database in this list.

File Edit DatabasesDatabasesDB1 DB2 DB4DB3

123-4567 234-5678 345-6789 345-7899 765-4321

Fig 2

76George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface (3)

A hierarchical submenu appears with legal queries for the chosen database. This submenu contains an average of four alternatives, and the user moves the mouse until it highlights the option "query on telephone number." The user then releases the mouse button.

File Edit DatabasesDatabasesDB1 DB2 DB4DB3 query on name

query on address query on businessquery on telephone

123-4567 234-5678 345-6789 345-7899 765-4321

Fig 3

77George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface (4)

This causes a standard-sized dialog box to appear in the middle of the screen as shown in Figure 4. The large field at the top is initially empty. This field will eventually list the telephone numbers to be submitted to the database as a query. The dialog box does not overlap the window with the list of telephone numbers.File Edit Databases

Inquiry Box

Inquiry on telephone number

OK Cancel

Add

Delete

Update

Help

123-4567 234-5678 345-6789 345-7899 765-4321

Fig 4

78George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface (5)

"The user clicks in the input field (the one-line field below the prompt "Inquiry on telephone number") and types the telephone number.

File Edit Databases

Inquiry Box

Inquiry on telephone number

OK Cancel

Add

Delete

Update

Help

123-4567

123-4567 234-5678 345-6789 345-7899 765-4321

File Edit Databases

Inquiry Box

Inquiry on telephone number

OK Cancel

Add

Delete

Update

Help

123-4567

123-4567 234-5678 345-6789 345-7899 765-4321

"The user clicks on the "Add" button to add the number to the query.Fig 5 Fig 6

79George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface (6)

The state of the dialog box after the user's click on "Add."

File Edit Databases

Inquiry Box

Inquiry on telephone number

OK Cancel

Add

Delete

Update

Help

123-4567

123-4567

123-4567 234-5678 345-6789 345-7899 765-4321

File Edit Databases

Inquiry Box

Inquiry on telephone number

OK Cancel

Add

Delete

Update

Help

123-4567

123-4567

123-4567 234-5678 345-6789 345-7899 765-4321

"If the query is for a single telephone number, the user then clicks on the "OK" button to submit the query.

Fig 7 Fig 8

80George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface (7)

Model 1: Dialog Box, one queryaction/operator sequence #ops KLM op time

Select dbase & query typepoint to "Databases" menu 1 P 1.10clickOn menu 1 BB 0.20point to "DB3" 1 P 1.10point to "query on telephone" 1 P 1.10clickOn 1 BB 0.20

Enter telephone numberpoint to "input field" 1 P 1.10clickOn input field 1 BB 0.20move hands to keyboard 1 H 0.40type in telephone number 8 K 2.24move hand to mouse 1 H 0.40point to "Add" button 1 P 1.10clickOn "Add" button 1 BB 0.20point to "OK" button 1 P 1.10clickOn "OK" button 1 BB 0.20

total time 10.64

81George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Dialog box interface (8)

Model 1: Dialog Box, one queryaction/operator sequence #ops KLM op time

Select dbase & query typebegin direct-manipulation sequence 1 M 1.20point to "Databases" menu 1 P 1.10clickOn menu 1 BB 0.20point to "DB3" 1 P 1.10point to "query on telephone" 1 P 1.10clickOn "query on telephone" 1 BB 0.20

Enter telephone numberbegin direct-manipulation sequence 1 M 1.20point to "input field" 1 P 1.10clickOn input field 1 BB 0.20move hands to keyboard 1 H 0.40get number 1 M 1.20 Must either reread it or recall it.type in telephone number 8 K 2.24begin terminate entry 1 M 1.20move hand to mouse 1 H 0.40point to "Add" button 1 P 1.10clickOn "Add" button 1 BB 0.20

Exit palettebegin close palette 1 M 1.20point to "OK" button 1 P 1.10clickOn "OK" button 1 BB 0.20

total time 16.64

82George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Pop-up menu (1)

Fig 9 Fig 10

File Edit Databases

123-4567 234-5678 345-6789 345-7899 765-4321

File Edit Databases

123-4567 234-5678 345-6789 345-7899 765-4321

DB1 DB2 DB4DB3

As previously mentioned, it can be assumed that the telephone number(s) in question is/are already visible on the screen. (Figure 9)

To query a number, the user moves the mouse cursor to the telephone number on the screen and clicks on the mouse button. This causes a pop-up menu to appear over the number with one element for each database for which queries can be performed with telephone numbers as keys. (Figure 10)

83George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Pop-up menu (2)

The user moves the mouse until the desired database is highlighted in the pop-up menu. (Figure 11)

The user then clicks the mouse button (the system knows which number to search on because the user pointed to it when calling up the menu). (Figure 12)

Fig 11 Fig 12

File Edit Databases

123-4567 234-5678 345-6789 345-7899 765-4321

DB1 DB2 DB4DB3

File Edit Databases

123-4567 234-5678 345-6789 345-7899 765-4321

84George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Pop-up menu (3)

Model 3: Pop-Up Menu, one queryaction/operator sequence #opsKLM op time

Enter telephone number

total time

•Your turn!! Action/operator sequence only. No mentals. (Our answer are on the next page, no peeking!!!)

85George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Pop-up menu (3)

Now try adding the mentals. (Our answer are on the next page, no peeking!!!)

Model 3: Pop-Up Menu, one queryaction/operator sequence #ops KLM op time

Enter telephone number

point to first telephone number 1 P 1.10

clickOn number 1 BB 0.20

point to "DB3" 1 P 1.10

clickOn 1 BB 0.20

total time 2.60

86George Mason University

Human Factors & Applied Cognitive Program

Keystroke-Level Model: Pop-up menu (4)

KLM w/mentals

Model 3: Pop-Up Menu, one queryaction/operator sequence #ops KLM op time

Enter telephone numberbegin direct-manipulation sequence 1 M 1.20point to first telephone number 1 P 1.10clickOn number 1 BB 0.20point to "DB3" 1 P 1.10clickOn 1 BB 0.20

total time 3.80

87George Mason University

Human Factors & Applied Cognitive Program

Dialog Box Pop-Up Menu CVN 1 query 2 queries 1 query 2 queries sd as % of

mean sd mean sd mean sd mean sd meansCold heuristic estimates 12 20.30 26.10 29.40 30.40 8.00 7.40 14.00 15.10 110%Warm heuristic estimates 10 12.90 10.70 21.40 21.60 4.70 2.70 9.10 5.30 84%Hot heuristic estimates 15 13.80 6.10 20.00 9.50 6.10 3.50 9.40 5.50 50%original GOMS (L&C College)19 16.60 3.70 22.60 5.40 5.80 0.80 11.20 1.90 21%CMU students 19 14.70 3.30 22.60 4.80 4.60 1.00 8.70 1.80 22%PSYC645 Sp96 12 15.76 2.97 24.66 3.28 5.02 1.78 9.30 3.09 20%

User testing 20 15.40 3.00 25.50 4.70 4.30 0.60 6.50 0.90 18%

Difference from UT [UT - <condition> = ??]Cold heuristic estimates -4.90 -3.90 -3.70 -7.50Warm heuristic estimates 2.50 4.10 -0.40 -2.60Hot heuristic estimates 1.60 5.50 -1.80 -2.90original GOMS (L&C College) -1.20 2.90 -1.50 -4.70CMU students 0.70 2.90 -0.30 -2.20PSYC645 Sp96 -0.36 0.84 -0.72 -2.80

KLM vs “expert judgment”

88George Mason University

Human Factors & Applied Cognitive Program

Break time

(Lunch Time!)

89George Mason University

Human Factors & Applied Cognitive Program

NGOMSL

Natural Language GOMS based on structured natural language notation and a procedure

for constructing them

models are in program form

Control Structure: Hierarchical goal stack

Serial or parallel: Serial

Level of Analysis: As necessary for your design question

90George Mason University

Human Factors & Applied Cognitive Program

NGOMSL - why?

More powerful than KLM. Much more useful for analyzing large systems

More built-in cognitive theory

Provides predictions of operator sequence, execution time, and time to learn the methods

91George Mason University

Human Factors & Applied Cognitive Program

NGOMSL - Overall Approach

Step 1: Perform goal/subgoal decomposition Step 2: Develop a method to accomplish each goal

List the actions/steps the user has to do goal (at as general and high-level as possible for the current level of analysis)

Identify similar methods/collapse where appropriate

Step 3: Add flow of control (decides) Step 4: Add verifies Step 5: Add perceptuals, etc. Step 6: Add mentals for retrieves, forgets, recalls Step 7: Add times for each step Step 8: Calculate total time

92George Mason University

Human Factors & Applied Cognitive Program

NGOMSL - Example

Car clock

Presetting radio stations simple

perverse

93George Mason University

Human Factors & Applied Cognitive Program

ssEJ

ssSEEK

– TUNE +

PUSH CLOCK

ON OFF

FMAM

– +BASS

– +TREBLE

L RBALANCE

L RFADE

2:411 2 3

4 5 6

Provides predictions of methods and operators used to complete a task. If you provide estimates of operator-duration, you can get predictions of error-free expert performance time.

NGOMSL--Car Radio example (1)

94George Mason University

Human Factors & Applied Cognitive Program

set clockset mode

set hourset minute

release knobturn knobdepress knob (and keep it depressed)

press tune “-” buttonpress tune “+” button

NGOMSL-- Car Radio example (2)

Goal/subgoal hierarchy

95George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- Car Radio example (3)

Assumptions User's hands are NOT on the buttons at the beginning.

Time to move hand & arm to time change level is estimated by 2ft "reach" from Barnes (1963): 410 msec.

D = device time, time for clock to move forward one number, = 500 msec (we estimate this on the next slide)

I am assuming a 12-hr clock. Seems good assumption for a car clock.

Radio is off at beginning and end.

Begin knowing the time you want to set the clock to.

96George Mason University

Human Factors & Applied Cognitive Program

Right Hand

Cognitive Operators

Visual Perception

Minimum time required to perceive clockTime=targetTime and to release the toggle lever

perceive info (X)

290

verify info (X)

50

initiate release (X)

50

release (X)

100

NGOMSL-- Car Radio example (4)

D = device time, time for clock to move forward one number, = 500 msec

97George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- Car Radio example (5)

stmt time oper

op time

tot time assumptions

Method for goal: SET-CLOCK 0.1 0.10

Step 1Accomplish goal: set Mode to set Clock 0.1 0.10

Step 2

Decide: If curHr≠targetHr Then: Accomplish goal: Change Hour display 0.1 0.10

Step 3

Decide: If curMin≠targetMin Then: Accomplish goal: Change Minute display 0.1 0.10

Step 4 Release "vol/on/off" knob 0.1 B 0.10 0.20

Step 5 Return with goal accomplished 0.1 0.10

98George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- Car Radio example (6)

stmt time oper

op time

tot time assumptions

Method for goal: set Mode to set Clock 0.1 0.10

Step 1Recall "vol/on/off" is modeSwitch 0.1 M 0.50 0.60

retrieval from LTM or equivalent (location and perception of labels?) needed and takes ≈ 500 msec

Step 2

Decide: If location of modeSwitch is not known then Locate modeSwitch 0.1 M 1.20 1.30

For this example we assume that location is NOT known. If known then tot. time for this step would be the stmt time, 0.10 sec

Step 3Reach & Grasp "volume/on/off" knob 0.1 R 0.48 0.58

source: Barnes, 1963, reach = 410 msec, grasp = 070 msec

Step 4 Turn knob 0.1 T 0.34 0.44 source: Barnes, 1963

Step 5Home thumb to "volume/on/off" knob 0.1 H 0.40 0.50

Step 6 Press and hold knob 0.1 B 0.10 0.20

Step 7 Return with goal accomplished 0.1 0.10

99George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- Car Radio example (7)

stmt time oper

op time

tot time assumptions

Method for goal: Change <time> display 0.1 0.10

Step 1

Decide: If location of setTime switch is not known then Locate setTime switch 0.1 M 1.20 1.30 assume that loc is not known

Step 2

Decide: If finger is NOT on setTime switch then: Reach to setTime switch 0.1 R 0.41 0.51

Step 3Retrieve-from-LTM that <time> = "+ or -" 0.1 M 0.50 0.60

retrieval from LTM or equivalent (location and perception of labels?) needed and takes ≈ 500 msec

Step 4 Determine current setting 0.1 P 0.32 0.42

based upon cpm-goms analysis it should take 420 msec for eye to move to clock and for user to perceive the hour.

Step 5Press and hold setTime switch to <time> 0.1 B 0.10 0.20

cpm-goms calc. is ≈250 msec, go with KLM

Step 6Hold until <curTime> = <targetTime>, (500 x #digits) W

System response time is 500 msec/digit, this is enough time for Ss to perceive and initiate keyUp response.

Step 7 Verify display <time> 0.1 M 1.20 1.30Step 8 Return with goal accomplished 0.1 0.10

100George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- Car Radio example (8)

time for SetClock goal 0.70time for setMode goal 3.82

time for setHour from 1 to 10 9.03

time to setMin from 56 to 50 31.53total 45.08

101George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting a station (1)

Locate Station

Select AM/FM

Initiate seek

Identify StationTerminate

seekSet Mode to

RadioetcSet PreSETEnter PreSet

MODEEnter PreSet

ButtonExit PreSet

MODEGeneral goal hierarchy for presetting a stationPreSET Network

Show

102George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting a station (2)

Note differences between this and previous slide.

Locate Station

Select AM/FM

PreSET Network Show

Set Mode to Radio

etc

Set PreSET

Goal hierarchy for optimal design

press & hold preset button until sound returns

seek & identify

103George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting a station (3)

MODEL 1: SETTING A PRESET TO A NETWORK SHOW - OPTIMAL DESIGN

Method for goal: preset network showStep1: Decide: IF radio is not playing, THEN Accomplish goal:

set-mode-to-radioStep2: Decide: IF show N.E. intended show, THEN Accomplish

goal: locate-stationStep3: Accomplish goal: set-presetStep4: Return with goal accomplished

Method for goal: locate-stationStep1: Decide: IF band N.E. desired band, THEN Accomplish

goal: select AM/FMStep2: Accomplish goal: seek&identifyStep3: Return with goal accomplished

Method for goal: select AM/FMStep1: Reach to button for desired bandStep2: Press button for desired bandStep3: Verify band correctStep4: Return with goal accomplished

104George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting a station (4)

Method for goal: seek&identifyStep1: Reach to right side of "Seek" button Step2: Press "Seek" buttonStep3: Decide: IF show N.E. intended show, THEN Goto 2Step4: Verify show correctStep5: Return with goal accomplished

Method for goal: set-presetStep1: Reach to desired "Preset" buttonStep2: Press and hold "Preset" buttonStep3: Wait until sound returnsStep4: Remove finger from "Preset" buttonStep5: Return with goal accomplished

105George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting perverse (1)

Locate StationSelect AM/FMInitiate seekExit seek

MODE

Set Mode to Radio

etcAssign new PreSETEnter

PreSet MODE

Enter PreSet ButtonExit

PreSet MODE

Goal hierarchy for perverse designPreSET Network ShowDelete old PreSETEnter delete MODE

Enter PreSet Button

Exit delete MODE

Set PreSETEnter seek mode

seek & identify

106George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting perverse (2)

ssEJ

– TUNE +

PUSH CLOCK

ON OFF

FMAM

– +BASS

– +TREBLE

L RBALANCE

L RFADE

2:411 2 3

4 5 6

Radio for Perverse Preset Design

SEEK MODE

ERASE PRESET

MEMORIZE PRESET

ssSEEK

107George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting perverse (3)

MODEL 2: SETTING A PRESET TO A NETWORK SHOW - PERVERSE DESIGN

Method for goal: preset network show (s/a optimal)Step1: Decide: IF radio is not playing, THEN Accomplish goal:

set-mode-to-radioStep2: Decide: IF show N.E. intended show, THEN Accomplish

goal: locate stationStep3: Accomplish goal: set-presetStep4: Return with goal accomplished

Method for goal: locate-station (s/a optimal)Step1: Decide: IF band N.E. desired band, THEN Accomplish

goal: select AM/FMStep2: Accomplish goal: initiate-seekStep3: Return with goal accomplished

Method for goal: select AM/FM (s/a optimal)Step1: Put finger on button for desired bandStep2: Press button for desired bandStep3: Verify band correctStep4: Return with goal accomplished

108George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting perverse (4)Method for goal: initiate-seek

Step1: Accomplish goal setMode-SEEKStep2: Accomplish goal: seek&identifyStep3: Accomplish goal: exit seek modeStep4: Return with goal accomplished

Method for goal: setMode-<mode>Step1: Reach for <mode> buttonStep2: Press <mode> buttonStep3: Verify <mode> correctStep4: Return with goal accomplished

Method for goal: seek&identify (s/a optimal)Step1: Reach to right side of "Seek" button Step2: Press "Seek" buttonStep3: Decide: IF show N.E. intended show, THEN Goto 2Step4: Verify show correctStep5: Return with goal accomplished

Method for goal: set-presetStep1: Accomplish goal: delete-previous-presetStep2: Accomplish goal: assign-new-presetStep3: Return with goal accomplished

109George Mason University

Human Factors & Applied Cognitive Program

NGOMSL-- PreSetting perverse (5)

Method for goal: delete-previous-presetStep1: Accomplish goal: setMode-deletePreSet-on

(setMode-<mode>)Step2: Accomplish goal: enter-preset-buttonStep3: Accomplish-goal: setMode-deletePreSet-off

(setMode-<mode>)Step4: Return with goal accomplished

Method for goal: assign-new-presetStep1: Accomplish goal: setMode-preset-on (setMode-

<mode>)Step2: Accomplish goal: enter-preset-buttonStep3: Accomplish-goal: setMode-preset-off (setMode-

<mode>)Step4: Return with goal accomplished

Method for goal: enter-preset-buttonStep1: Reach to desired "Preset" buttonStep2: Press "Preset" buttonStep3: Return with goal accomplished

110George Mason University

Human Factors & Applied Cognitive Program

NGOMSL--comparison of optimal and perverse designs

Optimal Perversenumber of different methods 6 11

number of methods used 6 18number of different steps 21 38number of steps required 21 66

111George Mason University

Human Factors & Applied Cognitive Program

Break time

112George Mason University

Human Factors & Applied Cognitive Program

CPM-GOMS: The Embodied Cognition level

Scale(sec)

Time Units System Analysis Activities/Processes

World (theory)

105 days

104 hours Task Task analysis Subtasks Bounded

103 10 min Rationality

102 min Subtask Unit Task Analysis Unit Tasks

101 10 sec Unit task Cognitive task analysis Activities

100 1 sec Activity Embodied cognition Microstrategies Cognitive Band

10-1 100 ms Microstrategy Computational modelsof embodied cognition Elements

10-2 10 ms Elements Architectural Parameters Biological

10-3 1 ms Parameters Band

Analyzing activities into microstrategies

113George Mason University

Human Factors & Applied Cognitive Program

CPM-GOMS Extension to GOMS

Critical Path Method (or Cognitive Perceptual Motor) -GOMS

Control structure: Relaxed hierarchy, can be interrupted and continued based on new information in the world

Serial or Parallel: Parallel

Level of Analysis: Primarily elementary perceptual, cognitive and motor operators

114George Mason University

Human Factors & Applied Cognitive Program

CPM-GOMS - why?

Need for analysis suitable for parallel activities

Human cognition is embodied cognition In some cases we need to understand the interleaving and

interdependencies between elementary cognitive, perceptual, and motor operations

115George Mason University

Human Factors & Applied Cognitive Program

Brief Introduction to PERT Charts

Aadapted from "MacProjectII: Getting Started" (Claris Corp., 1989)

Start project

3/15/893/15/89

0

Talk to real estate agents

3/15/893/23/89

7

Get approval from Board of Directors

3/15/894/11/89

20

Visit potential locations

3/24/894/20/89

20

Propose location

4/20/894/20/89

0

Prepare offer

4/21/895/11/89

15

Show site to management

4/21/895/18/89

20

Investigate zoning laws

4/21/895/4/89

10Make

presentation to Committee

5/19/895/19/89

1

116George Mason University

Human Factors & Applied Cognitive Program

SystemOperations

VisualPerceptualOperations Aural

CognitiveOperations

R-hand

L-handMotor Operations Verbal

EyeMovement

Call Arrival Tone

display 0+

Perceive displayPerceive

Tone

attendaural

attend display

initiate eyemovement

verifytone

verify0+

eyemovement

attend display

displayCOIN PRE

Perceive display

System Event System Event

verify COINPRE

Initiategreeting

New England Public Telephonemay I help you?

CustomerPause

Perceive CustomerSpeech

attendaural

customer

initiate functionkeypress

R-home tofunction keyCPM-GOMS Model

CPM-GOMS

Schedule Charts: A Notation for Representing Parallelism.

117George Mason University

Human Factors & Applied Cognitive Program

But are composites of simpler, basic-level activities Activities

Microstrategies

Unit-tasks

We will illustrate these in the context of moving and clicking a mouse

118George Mason University

Human Factors & Applied Cognitive Program

Predicted time:

74 5 msec

attend-

target

50

move-

cursor

545

initiate-

POG

50

POG

30

perceive-

cursor

@target

100

verify-

cursor

@target

50

initiate-

move-

cursor

50

perceive

-target

100

verify-

target -

pos

50

attend-

cursor

@target

50

new-cursor-location

0

Predicted time:

530 msec

attend-

target

50

move-

cursor

301

initiate

-POG

50

POG

30

perceive-

cursor

@target

100

verify-

cursor@

target

5050

perceive

-target

100

verify-

target -

pos

50

attend-

cursor

@target

50

new-cursor-location

0A B

initiate-

move-

cursor

119George Mason University

Human Factors & Applied Cognitive Program

Predicted time: 845 msec

initiate-mouseDn

50

mouseDn

100

SLOW-M/C

at t end-t arget

50

move-cursor

545

new-cursor-locat ion

init iat e-POG

50

POG30

perceive-cursor

@target

100

verify-cursor@

t arget

50

perceive-t arget

100

verify-t arget -

pos

50

at t end-cursor@

t arget

50START

init it at e-move-cursor

END

845

0

Predicted t ime:695 msec

FAST-M/C

69550

mouseDn

10 0

50

54 5

50

POG30

perceive t arget

100

50

0

at t end-t arget

init iat e-POG

verify-t arget -

pos

init iate-mouseDn END

new-cursor-locat ion

move-cursor

START init it at e-

move-cursor

120George Mason University

Human Factors & Applied Cognitive Program

Predicted time: 300 msec

mouseUp100

display-change

perceive-display-change100

verify-display-change

50

initiate-move-cursor

50

SLOW-C/M

at t end-display-change

50

START mouse

Dn

END move-cursor

Predicted t ime: 50 msec

FAST-C/M300

0

50

START mouse

Dn 50

Predicted t ime: 150 msec

mouseUp

100

50

MED-C/M

START mouse

Dn

150

init iat e-move-cursor

init iat e-move-cursor

END move-cursor

END move-cursor

121George Mason University

Human Factors & Applied Cognitive Program

The Button Study

home

?? ?? ??

?? ?? ??

?? ??

122George Mason University

Human Factors & Applied Cognitive Program

123George Mason University

Human Factors & Applied Cognitive Program

124George Mason University

Human Factors & Applied Cognitive Program

Experimental Data

HOME-to-TARGET

TARGET-to-HOME

Difference

Predicted 970 msec 820 msec 150 msecFound 994 msec

(19.7)840 msec(19.7)

154 msec

Absolute Difference 2.41% 2.38%

Data and examples are from: Gray, W. D., & Boehm-Davis, D. A. (in press). Milliseconds Matter: An introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experiment Psychology: Applied.

Advanced copy available from: http://hfac.gmu.edu/~mm

125George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation: The Real-World Problem A telephone company wanted to replace old

telephone operator workstations with a new workstation.

For this company, each second saved per call translates into a savings of $3 million/year in operating costs.

Will the new workstation be faster than the current workstation, and if so, how much will it save each year?

126George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation: The Task

Assist a customer in completing calls

Record the correct billing information

NOT directory assistance

127George Mason University

Human Factors & Applied Cognitive Program

Workstation: “Beep”TAO: “New England Telephone may I help you?”Customer: “Operator, bill this to 412-555-1212-1234.”TAO: Hits a series of function & numeric keysWorkstation: Displays the authorization for the calling cardTAO: “Thank you.”TAO: Hits a key to release the workstation so

another call can come in

Telephone Operator Workstation: An Example Call

128George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation: Goals

Overall Goal Handle the call

Subgoals explicitly trained Determine and enter who pays for the call

Determine and enter the appropriate billing rate

Determine when the call is complete and release the workstation

129George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation: Operators

Criteria for selecting a level of analysis Captures observed behavior

Captures distinctions between workstations

Different levels of analysis Unit Task Level

Functional Level

Cognitive task analysis Level

Embodied cognition Level

130George Mason University

Human Factors & Applied Cognitive Program

Operators reflect the activities theTAO must perform

• listen-to-beep• read-screen• greet-customer• listen-to-customer• enter-command• enter-credit-card-number• thank-customer

Telephone Operator WorkstationOperators: The cognitive task analysis level

131George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation:Cognitive Task Analysis

goal: handle-calls. goal: handle-call. . goal: initiate-call. . . goal: receive-information. . . . listen-to-beep Workstation: beep. . . . read-screen Workstation: displays information. . . goal: request-information. . . . greet-customer TAO: "New England Telephone

may I help you?". . goal: enter-who-pays. . . goal: receive-information. . . . listen-to-customer Customer: “Operator, bill this to

412-555-1212-1234.”. . . goal: enter-information. . . . enter-command TAO: hit F1 key. . . . enter-calling-card-number TAO: hit 14 numeric keys. . goal: enter-billing-rate. . . goal: receive-information. . . . read-screen and so on

132George Mason University

Human Factors & Applied Cognitive Program

System RT

LISTEN-TO-BEEP

READ-SCREEN

GREET-CUSTOMER

LISTEN-TO-CUSTOMER

ENTER-COMMAND

ENTER-CREDIT-

THANK-CUSTOMER

Time of call (seconds)

CARD-NUMBER

0 13

Telephone Operator Workstation:Cognitive Task AnalysisProblems with Assumption of Sequential Operators

133George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation: Embodied Cognition Level

Operators at the level of elementary operations of

Cognition

Perception

Motor acts

Uses the Critical Path Method

134George Mason University

Human Factors & Applied Cognitive Program

READ-SCREEN(1) activity-level goal

System RT

Perceptual Operators

Visual

Aural

Cognitive Operators

L-Hand Movements R-Hand Movements Verbal Responses Eye Movements

Motor Operators

400

100

eye-movement (1)

30

system-rt(2)330

display-info(2)30

100

150

other systems

workstation display time

LISTEN-FOR-BEEP activity-level goal READ-SCREEN(2)

activity-level goal

initiate-eye-movement(1)

GREET-CUSTOMER activity-level goal

50attend-info(1)

0

attend-info(2)

perceive-binary-info(2)

50 50 50 50 50 50initiate-greeting

1570"New England Telephone

may I help you?"

perceive-silence(a)

verify-info(2)

verify-info(1)

verify-BEEP

50attend-aural-

BEEP

290perceive-complex-info(1)

30display-info(1)begin-call

system-rt(1)

perceive-BEEP

Telephone Operator Workstation:CPM-GOMS Level

135George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation:Making Quantitative Predictions Use the CPM-GOMS model of the benchmark call on

the current workstation as a baseline

Modify the model to reflect design decisions substitute the layout, response times and procedures of the

proposed workstation

propose new features or designs

obtain quantitative predictions of the effects on performance time

136George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation:Quantitative Predictions

Call Category

Seconds

-0.5

0

0.5

1

1.5

2

cc01 cc02 cc03 cc04 cc05 cc06 cc07 cc08 cc09 cc10 cc11 cc12 cc13 cc16 cc18

Difference between the proposed workstation and the current workstation for 15 benchmark calls

137George Mason University

Human Factors & Applied Cognitive Program

Telephone Operator Workstation:Qualitative Explanations

Examine the critical path of the different models to explain the quantitative predictions. E.g., why this call is longer on the proposed than on the current workstation.

Beginning of call

Current Workstation

Proposed Workstation

operators removed from slack time

138George Mason University

Human Factors & Applied Cognitive Program

Qualitative Explanations

Models as explanation.

Current Workstation

Proposed Workstation

operators added to critical path

End of call

139George Mason University

Human Factors & Applied Cognitive Program

Uses of CPM-GOMS in Design

Project Ernestine emphasized Quantitative comparisons between alternative systems

Qualitative explanations for differences between alternative systems

140George Mason University

Human Factors & Applied Cognitive Program

Uses of CPM-GOMS in Design

CPM-GOMS can also be used to Direct design effort

Bracketing, Profiling, and Diagnosis

141George Mason University

Human Factors & Applied Cognitive Program

Directing Design Effort

Designing telephone operator workstations: Should we redesign

keyboards?

screens?

system response times?

142George Mason University

Human Factors & Applied Cognitive Program

Directing Design Effort

Keyboards & Screens

How much of the time is each

activity on the critical path?

Call Cat. Sys RT Talking Keying Reading Ring/Coincc01 25% 40% 1% 3% 31%cc02 3% 93% 0% 4% 0%cc03 20% 71% 2% 6% 0%cco4 25% 31% 6% 4% 35%cc05 19% 44% 3% 2% 22%cc06 12% 79% 2% 6% 0%cc07 30% 57% 8% 3% 0%cc08 26% 41% 23% 10% 0%

... ... ... ... ... ...Average 16% 64% 6% 5% 8%

143George Mason University

Human Factors & Applied Cognitive Program

Directing Design Effort

Bracketing

Profiling

Diagnosis

145George Mason University

Human Factors & Applied Cognitive Program

Bracketing: Slow & Fastest ReasonableDescription Microstrategy SLOW BEST FIT FASTEST

REASONABLE

Part 1: K3 to mDn@zoomPtfrom keypress to move cursor KEYPRESS-

MOVESLOW-KP/M P1 FAST-KP/M

P2 MED-KP/MFAST-KP/M

mouse down on zoom-point MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

Part 2: mDn@zoomPt to mDn@EXP2 btnmove from zoom-point to EXP2 button CLICK-MOVE SLOW-C/M P1 SLOW-C/M

P2 MED-C/MFAST-C/M

mouse down on EXP2 button MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

Part 3: mDn @ EXP2 btn to mDn @ trgtmove from EXP2 button to way-point CLICK-MOVE SLOW-C/M P1 MED-C/M

P2 SLOW-C/MMED-C/M

mouse down on way-point MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

Part 4: mDn @ trgt to K2keypress to select NAV-his display CLICK-

KEYPRESSSLOW-C/KP P1 SLOW-C/KP

P2 MED-C/KPFAST-C/KP

Part 5: K2 to mDn @ NavDesg Btnmove from center to NAVDESG button KEYPRESS-

MOVESLOW-KP/M P1 SLOW-KP/M

P2 MED-KP/MFAST-KP/M

mouse down on NAVDESG button MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

146George Mason University

Human Factors & Applied Cognitive Program

Bracketing

147George Mason University

Human Factors & Applied Cognitive Program

Profiling: BEST FIT modelsDescription Microstrategy SLOW BEST FIT FASTEST

REASONABLE

Part 1: K3 to mDn@zoomPtfrom keypress to move cursor KEYPRESS-

MOVESLOW-KP/M P1 FAST-KP/M

P2 MED-KP/MFAST-KP/M

mouse down on zoom-point MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

Part 2: mDn@zoomPt to mDn@EXP2 btnmove from zoom-point to EXP2 button CLICK-MOVE SLOW-C/M P1 SLOW-C/M

P2 MED-C/MFAST-C/M

mouse down on EXP2 button MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

Part 3: mDn @ EXP2 btn to mDn @ trgtmove from EXP2 button to way-point CLICK-MOVE SLOW-C/M P1 MED-C/M

P2 SLOW-C/MMED-C/M

mouse down on way-point MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

Part 4: mDn @ trgt to K2keypress to select NAV-his display CLICK-

KEYPRESSSLOW-C/KP P1 SLOW-C/KP

P2 MED-C/KPFAST-C/KP

Part 5: K2 to mDn @ NavDesg Btnmove from center to NAVDESG button KEYPRESS-

MOVESLOW-KP/M P1 SLOW-KP/M

P2 MED-KP/MFAST-KP/M

mouse down on NAVDESG button MOVE-CLICK SLOW-M/C SLOW-M/C SLOW-M/C

148George Mason University

Human Factors & Applied Cognitive Program

Profiling

149George Mason University

Human Factors & Applied Cognitive Program

Diagnosis: example

CLICK-MOVE Used in part 2 and part 3

Use of slower microstrategies may reflect a confusion among the three unit tasks

Not perceptually distinct

Participants might have had to rely more on memory than is usual for interactive behavior with current interactive technology

150George Mason University

Human Factors & Applied Cognitive Program

Summary of CPM-GOMS

Predicts expert performance time and provides qualitative explanations for tasks involving parallel activities

Once the models are built, schedule charts allow designers to rapidly play what-if games with design ideas and parameters

Not intended to predict the occurrence of learning time, or errors - other forms of GOMS models are more helpful there

Subject to the limitations of GOMS models in general (no casual use, fatigue, etc.)

151George Mason University

Human Factors & Applied Cognitive Program

Summary of Workshop

Intro to the GOMS family of models

Keystroke Level GOMS

NGOMSL GOMS

CPM-GOMS

http://hfac.gmu.edu/~graypubshttp://hfac.gmu.edu/~graypubs

http://hfac.gmu.edu/~mmhttp://hfac.gmu.edu/~mm

152George Mason University

Human Factors & Applied Cognitive Program

153George Mason University

Human Factors & Applied Cognitive Program

154George Mason University

Human Factors & Applied Cognitive Program