Interaction modalities

Command languages, direct manipulation, and WIMP

Midterm – date change?

Monday, Oct. 6

What’s wrong with this?

0. Borrow a book from the library 1. go to the library 2. log in to library catalogue

2.1 access the search screen2.2 enter search criteria2.3 identify required book 2.4 note location

3. go to correct shelf and retrieve book3.1 use library map to find shelf

4. take book to checkout counter

Now what’s wrong?

0. In order to borrow a book from the library 1. go to the library 2. if you don’t know where the book is, find the required book

2.1 access library catalogue2.2 access the search screen2.3 enter search criteria2.4 identify required book 2.5 note location

3. go to correct shelf and retrieve book4. check out book

Don’t forget your plans!

0. In order to borrow a book from the library 1. go to the library 2. find the required book

2.1 access library catalogue2.2 access the search screen2.3 enter search criteria2.4 identify required book 2.5 note location

3. go to correct shelf and retrieve book4. check out book

Plan 0: do 1, 2, 3; repeat 2, 3, if book not found; then do 4Plan 2: do 2.1, 2.2; repeat 2.3 until found; 2.4, 2.5

Interface types

1980s interfaces

Command line

WIMP/GUI 1990s interfaces

Advanced graphical (multimedia, virtual reality, information visualization)

Web Speech (voice) Pen, gesture, and touch Appliance

2000s interfaces

Mobile

MultimodalShareable Tangible Augmented and mixed reality Wearable Robotic

Interaction Styles

1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/Natural language 5. Gesture & pen

General Issues in Interaction Style

Who is in control - user or computer Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Command Languages

Earliest UI interaction paradigm Examples: MS-DOS shell, UNIX, Linux Little or nothing is visible so…

– Work primarily by recall, not recognition– Heavy memory load

Poor choice for novices but...

CL Advantages

Advantages for experts– Speed, conciseness

% ls (hard to beat)

– Can express actions beyond a limited set Flags, piping one command to another

– Repetition, extensibility Scripting, macros

– Easier implementation, less overhead– Power

Abstraction, wild cards

CL Dangers

With added power, comes added responsibility and danger– UNIX

% rm -r * Deletes every file that you have, and you can’t get them

back

CL Design Goals

Consistency– Have options and arguments expressed the

same way everywhere Good naming and abbreviations

UNIX fails here because commands were developed by lots of different people at different organizations

No guidelines provided

Names and Abbreviations

Specificity versus Generality– General words

More familiar, easier to accept– Specific (typically better)

More descriptive, meaningful, distinctive– (Nonsense does surprisingly well in small set)

Abbrevs. allow for faster actions– Expert performance begins to be dominated by motor times

such as # of keystrokes– Not good idea for novices– (Allow but don’t require)

General Issues - CL

Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Interaction Styles

1. Command language 2. Direct manipulation 3. WIMP 4. Speech/Natural language 5. Gesture, pen, VR

Direct Manipulation Essence

Representation of reality that can be manipulated

The user is able to apply intellect directly to the task

The tool itself seems to disappear

Direct Manipulation

Definition: 1) Continuous visibility of the objects and

actions of interest 2) Rapid, reversible, incremental actions

whose effect is immediately noticeable 3) Replacement of command language

syntax by direct manipulation of object of interest (physical actions, buttons, etc.)

Shneiderman ‘82

Direct Manipulation

Examples– WYSIWYG editors and word processors– VISICALC - 1st electronic spreadsheet– CAD– Desktop metaphor– Video games

Example: maps

More Psychological View

What is directness? (not always done well) Related to two things:

– Distance

– Engagement Unobtrusive and responsive

Hutchins, Hollan, Norman ‘86

Goals System

Execution

Evaluation

Example: Homefinder

DM Advantages

Easier to learn & remember, particularly for novices

Flexible, easily reversible actions helps reduce anxiety in users

Provides context & instant visual feedback so user can tell if objectives are being achieved

Exploits human use of visual spatial cues Limits types of errors that can be made

DM Problems

Screen space intensive (info not all that dense)

Need to learn meaning of components of visual representation

Visual representation may be misleading Mouse ops may be slower than typing Not self-explanatory (no prompts) Not everything can be represented visually

DM Problems

Not good at– Repetition– History keeping (harder)– Certain tasks (Change all italics to bold)– Abstract elements (variables)– Macros harder

General Issues - DM

Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Dialog Design

1. Command language 2. Direct manipulation 3. WIMP 4. Speech/Natural language 5. Gesture, pen, VR

WIMP

Windows, Icons, Menus, Pointers– Focus: Menus, Buttons, Forms

Predominant interface paradigm now (with some direct manipulation added)

Advantages:– ?

Menus

Advantages:– 1 keystroke or mouse operation vs. many– No memorization of commands– Limited input set

Disadvantages:– Less direct user control - have to find correct menu / menu

item– Not so readily extensible– Slower than keyboarding for experienced users, at least

without accelerators

Menu Items

Various types:– Flat– Cascading– Pop-up (contextual)

Organization strategies– Create groups of logically similar items– Cover all possibilities– Ensure that items are non-overlapping– Keep wording concise, understandable

A Good Example

Logical grouping Visual separation of

groups Disabled items “grayed

out” Shortcuts shown … indicates leads to

dialogue

Presentation Sequence

Forms, dialogue boxes, menus Use natural if available

– Time e.g. Breakfast, Lunch, Dinner

– Numeric ordering e.g. Point sizes for font

Other possibilities:– Alphabetical– Group related items– Frequently used first– Most important first

Pointers

Allows users to specify location parameter of a command

– Provides for some of that direct manipulation– Put this file HERE

Also displays system state info:– Tracking– Busy– Hints– Modes (text cursor vs. selection pointer)

Other WIMP issues

Windows management– How to locate, move, find– Transfer information between– Switch attention between

Icons– Need graphic design attention

Toolbars

Questions

How many windows do you generally have open at once?

How do you arrange them? How do you switch between them?

Newer icons

Simple icons

Simple icons

General Issues - WIMP

Who is in control - user or computer Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Dialog Design: next time

1. Command language 2. Direct manipulation 3. WIMP 4. Speech/Natural language 5. Gesture, pen, VR