1

11/25/01 CS4: HCI lecture 13 1

Interface Design V: Beyond the Desktop

Rob Procter

Further Reading

•Dix et al., chapter 4, p. 153-161 and chapter 15.

•Norman, The Invisible Computer, MIT Press, 1998, chapters 4 and 15.

11/25/01 CS4: HCI lecture 13 2

Introduction: Interactional Styles and Paradigms

• Command Line

• Desktop

• Speech

• Virtual Reality

• Assistant and agent-based Interaction

• Ubiquitous Computing

2

11/25/01 CS4: HCI lecture 13 3

Desktop Metaphor

• The scalability problem– Early GUIs worked:

• Making everything visible worked: not much to be visible.• Learning by doing worked: not much to try out.

– But now:• Desktop RAM has increased 103-fold.• Desktop disk space has increased 105-fold.• The entire Web is only a click away.

– It is not possible to make everything visible}• All at once = chaos• Not all at once = things get lost

11/25/01 CS4: HCI lecture 13 4

Desktop Metaphor

• Is the desktop metaphor broken?

• Perhaps the main problem is ‘slavish adherence’ to the predominant metaphor

• There are alternatives:– Some for standard PCs

• Lifestreams historical metaphor

• Adding another dimension

– Some for smaller devices• Newton OS notepad metaphor

3

11/25/01 CS4: HCI lecture 13 5

Lifestreams

11/25/01 CS4: HCI lecture 13 6

Adding Another Dimension: Pad++

4

11/25/01 CS4: HCI lecture 13 7

Norman’s Diagnosis

• The difficulty goes deeper• WIMP and DM try, and fail, to conceal the sheer

complexity and scale of the system beneath them• Why is the desktop PC so complex?

– One device, many tasks– One device, many people

• Each demands the incremental addition of features• So far, the solution has been to fit ever more complex

help and support services

11/25/01 CS4: HCI lecture 13 8

Alternatives to the Desktop• Speech interfaces

– Return to conversational interfaces– Very natural

• Virtual reality– Better state display --- only visible state?– Better object manipulation --- data gloves

• Intelligent interface assistants and agents– Conversational– Autonomous and adaptive

• Ubiquitous computing– Better object manipulation --- two hands– Wearable computers– Context sensitive, mixed reality

5

11/25/01 CS4: HCI lecture 13 9

Speech Interfaces• Distinguish speech recognition from speech synthesis• Speech recognition systems already work:

– Dictation, command and control, inventory

• Useful whenever user cannot type:– No space for keyboard– hands busy, eyes busy (e.g: operating, driving)– low typing skills– motor disability– sensory disability

• Note naturalness of dialogue for feedback

11/25/01 CS4: HCI lecture 13 10

Problems with Speech Interfaces• Speech recognition doesn’ t equate to robust user

understanding– For language understanding:

• shared assumptions, deep knowledge required

– For robust speech recognition:• noise must be ignored via selective focus of attention

• Furthermore: for task success, talk is not enough– Some tasks have single, known solutions -- but cannot be

described in words:• E.g.: “ tie your shoelace” '

– Other tasks are open-ended, and have no prior solution:• E.g.: “check the manuscript for mistakes” '

• Naturalness of interface is misleading: language understanding isn't mind-reading

6

11/25/01 CS4: HCI lecture 13 11

Agents

• Learning proceeds by:– Observing and imitating user

– Receiving positive and negative feedback from user

– Receiving explicit instructions

• It is very difficult to infer user intentions accurately from a finite set of data– Sometimes agents help, but are often wrong,

which is worse than no help at all

What happened to the paper clip?

11/25/01 CS4: HCI lecture 13 12

Virtual Reality Interfaces• Virtual reality, interface as immersion in a computed world,

represents natural development of interface as shared object concept

• It is claimed that “people are visual animals”• For dealing with information overload, more content can be

packed onto the desktop by using 3D rendering• For training and teleoperation purposes, accurate visual

simulations of real or imagined situations can be:– very useful– cheaper than any alternative

• Interface to VR world can use vision, touch, speech• Avatars -- virtual inhabitants

Seonaid

7

11/25/01 CS4: HCI lecture 13 13

VR Example: A Museum

11/25/01 CS4: HCI lecture 13 14

Problems with VR• We do navigate well in space

– But not perfectly; much individual difference

• In desktop VR, the (picture of the) world moves; we don’ t:– We experience movement through prediction and

proprioception, as well as through vision– In reality, vision and proprioception usually match

• Problems when:– Lack of motion control; receptors non-functional; zero-g

• Desktop VR susceptible to this mismatch

8

11/25/01 CS4: HCI lecture 13 15

Ubiquitous Computing

• Anything that lets people escape their workstations:– Cellphones, PDAs, smart clothing, big screens, handheld and wearable

devices, wirelessly networked to the bigger infrastructure.

• Explores ways of embedding digital systems into real spaces and objects, making them literally tangible

• Automatic capture of everyday experience– capture, integration, access

• Context-aware computing– mostly physical location only

• Seamless services– unobtrusive computing

11/25/01 CS4: HCI lecture 13 16

Ubiquitous Computing Concepts

9

11/25/01 CS4: HCI lecture 13 17

Ubiquitous Computing: Tangible Bits

11/25/01 CS4: HCI lecture 13 18

Ubiquitous Computing: Tangible Bits

A positioning device

10

11/25/01 CS4: HCI lecture 13 19

Ubiquitous Computing: Tangible Bits

A measuring device

11/25/01 CS4: HCI lecture 13 20

Ubiquitous Computing: PDA with GPS

11

11/25/01 CS4: HCI lecture 13 21

Mixed RealitiesBy tracking a person (& PDA) in a room, the person is also in a VR model of the same room. A map on the PDA shows VR avatars, and audio also links the two rooms.

11/25/01 CS4: HCI lecture 13 22

Making Do With Small Interfaces

• The device shrinks; the interface shrinks• Replace keyboard, pointer and screen with:

– a small, bad keyboard, pointer and screen– speech recognition and synthesis– pen-based input and handwriting or gesture recognition– spectacle-mounted screens and speech input

• If the device itself becomes less complex, good:– Compare the Newton

• high functionality … discontinued

– With the Palm Pilot• low functionality … selling well

12

11/25/01 CS4: HCI lecture 13 23

Making Do With Small Interfaces

• Don Norman’s solution:– Information appliances are the true alternative– Many, specialised systems are better than few

general ones• So long as they communicate with each other

“Devices that are easy to use, not only because they will be inherently simpler, but because they fit the task so well that to learn the task is to learn the appliance.”

11/25/01 CS4: HCI lecture 13 24

Guidelines for Small Devices

• Use(fulness) immediately apparent

• Structure interface to task

• Short cuts and flexibility

• Minimise memory load

• Use consistent screen templates

• Provide a Back function on every screen

• Selection is better than writing

13

11/25/01 CS4: HCI lecture 13 25

Some Examples