To Err is HumanComputational Limits to Human Thinking :

Implications for the Design of Human Centered Interfaces

Raj ReddyCarnegie Mellon University

Pittsburgh, PA 15213January 21, 2010

www.rr.cs.cmu.edu

Outline of Talk

· Limitations of Current Interfaces from the Perspective of Cognitive Science

· Missing Science of Multimedia Interfaces

· Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

Outline of Talk

· Limitations of Current Interfaces from the Perspective of Cognitive Science

· Missing Science of Multimedia Interfaces

· Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

Human Strengths and Human LimitationsLessons from Cognitive Science

· Strengths: People· communicate using speech and natural language· tolerate errorful, ambiguous and imprecise input· exploit vast amounts of knowledge· learn from the environment

· Limitations: People· make errors· tend to forget· become impatient· tend to be lazy

To Err Is Human• Problem:

– Catastrophic loss of data: “I didn’t mean to do that ”– Unanticipated side-effects: “how did that happen??”

• Causes:– Sensory, Cognitive and Motor overload– Information Overload: “Like being in a traffic jam”– Timing Errors: Simplify the task

• Present Solutions:– Partial Undo– Dialog Boxes

• Future Opportunities:• Unlimited Undo• Anytime Anywhere Abort• DWIM (Do What I Mean)

To Forget Is Human• Problem:

– A non-expert occasional user can’t be expected to remember the details

• Causes: – Forgetting is the loss of indexing structure– Redundancy in the indexing structure is the key

• Present Solutions:– Recognition vs. recall: GUIs and Menus– On-line manuals

• Future Opportunities:• Use of color, fonts, voice responses for focusing attention• Intelligent Help

• “How do I” and “What if” MultiMedia documentation• “Reference librarian” agent

To be Impatient Is Human• Problem:

– Time to get the answer in interactive problem solving• Long Start-up and Shutdown Times• Network operations: Unpredictable retrieval and browsing times

• Present Solutions:– Hour glasses and wheels

• No idea how long it will take – Progress bars

• Future Opportunities:• Updates in the background • Learn from experience: self-aware systems• Look ahead retrieval and computation• Hurry-up algorithms• Keystroke model

To be Lazy Is Human• Problem:

– Most people use a minimal subset of functionality in Word, PowerPoint, etc.

– Most people avoid tasks requiring too much cognitive effort• PGP - too much work• FTP - too complex

• Causes– Principle of Least Effort

• Future opportunities:• Advice giving agents that look over your shoulder• Just-in-time learning• Gentle slope systems: Easy Things should be Easy to Do• Agents (wizards!) that know about PGP, FTP, or whatever

Outline of Talk

· Limitations of Current Interfaces from the Perspective of Cognitive Science

· Missing Science of Multimedia Interfaces

· Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

Missing Science of Multimedia InterfacesLessons from Flash Forward Video

• SILKy Interfaces• Speech, Image, Language and Knowledge

• Forgiving Interfaces: Tolerate error and ambiguity– “You Bet” --- “Did That Mean Yes?”

• Self Improving Interfaces: Incremental non-intrusive knowledge acquisition• “Darn” --- “Would you Care to Define It?”

• Intelligent Help, Advice and Tutoring– “ Enthusiasm Detected --- Care to View Large Screen?”

• Intelligent Agents– Mail Serve: Junk Mail– Communications: Telephone Tag– Intelligent Tutoring Tools– Agent Creation Technology

Outline of Talk

· Limitations of Current Interfaces from the Perspective of Cognitive Science

· Missing Science of Multimedia Interfaces

· Interfaces for Enabling Access to the Five Billion people at the Bottom of the Pyramid

User Interfaces for Rural Environments

• Low cost PC: target of less than $100• A Multifunction Information Appliance: PCtvt

– PCtvt: TV, PVR, Video Phone, IP Phone and PC

• PCs for consumption, not creation – For Most People in a Village, Entertainment and

Communication are of Greater Importance than PC Functionality

PCtvt UI Design for Use by Illiterate Persons

• An Illiterate person needs a more powerful PC than a Microsoft employee– If not e-mail, use voice-mail– Replace Text Help by Video Help

• Radically simple design– One minute learning time– Two click model– Three modes of communication: Video, Audio and Text

– Both Synchronous and Asynchronous • All-Iconic interfaces• Multiple input modalities

– TV-remote, Speech I/O, Keyboard, Mouse or Cell phone

Mapping TV Remote to Iconic Screen Layout

OK

1 2 3

4 5 6

7 8 9

BACK 0 MENU 9

OK

1 2 3

4 6

7 9

TV Menu Video Phone Menu

OKOK

2 3

4 5 6

7 9

0

The Unfinished Research Agenda

· Add Features to Assist in Overcoming Human Limitations such as “Unlimited Undo”, Instant On and Off, and Multimedia Help.

· WIMPy to SILKy: Move Towards Intelligent Multimedia Interfaces

· Eliminate Language Divide and Literacy Divide in Future Systems: Interfaces that Enable Access to People at the Bottom of the Pyramid