augmented large scale cognition · 2015. 1. 21. · augmented large scale cognition stuart card...

AUGMENTED LARGE SCALE COGNITION

Stuart Card Palo Alto Research Center (PARC) (Visiting Professor, Stanford)

Stanford Computer Forum Stanford University April 15, 2009

ENGINEERING HUMAN PROSTHESES

HOW DO WE DESIGN COGNITIVE PROSTHESES?

? Meg Stewart

Paul MacCready

Motivating Problem: Large-Scale Cognition

7.3 million pages/day

Comparisons

1999 2009 (EB) (EB)

  Unique Info 0.6 36.0   Store in earth’s population

memory in 1 yr 0.1 0.1   Record all words in all

lives 3.6 3.6

Source: Berkeley SIMS + Computations

Man the informavore (George Miller, 1983)

  Informavores   Hunger for information about

the world   Use information to adapt to the

world

COGNITION

  Find . . . (Perceive, Learn, Remember)   Think . . . (Decide)   Do . . . (Create, Act)

LEVELS OF COGNITION

  Immediate behavior   Experiential cognition   Routine cognitive skill

LEVELS OF COGNITION

  Problem Solving   Reflective cognition

LEVELS OF COGNITION

  Social cognition   People finding, thinking, and doing

together

TIME LEVELS OF BEHAVIOR

107 (months) SOCIAL Social Behavior 106 (weeks) 105 (days) 104 (hours) RATIONAL Problem Solving 103 102 (minutes) 101 COGNITIVE Immediate Behavior 100 (seconds) 10-1 10-2 BIOLOGICAL Automatic Behavior 10-3 (msec) 10-4

IMMEDIATE BEHAVIOR

107 (months) SOCIAL Social Behavior 106 (weeks) 105 (days) 104 (hours) RATIONAL Problem Solving 103 102 (minutes) 101 COGNITIVE Immediate Behavior 100 (seconds) 10-1 10-2 BIOLOGICAL 10-3 (msec) 10-4

COGNITIVE LEVEL (Immediate behavior)   Minimize mental time and motion)

MODEL HUMAN PROCESSOR

  Processors and Memories applied to human

  Used for routine cognitive skill

EXAMPLE: ZERO-PARAMETER CALC

  Problem: Inventor claims he invented 600 wpm typewriter. License and develop?

  Solution 1: Half stroke: τM = 70 ms/char Whole stroke: τM + τM = 140 ms/char

but if between hands, overlap: τM = 70 ms = 131 words/min

EXAMPLE: ZERO-PARAMETER CALC

  Solution 2: (range calculation) Half stroke: τM=70 [30~100] ms/char = 131 [308~92] words/min

  Conclusion: Bogus claim. Throw him

out!

TASK ANALYSIS: GOMS (GOALS, OPERATORS, METHODS, SELECTION RULES)

GOAL: EDIT-MANUSCRIPT • repeat until done GOAL: EDIT-UNIT-TASK

GOAL: ACQUIRE-UNIT-TASK • if not remembered GET-NEXT-PAGE • if at end of page GET-NEXT-TASK • if an edit task found

GOAL: EXECUTE-UNIT-TASK GOAL: LOCATE-LINE • if task not on line

[select : USE-QS-METHOD USE-LF-METHOD]

GOAL: MODIFY-TEXT [select USE-S-COMMAND

USE-M-COMMAND]

task analysis

PREDICTS TIME WITHIN ABOUT 20%

SAE RECOMMENDED PRACTICE J2365

  Predict task times for car navigation systems

  Check compliance with SAE J2364 (15-Second Rule)

  Note: To estimate times while driving, multiply by 1.3 to 1.5.

  Based on GOMS and work by Paul Green at Univ. of Michigan Transportation Research Institute.

Dario Salvucci

SAE J2365 OPERATOR TIMES Time (s)

Code Name Young (18-30

Old (55-60)

Rn Reach near 0.31 0.53 Rf Reach far 0.45 0.77 C1 Cursor once 0.80 1.36 C2 Cursor 2 times or more 0.40 0.68 L1 Letter or space 1 1.00 1.70 L2 Letter or space 2 times or more 0.50 0.85 N1 Number once 0.90 1.53 N2 Number 2 times or more 0.45 0.77 E Enter 1.20 2.04 F Function keys or shift 1.20 2.04 M Mental 1.50 2.55 S Search 2.30 3.91 Rs Response time of system-scroll 0.00 0.00 Rm Response time of system-new menu 0.50 0.50 Paul Green UMITRI

COGNITIVE LEVEL (Immediate behavior)   Minimize mental time and motion (Efficient communication)   Input on output

INTERACTIVE COMPUTING

  typewriter I/O  Graphical CRT

Whirlwind (MIT)

DIRECT MANIPULATION

Sketchpad (Sutherland, 1963)

  Input on Output

J. C. R. LICKLIDER

(Cognitive Prosthetic) HUMAN-MACHINE SYMBIOSIS: “The hope is that in not too many years, human brains and computing machines will be coupled together very tightly, and that the resulting partnership will think as no human brain ever thought.”

THREE THEMES FOR LARGE SCALE COGNITION

Efficient Communication Tight Coupling

Representation Shift

LOOK AT COMBINATORICS OF COGNITIVE LEVELS X THEMES

PROBLEM SOLVING


Meg Stewart

COGNITIVE LEVEL (Immediate behavior)   Minimize mental time and motion (Efficient communication)   Input on output (Representation shift) RATIONAL LEVEL (Problem solving)   Maximize (information gain)/(time cost) (Efficient search)

IMMEDIATE BEHAVIOR

  Routine cognitive skill   Well-known path

Information Search

  Problem solving   Heuristic search   Exponential if

don’t know what to do

PHASE TRANSITION IN NAVIGATION COSTS AS FUNCTION OF INFORMATION SCENT

Notes: Average branching factor = 10 Depth = 10

0 2 4 6 8 10 0

50

100

150

Depth

Num

ber o

f pag

es v

isite

d

.100

.125

.150

0 2 4 6 8 10 0

50

100

150

.100

.150

Probability of choosing wrong link (f)

0 0.05 0.1 0.15 0.2 0

20

40

60

80

100

f

Num

ber o

f Pag

es V

isite

d pe

r Lev

el

Linear Exponential

OPTIMALITY THEORY

Max Useful info

Time Max Energy

Time [ ] [ ]

Optimal Foraging Theory Information Foraging Theory

Information Foraging Theory:

patchWithinpatchBetweenWB TTGain

TTGR

−− +=

+=

COGNITIVE LEVEL (Immediate behavior)   Minimize mental time and motion (Efficient communication)   Input on output (Representation shift) RATIONAL LEVEL (Problem solving)   Maximize (information gain)/(time cost) (Efficient search)   Information scent (by design) (Efficient semantic

search)

WITHIN-PATCH ENRICHMENT: INFORMATION SCENT

Tokyo

San Francisco

New York

perception of value and cost of a path to a source based on proximal cues

Boosting Information Scent

IMPORTANCE FOR WEB DESIGN

Jarad Spool, UIE


search)   Information scent (by machine) (Semantic coupling)

RELEVANCE-ENHANCED THUMBNAILS   Emphasize text that

is relevant to query   Text callouts

  Enlarge text that might be helpful in assessing page   Enlarge headers

020406080100120140160180

Picture Homepage E-commerce Side-effects

Tota

l Sea

rch

Tim

e (s

)

Text Plain Enhanced

Allison Woodruff

MACHINE MODELING OF INFORMATION SCENT

cell

patient

dose

beam

new

medical

treatments

procedures

Information Goal Link Text

PREDICTION OF LINK CHOICE

R2 = 0.72

0

5

10

15

20

25

30

35

0 5 10 15 20 25 30 35 Observed frequency

Pre

dict

ed fr

eque

ncy

R2 = 0.90

0

10

20

30

40

50

0 10 20 30 40 50 Observed frequency

Pre

dict

ed fr

eque

ncy

(a) ParcWeb (b) Yahoo

Piroli, PARC

BLOODHOUND PROJECT

Starting Point: www.xerox.com Task: look for “high end copiers”

OUTPUT usability metrics

INPUT

Chi, et al

Smart Book Semantic Index and Scent Highlighting Aids the analyst in finding the most

relevant information quickly.

Information Scent


search)   Information scent (by machine) (Semantic coupling)   Information visualization (Visual representation

shift)

VISUALIZATION   Re-represent to see patterns (eg.,

amphidromic points).

Ray

MACRO-MICRO READING: 6M POINTS

VISUALIZATION REFERENCE MODEL

Human InteractionHuman Interaction

VisualVisualMappingsMappings

VisualVisualStructuresStructures

ViewViewTransformationsTransformations

Visual FormVisual Form

ViewsViews

Raw Data: idiosyncratic formatsData Tables: relations (cases by variables) + meta-dataVisual Structures: spatial substrates + marks + graphical propertiesViews: graphical parameters (position, scaling, clipping, …)

RawRawDataData

DataDataTransformationsTransformations

DataData

DataDataTablesTables

TaskTask



shift)   Input on data (Data coupling)

INTERACTIVE






ViewsViews


RawRawDataData


DataData


TaskTask

Dynamic Queries Magic Lens Overview + Detail Linking & Brushing Extraction & Comparison Attribute Explorer

DYNAMIC QUERIES

Home Finder (U. Maryland)



shift)   Input on data (Data coupling)   Attention-reactive displays (Implicit coupling)

ATTENTION REACTIVE






ViewsViews


RawRawDataData


DataData


TaskTask

ATTENTION-REACTIVE

Degree-of-Interest Trees



shift)   Input on data (Data coupling)   Attention-reactive displays (Implicit coupling)   Sensemaking tools (Representation shift)

SENSE MAKING TASKS   Characteristics

  Massive amounts of data   Ill-structured task   Organization,

interpretation, insight needed

  Output, decision, solution required

  Examples   Understanding a health

problem and making a medical decision

  Buying a new laptop   Weather forecasting   Producing an intelligence

report

IMPORTANCE OF SENSE MAKING

  75% of “significant tasks” on the Web are more than simple “finding” of information (Morrison et al., 2001)   Understanding a topic (e.g., about health)   Comparing/choosing products

  Information retrieval does not support these tasks (Bhavnani et al., 2002)   E.g., Estimated that one must visit 25 Web

pages in order to read about 12 basic concepts about skin cancer

SENSEMAKING

SHOEBOX

EVIDENCE FILE

Search & Filter

Read & Extract

Schematize

Build Case

Tell Story

Search for Information

Search for Relations

Search for Evidence

Search for Support

Reevaluate

TIME or EFFORT

STRU

CTUR

E SCHEMAS

HYPOTHESES

PRESENTATION

EXTERNAL DATA

SOURCES

SENSEMAKING

SHOEBOX

EVIDENCE FILE

Search & Filter

Read & Extract

Schematize

Build Case

Tell Story

Search for Information

Search for Relations

Search for Evidence

Search for Support

Reevaluate

TIME or EFFORT

STRU

CTUR

E SCHEMAS

HYPOTHESES

PRESENTATION

EXTERNAL DATA

SOURCES

Sensemaking Loop

Foraging Loop

Entity Workspace Notebook

Drag-and-drop interface for capturing knowledge

Snap-together knowledge

Captures the user’s degree of interest

Controls automatic highlighting

AUGMENTED SOCIAL COGNITION


Newell

Meg Stewart



shift)   Input on data (Data coupling)   Attention-reactive displays (Implicit coupling)   Sensemaking tools (Representation shift) SOCIAL LEVEL (Social cognition)   Social sensemaking (Social representation

shift)

TAGGING WORKS IF MANY PEOPLE

1 User

  1 User: 6 tags   Many users: 100

tags   Need ~ 20

Furnas

Info

rmat

ion

Con

nect

ivity

Social Connectivity

Nova Spivack

Ben Wattenberg

Viégas & Wattenberg

MANY EYES

Social Amplification

Rep

rese

ntat

ion

Am

plifi

catio

n




shift)   Input on social context (Social representation

shift)

WikiDashboard Social Transparency: Make socially significant information visible.

Bongwon Suh & Ed Chi

lowering system costs increases sharing

Ma.gnolia

Ma.gnolia

Google Reader

Media Wiki Google Notebook

Google Notebook

intended recipient:




shift)   Input on social context (Social representation

shift)

Efficient Communication Tight Coupling

Representation Shift

-- Minimize mental time and motion -- Maximize (information gain/(time cost) -- Information scent (by design)

-- Input on data -- Implicit coupling -- Information scent (by machine)

-- Input on output -- Information visualization -- Sensemaking tools -- Social sensemaking

augmented large scale cognition · 2015. 1. 21. · augmented large scale cognition stuart card...

Documents