collaboration via the info. utility john canny endeavour mini-retreat 1/20/2000

Collaboration via the Info. Utility

John CannyEndeavour Mini-retreat

1/20/2000

Consequences of Ubiquitous Computing and Calm Technology:• As computing becomes ubiquitous and invisible, it

also ceases to be the focus of activity.

• The focus shifts instead to people and the tasks they are working on.

• This implies a Human-Centered Computing approach (HCC) to designing applications.

• Our work involves collaborations with several faculty outside CS who are expert in aspects of human behavior, and especially group behavior, via a HCC center.

Three takes on collaboration• Design media for natural (physical)

interaction: Desks, PRoPs, Bearables.• Model behavior: Activity-Based

Computing (ABC): – uses tacit data to infer structure in group

members’ activities.

• Learning as collaboration: – Large classes create opportunity via group

cooperation and student perspectives to help other learners.

Designing Media: Distributed Cognition Theory• People use artifacts in the world to share their

thoughts (Norman, Hutchins, Pea).• Computer-enhanced artifacts have richer

behaviors and be shared across distance.

A desk that loses entropy...• The UPM (Universal Planar Manipulator) is a

desk that can move many objects placed on it. It is a tangible input/output device.

Applications of the UPM• Teaching Chemistry: A tangible simulation of

planar organic chemistry. Colored disks for atoms, simulation of bond forces and reactions, movement of atoms by the desk.

• Urban planning. Phycons of buildings moved by initially by hand, then aligned by the desk according to a set of rules.

• Other layout problems: web site design (moving postit-like notes).

UPM principle of operation

PRoPs: Proxys for remote collab.

• There is still a huge gap between face-to-face and videoconferencing.

• First difference: Physical cues: gaze, posture, proxemics... are missing or distorted.

• Much collaboration involves “being in” a remote space. You need to have a body there...

PRoPs: Proxys for remote collab.

• PRoPs are internet-controlled robot avatars that allow you to have a second presence somewhere.

• The user sits at a computer console watching video from the PROP, and controlling it with mouse and joystick.

Theoretical underpinnings

• Right now, much of the theory driving PROP design comes from outside CS:

• Non-verbal communication (D. Keltner psychology).

• Theory of interpersonal psychology (G. Mendelsohn).

• Spatial reasoning and spatial memory.

Bearable computers• The laptop can serve as a “bearable” computer,

linked wirelessly to its owner.

Laptop in briefcase with wireless TV

and keyboard TXs.

Pocket-worn portable LCD TV

Double-mirror element in glassesHeads-up virtual image

Wireless chordal keyboard or Palmpilot in pocket for input.

Advantages of Bearables vs. Wearables

• Short term:– Leverage higher performance of laptops and

mini-TV displays compared to wearable components.

• Long term:– No wires between components.– No power source to the head.– The laptop display is preferable sometimes. – Some optical benefits for augmented reality.

ABC: Activity-Based Computing

• How do you build a UI when the computer is invisible, e.g. a smart space?

• What is context and how do you use it?

• How does ABC help collaboration?



• Build a conceptual model instead.• What is context and how do you use it?

• How does ABC help collaboration?



• Build a conceptual model instead.• What is context and how do you use it? • Use Activities as the context.• How does ABC help collaboration?



• Build a conceptual model instead.• What is context and how do you use it? • Use Activities as the context.• How does ABC help collaboration?• Pro-active document sharing;

Understanding roles; Document context;


• Activities are clusters with:– Users– Documents– Tools

• The model can be implemented as a GUI, or with speech, etc.

• Nearness encodes awareness.

ABC: Motivation• Based on “activity theory”, a rich psychological

theory of human behavior, especially collaboration.

• AT Emphasizes on tools (such as computers) as mediators of activity.

• Includes the role of symbol systems (language, documents) as mediators.

• Ties together development (learning) and everyday activity.

• Rich models of context.

ABC: Uses• Context (disambiguation) of person, object, tool.• Pro-active sharing of information. • Skill mining (tracking user expertise). • Document visibility and access rights. • Understanding individual’s roles in activities.• Managing work relationships.• Inferring quality of documents/tools.• Sensemaking for documents. • Perspectives on a document or meeting.• Attention management/awareness.

ABC: Under development now• Context (disambiguation) of person, document,

tool.• Pro-active sharing of information. • Skill mining (tracking user expertise). • Document visibility and access rights. • Understanding an individual’s roles in activities.• Managing work relationships.• Inferring quality of documents/tools.• Sensemaking for documents. • Perspectives on a document or meeting.• Attention management/awareness.

ABC: Tacit data sources• Direct communication: 1-1 email, phone, F2F.• Topical discussions, forums, F2F meetings.• Document access, reading, doc. markup.• Document search and browsing, who, when.

Knowledgescapes: Info Retrieval with tacit data

InformationNeed

Query(terms)

SearchEngine

Document1

Document2

Document3

Document4

Rankings from reading time.Treat as probability of interest

or E(interest)

Info Needs and tacit rankings

Infoneed1 Document1

Document2

Document3

Document4

Infoneed2

Infoneed3

Document5

P(D1|N1)

P(D5|N3)

• Start with a query string and use all the information need nodes recorded for that query.

• Return highest weighted docs.

First use: Most popular hits

Document1

Document2

Document3

Document4

Infoneed2

Infoneed3

Document5

Query

• The IN is usually more specific than the query, and this is encoded in the user’s tacit rankings.

Tacit knowledge in user queries

Document2

Document3

Document4

Infoneed1

Infoneed3 Document5

“Airlines” Info

need2

Document1

West to East Coast

US to France

• Tacit user ranking while browsing boosts most probable info needs, improving recall.

Tacit knowledge in user queries

Document2

Document3

Document4

Infoneed1

Infoneed3 Document5

“Airlines” Info

need2

Document1

West to East Coast

US to France

Current testbeds:

• Berkeley DigLibs (environmental) collections. Knowledgescapes on Dlib is running at: http://indios.cs.berkeley.edu/knowledgescapes.html

• IMDB• Philip Stark’s statistics course.• NEEDs: the ME courseware database.

Future:• My groups web and campus library access.

Education as collaboration:Opportunities of scale. • In lecture format classrooms, attention is the

critical resource (Norman, Papert...).• Attention is best gained by interaction: with

artifacts (the LOGO model) or with other students (the Stanford TVI model).

An augmented reality classroomVirtual image with notes,

questions, private chat space

Physical space

Glasses to rotatethe laptop image

Livenotes in the classroom• Students work in groups of 4; communicate

silently via pen or keyboard chat.

• Each group has one main note-taker; others add their own comments or questions to the transcript.

• Students can mark up a group transcript, the lecturer’s notes, or a non-archived window.

• One student per group works as facilitator or TA, posing questions to the others.

• The group transcripts include notes, plus student comments and questions.

• Remote live participants should be equally engaged in the note-taking.

Using Livenotes

• 4 Students used Livenotes in a grad course in F99 on IBM laptops running Netmeeting on a wireless net.

• Reactions:

– Overlay touch screens were bad, everyone used keyboard chat.

– Difficulty in listening and chatting simultaneously only in first lecture.

– After that, attention level higher. No chance of falling asleep.

– Many notes: two parallel threads, the note-taker and the group chat. Group chat periodically comes back to lecture content as new notes appear from the note-taker.

Initial feedback

• Move to Vadem Clios with wireless:

– Support both keyboard and pen note-taking.

– Cheaper, lighter, batteries last all day.

• Develop custom software based on feedback:

– Support conversation threads.

– Include hyperlinks (or hyper-ink).

– Include timestamping to allow synchronization with MM transcript of the lecture.

– In-class “lecture rewind” was proposed as a useful feature.

Next Step

• Offline students lose interaction, but gain through access to live student perspectives (their notes).

• By organizing live student notes according to expertise and a few dimensions of perspective, we can provide offline students with an ideal “scaffolding” perspective.

Further along...

The End..

PROPs and Bearables are complementary

• A user with a bearable display can see a video overlay of the PROPs pilot.



• This display can cover a much larger area than the PROPs built-in display, and provide body language cues.



• This display can cover a much larger area than the PROPs built-in display, and provide body language cues.

• In this display, the pilot is standing up, implying he is controlling the PROP with a wearable computer.

collaboration via the info. utility john canny endeavour mini-retreat 1/20/2000

Documents

group behavior

collaboration design

group cooperation

activitybased computing

prop design

model behavior

group members activities

remote collab