context-aware / multimodal ui breakout summary
DESCRIPTION
Context-aware / Multimodal UI Breakout Summary. James A. Landay, et. al. HCC Retreat July 7, 2000. James Landay Anoop Sinha Jimmy Lin Trevor Perring Greg Heinzinger Chris Long Ed Chi. Christine Halverson Gian Gonzaga Ken Fishkin John Lowe Adam Janin Russell Eames Elin Pedersen. - PowerPoint PPT PresentationTRANSCRIPT
Context-aware / Multimodal UI Breakout Summary
James A. Landay, et. al.
HCC RetreatJuly 7, 2000
7/5/2000 2
Participants
James Landay Anoop Sinha Jimmy Lin Trevor Perring Greg Heinzinger Chris Long Ed Chi
Christine Halverson Gian Gonzaga Ken Fishkin John Lowe Adam Janin Russell Eames Elin Pedersen
7/5/2000 3
Applications
Alert management* sites beacon context
+“this is a quiet place, no interruptions please”+e.g., movie theater or restaurant
* devices use context to avoid interruptions* wearable t-shirt that jams local cell phones!
“Elvis has left the meeting”* easily share documents from meetings* beam tokens of documents to participants or* use shared context to find docs later
+e.g., I was in a meeting with Ken at Lake Tahoe, find docs
7/5/2000 4
Context Events
Signal changes* like a windowing system
Can use as triggers to cause other actions* change my phone forwarding when I change
locations
Can be immediate or logged for later tacit information mining
7/5/2000 5
Context Implementation Issues
Apps need to share context easily* build-in to apps like cut & paste* context dial tone or infrastructure
Global file system* easier to share context & not have to transfer it* just use pointers
How to search / browse* computers are good at searching large spaces* humans good at making associations
Why not search with Google instead of browser history?* Google easier to get at & seems to work well
7/5/2000 6
Context Toolkits / APIs / Refs
Bill Schilit’s Columbia / PARC Ph.D. GA Tech GVU (Anind Dey) IBM (Maria Ebling) MIT (?) ESPIRIT projects have looked at
context* German? project according to Elin Pedersen
7/5/2000 7
Interface Between Context & Multimodal UIs
“Context is just another kind of input”* different to the user, but similar to the system* user input caused by EXPLICIT user action* context is IMPLICT or DERRIVED
Context and multimodal UIs have similar privacy problems* natural inputs are human “readable”* may not want to share context or my input
7/5/2000 8
Interface Between Context & Multimodal UIs
Context to choose output modality* e.g., user is in a meeting, don’t use speech
Context to disambiguate input(s)* help fusion: there is noise, don’t rely on speech* “the clutching problem” – infer user intent
Modality used to help inter context* e.g., talking to device -> user is alone?
7/5/2000 9
Initial Design for Multimodal UI Design Tool
Create “rough cuts”* informal (sketching / “Wizard of Oz”)* iterative design (user testing/fast
mods)
Infer models from design* designer can augment model over time
Generate initial prototypes* UIs for multiple devices* designer adds detail / improve UI
+or even removes detail
Model