Software-based Assistant for Personal Information Management
By: Nuno Magalhaes Ribeiro
Hypermedia & Multimedia Group
Department of Computer Science
The University of York
Supervisor: Dr. Ian BenestAssessor: Dr. Patrick Olivier
December 14, 1998
14/12/98 Department of Computer ScienceThe University of York
2
MotivationI enter my office for the first time today. My computer screen is blank as always.
My computer senses me and asks me (through speech) if I want to read my new e-mail.
I say “Not right now, I have an important call to make first !”
I dial the number and wait for the connection while thinking of everyone I must call to tomorrow’s important project meeting. “I’d like to have every thinking brain with me to help me decide what to do next”, I think.
In a moment I am on the phone with Paul, the project leader of my group, telling him the names of everyone I want to meet …Julia, Mike, Joan,…
Suddenly, my computer screen displays a space ship moving around and the four letters that compose a foreign name…JOSE
Oh..and Paul, please don’t forget to also invite that new project member, Jose, you know the one that wrote that interesting report about spaceships fuels. O.K. Bye, bye. See you tomorrow.
I handle the phone and briefly think “I almost forgot to tell him about Jose”, before leaving my office thinking about the ideas I read in that report and how to make the most of them…
14/12/98 Department of Computer ScienceThe University of York
3
Identifying the Problem
[Maes, 97] “mismatch between the complexity of our lives and our cognitive abilities”:
• “too many things to keep track of”
• “information overload”
• “learn and remember more” information
The amount of (personal) information is increasing at a fast pace:
• difficult to remember information (what we know but can’t recall)
• difficult to find existing information (what we need but can’t find)
• time consuming for us to manage every piece of information we gather
14/12/98 Department of Computer ScienceThe University of York
4
An approach to tackle the problem
• [Bush, 45] “A record…must be continuously extended, it must be stored, and above all it must be consulted.”
• [Lamming et al., 94] “a memory prostheses should become a companion in everyday life, automatically capturing data that may be of use later”
• [Bell, 97] “…one can imagine…a guardian angel that can capture and retrieve everything we hear, read and see.”
• [Maes, 97] “prosthetics for the mind (memory augmentation devices) in order to overcome our cognitive limitations”:
• “poor memory for details”
• only “deal with one thing at a time”
• “slow to process large amounts of information”
14/12/98 Department of Computer ScienceThe University of York
5
Notion of a Shadow Assistant
• A shadow assistant stays in the background and observes what we do without interfering, learns about us and our tasks and augments our memory.
1] Observes the user and gathers information:
• monitors the information the user is collecting
• captures relevant data about the context of user’s tasks
2] Uses this information to assist:
• offers to automate mechanistic tasks
• helps recall important details about information needed for a task
• suggests relevant information to a task
[Bush, 45], [ Lamming et al, 94], [Maes, 97], [Bell, 97]
14/12/98 Department of Computer ScienceThe University of York
6
Why is context important in PIM?
• [Lansdale, 88] poor memory for details.
• Therefore, systems which rely on the user to remember details such as filenames are bound to produce low levels of recall. [Lansdale, 88]
• Recall will be best when the cues present at the time of learning and at the time of recall are most alike. [Tulving, 83] (Encoding Specificity Principle)
• physical location of an event, who was there, what was happening at the same time and what happened imediately before and after. [Tulving, 83]
• presenting partial context information about an episode helps people remember more about it. [Tulving, 83] (reconstructing the context).
• “associate our current environment with past experiences that might be related”, in order to suggest existing relevant information to our current task. [Rhodes, 97] (context matching).
14/12/98 Department of Computer ScienceThe University of York
7
What is a Personal Assistant?• [Hoschka, 96] ...some functions of good human assistance
• [Lennon and Vermeer, 95] “…It thus supports the user at all levels of activity…making predictions from repetitive tasks, it saves us both time and frustration…it will be our augmented eyes and ears, an alter ego we create for ourselves”
Assistant Required Properties
Domain Competence
CompetenceAssessment
Learning and adaptingbehaviour
Processing impreciseinstructions
Explaining abilities
Cooperation support
14/12/98 Department of Computer ScienceThe University of York
8
What is an Automated Personal Assistant?[FIPA, 97] Overview of the Personal Assistant Domain
Software Agent:
• acts semi-autonomously on behalf of a user (wo human guidance)
• models the interests of the user,
• provides services to the user or other people/personal assistants,
• accomplishes routine support tasks (real job),
• is unobtrusive but ready when needed.
14/12/98 Department of Computer ScienceThe University of York
9
What is an Automated Personal Assistant?
Interface Agents
learn from, act on behalf of and collaborate with the user [Maes, 94]
and
Information Agents
look, find and compose information [Nwana, 96]
• An Automated Personal Assistant is a software agent that both exhibits Interface Agent’s properties and performs Information Agent’s tasks.
14/12/98 Department of Computer ScienceThe University of York
10
Analysis of existing approachesto capture user’s activities context
Issues:
1] What contextual information about user’s tasks is gathered?
2] How is this contextual information used?
Survey brings together a set of papers which common purpose is:
• to collect context elements,
• to provide ways to explore these elements to help retrieving information (once known but know forgotten).
14/12/98 Department of Computer ScienceThe University of York
11
What context information about user’s tasks is gathered?
Events are used to describe context of tasks
Time-stamp on documents and events
Operations performed on documents /files (edit, save, create, print, exchange)
Physical events (people met, location, weather report)
Communication events (e-mail received / sent, phone number called, news)
Manual annotations (notes, diary entries for meetings, schedules, visits)
Textual context (word vector for text documents, subject of e-mail messages)
Memoirs
[Lansdale and Edmonds, 92]
Forget-Me-Not
[Lamming and Flynn, 94],
[Lamming et al., 94]
Hive
[Bovey, 96]
Lifestreams
[Freeman, 96]
Remembrance Agent
[Rhodes, 97],
[Crabtree and Rhodes, 98]
14/12/98 Department of Computer ScienceThe University of York
12
Analysis• Common characteristic: time
temporal relations between events helps recall.
• “although precise data about an episode is not usually remembered, temporal relations between episodes are remembered very well” [Lamming et al., 94].
• The more information a system collects about context of user’s activities, the more help it can be to recall details.
• “Forget-Me-Not”, “Hive”,“Remembrance Agent”
a system that is sensitive to context (location and people) can provide reminders / suggestions appropriate to current activities.
• Indexing documents by context improves recall (document retrieval).
context as a retrieval key (only events / groups of events, not tasks)
14/12/98 Department of Computer ScienceThe University of York
13
How is contextual information used?1] Retrieve by context reconstruction (you tell what you remember)
• User searches for information by specifying remembered context details,
• System matches user’s details with existing stored context elements,
• System presents users with a subset of events that matches details,
• Users browse existing events and select those relevant.
2] Suggest by context matching (you automatically receive suggestions)
• “Remembrance Agent”, pro-actively matches current user context with stored contexts about past experiences,
• makes automatic suggestions (only works for text processing).
• Useful utilisation of context elements, particularly in the case of pro-active suggestions if we can extend the idea to activities other than text processing.
14/12/98 Department of Computer ScienceThe University of York
14
Critique
What we did Why we did it
What resulted
How we did it
What did we want
When we did it
What problems did we solve
What caused it
Who was there Where we were What was happeningIt is possible today, but not extremely
usefulIndex by Events
It is much more difficult, but also much more useful because it would allow the shadow assistant to “know” what we were doing
Index by Tasks
Hig
h L
evel
Low
Lev
el
Is it possible to infer High Level events from Low Level events?
14/12/98 Department of Computer ScienceThe University of York
15
Open Issues
• How to infer user’s tasks from low-level events?
• What is relevant information about each user’s context?
• Do we want to be constantly observed? (Privacy)
• What if our personal information falls in the wrong hands?
• What happens if we can remember everything all the time? [Bos, 95]
• Where can observations take place?
• Do we want a humanised interface?
