literature review
DESCRIPTION
TRANSCRIPT
LOCATION MODELLING AND MACHINE LEARNING IN SMART ENVIRONMENTS
University of SydneySchool of Information Technologies
Literature Review
Supervisor: Associate Professor Judy KayAssociate Professor Bob Kummerfeld
Robert Whitaker0111139
Table of Contents1. INTRODUCTION.................................................................................................................................................3
2. UBIQUITOUS COMPUTING SENSORS.........................................................................................................4
2.2. COMMUNICATION FACILITIES.........................................................................................................................5
2.3. CONNECTIVITY................................................................................................................................................5
2.4. INTERACTION..................................................................................................................................................5
3. LOCATION AND ACTIVITY MODELLING.................................................................................................6
3.1. USER MODELS................................................................................................................................................6
3.2. USER MODELLING SYSTEMS...........................................................................................................................6
4. MACHINE LEARNING FOR PREDICTING USER LOCATION AND ACTIVITY................................7
4.1. LEARNING TECHNIQUE...................................................................................................................................7
4.2. LEARNER.........................................................................................................................................................8
4.3. SOURCE...........................................................................................................................................................8
5. PREVIOUS WORK.............................................................................................................................................9
5.1. DETERMINING LOCATION.............................................................................................................................10
5.2. DETERMINING AND PREDICTING LOCATION.................................................................................................11
5.3. DETERMINING AND PREDICTING ACTIVITIES................................................................................................13
5.4. DETERMINING LOCATION AND ACTIVITIES..................................................................................................16
5.5. MODELLING ENVIRONMENTS.......................................................................................................................17
6. RESEARCH PROBLEM..................................................................................................................................19
7. RESEARCH CONTRIBUTIONS....................................................................................................................21
8. RESEARCH PLAN............................................................................................................................................22
9. SUMMARY........................................................................................................................................................22
APPENDIX A – LANCASTER GUIDE PROJECT...........................................................................................24
APPENDIX B – ASSISTED COGNITION PROJECT......................................................................................25
APPENDIX C – DOPPELGANGER PROJECT................................................................................................27
APPENDIX D – ACTIVE BADGE AND BAT PROJECT................................................................................29
APPENDIX E – WEARABLE SENSORS...........................................................................................................30
APPENDIX F – BSPY...........................................................................................................................................31
References...............................................................................................................................................................32
- 2 -
1. Introduction
Ways of determining the locations and activities of people has been a strong focus of research
into ubiquitous computing. The establishment of smart environments through the development
of various sensors has allowed vast amounts of data to be collected on human behaviour and
their use of such environments. Through this data it is now possible to model these
environments using a range of user models, and from these perform predictions using various
machine learning techniques.
The components of such systems can best be summarised in figure 1 and include
Sensors/Devices – responsible for the acquisition of data from the user and the devices
in surrounding environment; such as mobile phones, PDAs, laptops, motion sensors,
and pressure pads.
Location and Activity Modelling – The storing of acquired data to build models of the
user’s use of the environment. For example, the modelling of the users movements
within the environment using the data acquired from location sensors.
- 3 -
Applications
Location andActivity Modelling
Sensors/Devices
Location andActivity Prediction
MachineLearning
User A User B
Figure 1: High level view of modelling and prediction systems, showing the relationships between the various components
Prediction – Uses the data acquired and stored in user models to formulate some
conclusion using machine learning techniques. For example predicting where a user
will be in two hours time based on their current location and past movements over the
last two hours.
Applications – The programs that use the generated models and prediction results to
personalise themselves to the users behaviour. A scenario application is a smart
meeting system, whereby User B interacts with an application that allows a meeting to
be booked with User A based on User A’s passed movements and activities modelled
in the system.
This review will focus on each of the components in figure 1: ubiquitous computing sensors;
location and activity modelling; and machine learning for predicting user location and activity.
In addition to each component, the previous work in the research domain, research problem,
research contributions, and research plan will also be discussed.
2. Ubiquitous Computing SensorsUbiquitous computing is being considered the third wave of computing. It is focused around
the concept that each person has many devices and that many more devices will be embedded
into the surrounding environment. The concept of ubiquitous computing was introduced by
(Weiser 1991) where the ideas of a computer knowing its location and adapting its behaviour
were first discussed.
A key component of ubiquitous computing is the devices that make up the surrounding
environment. These devices can range in size, cost, power consumption, application and
connectivity. They can range from a person’s laptop computer to their mobile phone to a
- 4 -
security lock on a door. The list of the different devices found in these environments is
endless, however the following are observed common characteristics.
2.1. Positioning Systems
Many devices today have the ability to determine their location based on a predefined set of
rules. There is a variety of positioning technologies; such as Bluetooth and GPS. Devices
implementing these technologies provide the ability of determining its location based on a
predefined coordinate grid; for example, GPS uses longitude and latitude.
2.2. Communication Facilities
Many ubiquitous devices are built on top of personal communication devices such as mobile
phones and PDAs. These all provide ways of the device or environment communicating with
its users. But the communication between the environment and its users is not limited to these
types of devices. Communication can take place in many other forms. For example the key
locks in the Madsen Building, use sound to communicate with users who are permitted to
access the respective room or not.
2.3. Connectivity
Devices in a ubiquitous environment require some mechanism to communicate with one
another; there are a range of connectivity measures available, including local area network and
wireless connections.
2.4. Interaction
Ubiquitous devices can be categorised as either interactive or passive devices. For interactive
devices users are continuously supplying information, such as in the example of a laptop
computer or PDA, while passive devices are those that acquire data without any human
interaction, such as a motion sensor.
- 5 -
3. Location and Activity ModellingLocation and activity modelling is concerned with modelling peoples’ locations and actions. It
is a subset or specialised form of user modelling and will be used interchangeably for the
remainder of this review. (Csinger 1995) defined user modelling as “the acquisition or
exploitation of explicit, consultable models of the human users of systems”; similar definitions
were made by (Pohl 1997) and (Orwant 1992) in their discussion on the implementation of
various user modelling systems.
There are two aspects of user modelling. The first is the actual user model and the second is
the system responsible for the acquisition and exploitation of the data collected. Neither can
exist without the other.
3.1. User Models
A user model is the structure that acquired data can be stored in to allow for personalisation of
systems and prediction of user behaviour. (Ashbrook, et al. 2003) considered a user model to
be “a collection of data on some particular aspect of a human user’s behaviour, when
associated with a limited set of contextual clues, yields predictions on what behaviour the
human will engage in next”.
3.2. User Modelling Systems
A user modelling system is the application system that acquires and stores new evidence about
a person, identifies consistencies and inconsistencies held about a user and answers queries of
the application concerning the currently held assumptions on the user (Kobsa 2000). These
systems are often referred to as user modelling shells as defined in (Kobsa 1990).
- 6 -
User modelling systems have seen many developments, both in the commercial and academic
fields. (Kobsa 2000) identified and summarised some the key systems including BGP-MS,
Doppelganer and um. From this work, Kobsa formulated a list of common characteristics,
including:
The representation of assumptions about user characteristics in individual user models;
Representation of common characteristics between user models leading to the creation
of stereotypes;
Classifying a user as belonging to a given stereotype;
Methods of recording a user’s behaviour and interaction with the system;
Ability to make assumptions about a user based on information recorded in their user
model;
Methods of justifying assumptions made based on a user model;
Evaluation of user models.
4. Machine Learning for Predicting User Location and ActivityMachine Learning has three core concepts the learning technique, the learner and the source,
each are discussed below.
4.1. Learning Technique
Learning is the process of acquiring new knowledge and gaining a greater understanding of
knowledge previously held. (Shavlik, et al. 1990) discussed the two main ways in which a
system can learn:
1. The system can acquire new knowledge from external sources;
2. The system can modify itself to exploit its current knowledge more effectively.
- 7 -
When applying learning algorithms to location/activity modelling and ubiquitous computing,
these systems use the first of these techniques, as they are continually acquiring data from the
surrounding environment. The actual algorithms these systems use to perform learning can
vary and may use more than one. Some of the more common algorithms used by systems in
this area include Linear Prediction, Markov models, Bayesian Techniques and Beta
Distribution.
4.2. Learner
A machine learning system is an application that acquires data on the use of another machine
and performs calculations on that data to return some result. There are several key
characteristics to any machine learning system. (Orwant 1992) highlighted five of these
characteristics and discussed their importance to user modelling, summarised in table 1.
Characteristic Description
Learner The system which is actually performing the learning
Domain Refers to the types of data that learning algorithms will be applied to
Information Sources Refers to the array of sensors data is acquired from
Prior Knowledge The data already known by the system that the learning algorithms can be applied to
Performance Criteria Analysing the results of the algorithm against those which are expected, this is normally
not applied in user modelling as a user modelling system never stops learning.
However, many papers still do perform some performance evaluations.
Table 1: Characteristics of a system which learns and their application to user modelling (Orwant 1992)
4.3. Source
When discussing a system in this context, it is referred to in broad terms as the object we are
learning information about. This could be anything from a physical machine to an individual
or group of people. The tools used are dependent on the system or object learning is being
performed on. The common data sources for systems in this area are the ubiquitous devices
embedded in the surrounding environment.
- 8 -
5. Previous WorkThe fields of ubiquitous computing, location modelling and machine learning are very active
research areas that have seen the development of many projects. When examining work in this
area, the relevant projects can be broken into five categories: determining location;
determining and predicting location; determining and predicting activities; determining
location and activity; and modelling environments. Table 2 provides a summary of these
projects with details on what categories they satisfied as well as any learning algorithms they
may have used.
Project
Categories Satisfied
Learning Algorithm
Lo
cati
on
D
etec
tio
n
Lo
cati
on
D
etec
tio
n
and
P
red
icti
on
Act
ivit
y
Det
ecti
on
an
d
Pre
dic
tio
n
Lo
cati
on
an
d
Act
ivit
y
Pre
dic
tio
n
Mo
del
ling
Active Badge X N/A
Active Bat X N/A
Lancaster Guide X N/A
Assisted CognitionX X X
Bayesian, Relational Markov
Models
Multiple User
DetectionX X
Markov Models, Clustering
MyPlace X X N/A
DoppelgangerX X
Beta Distribution, Linear Prediction,
Markov Models
LaboUr X X Markov Models
Wearable Sensors X X X No explicit algorithm
Activity Zones X X X No explicit algorithm
Table 2: Overview of the relevant projects in the research area, shows the categories each project addresses as well as any learning algorithms they may use
The combining of these five properties assist in the development of a location authority
(Shafer 2003) system. This section will examine important projects from these five categories
in greater detail.
- 9 -
5.1. Determining Location
The first step in performing any location prediction is first calculating where the user or object
currently is. Projects in this category use devices to emit or query its current location.
The Active Badge Project (Davies, et al. 2002, Wang, et al. 2003, Want, et al. 1992) was one
of the first indoor positioning system. The system required users to wear badges that emitted
signals to sensors that determined the location of the user. The project was driven by the
notion of locating people in an office environment and could be applied to any problem
needing to determine a person’s location. An application that implemented this system was
teleporting (Richardson, et al. 1994), which involved determining the current location of the
user and having their desktop settings sent to the nearest terminal.
An extension on the Active Badge Project is Active Bat (Addlesee, et al. 2001, Harter, et al.
2002), which is a 3D indoor position system. In these systems small units called Bats (see
Appendix D) are attached to objects in the space, for example bats would be attached to users,
desks, chairs, and computers. To determine the location of a bat, base stations periodically
transmit a radio message causing the corresponding bat to emit a short ultrasound pulse. If the
distance from three base stations can be determined, the exact position of the bat in the 3D
space can be calculated.
Active Badge and Active Bat both perform well for indoor environments. However different
technologies are required for outdoors. Most outdoor positioning systems use GPS to
determine location. An example of such system is the Lancaster Guide Project (Davies, et al.
- 10 -
2002, Distributed Multimedia Research Group 2004). The aim of this project was to provide
information normally found in a tour guide but also to give the user the information based on
their interests, current context and movements. The application was implemented using tablet
PC’s as end-systems connected to information servers through an 802.11 network deployed
around the city; Appendix A contains images of the system interface. The system determined a
user’s location by querying the base station the user was currently connected to and displaying
the information associated with that base station. A field trial was performed on the system in
1999 for a period of four weeks. This highlighted several problems (Davies, et al. 2002), with
a conclusion that such systems were restricted by the infrastructure available to them.
5.2. Determining and Predicting Location
Determining an object’s current location can be performed by the methods discussed in the
previous section. However, the challenge is using these techniques to perform some prediction
of where an object’s location will be in the future.
The Assisted Cognition Project (Kautz, et al. 2003, Kautz, et al. 2002) uses ubiquitous
computing environments to enhance human capabilities with a particular emphasis on
assisting people with cognitive limitations. The project uses various sensors, handheld and
wireless devices to assist building models of patient behaviour and routines (See Appendix B
for the system architecture). An important application is the Activity Compass (Patterson, et
al. 2002). Shown in Appendix B, it uses GPS readings to generate models of user behaviour.
From these, it attempts to direct the user to the location which best suits their current
movement patterns. The Activity Compass incorporates learning algorithms to make its
- 11 -
predictions. (Kautz, et al. 2003) reported the use of two learning techniques, Bayesian User
Modelling (Fox, et al. 2003) and Relational Markov Models (Anderson, et al. 2002).
Another product of the Assisted Cognition Project has been the development of prediction
algorithms based on the relationship a person’s position on a map. (Patterson, et al. 2003) and
(Hightower 2003) discuss the ideas of moving away from the traditional mathematical
techniques to turning a street map into a graph. Predictions of locations are made from the
graph, using past historical data, measurements of recent movements and some common sense
(eg. Buses normally stop at bus stops and traffic lights, cars drive along roads).
The Assisted Cognition Project focuses on developing solutions for a single user. Another
similar project (Ashbrook, et al. 2003) extends this, predicting multiple user locations and
movements. The system collects data using GPS hardware and storing the results in a location
model. It uses clustering and Markov Models to analyse and predict the location and
movements of multiple users from the models. This work provides foundation principles for
developing a location prediction system for multiple users.
An important aspect of the prediction process is the generation of models to formulate patterns
in the acquired data. The MyPlace Project (Kay, et al. 2004) explores techniques for
developing models of users, sensors and locations using accretion user modelling techniques
and developing various applications to use these models. The project explores two main
scenarios, the Locator Scenario and the Invisibility Scenario which attempt to use the models
to perform some prediction and personalisation of the user’s environment.
- 12 -
On the basis of these systems, we can identify three main components
Training data and a data store of historical data on the use of the application;
Pre-processing the raw data so it can be modelled;
Learning Algorithms to analysis the historical information and perform predictions.
These observed components would be the foundation of any prediction system, be it location
or something else. One negative aspect of these projects is the lack of evaluation currently
performed; this is most likely due to the immaturity of these projects.
5.3. Determining and Predicting Activities
The concepts previously described on location could also be applied to this category as well.
Two projects which explore predicting of activities are DOPPELGANER and LaboUr.
The DOPPELGANGER (Orwant 1991) system was a generalized tool for gathering and
processing information about users. The developed system had three main components: (See
Appendix C for architecture diagram)
Sensors – the devices that acquired data for the system, these could be hardware or
software devices. Examples of suggested sensors included pressure pads and Unix
commands like finger (provides information about the user contained in the system
password file) and lastcomm (returns the name of the last command executed by the
user).
Learning Toolbox (Orwant 1995) – implemented various learning algorithms each
used under different circumstances and used these to form results that were returned to
various applications. These techniques are summarised in table 3.
- 13 -
Applications – the programs which use the models created to customize themselves to
the user’s interests and needs. Examples include Dopmail (Orwant 1995) and an
electronic newspaper.
Technique Description Reasons for Use
Beta Distribution Used when determining a user’s
preference for some feature or
interest on a topic
When calculating a person’s interest in a topic it
can be either yes or no, using this technique then
allows the cumulation of these opinions.
Linear Prediction Used for projecting a data set into
the future based on past recordings
made
The occurrences of past events are known. This
technique allowed the knowledge of these events
to predict when that event will reoccur next.
Markov Models Used for determining the future
physical or working state of the
user
Can treat each location being modelled as a
different state. This technique then allows the
prediction of movements between different states.
Table 3: Summary of the learning techniques used in Doppelganger
The architecture of this system is similar to that shown in figure 1. Both architectures acquire
data from a range of sensors and are used by many different applications. The main
differences exist in the modelling and prediction processes. Under the architecture in figure 1
there has been a separation of the learning and modelling processes which was not present in
the DOPPELGANGER architecture.
One feature implemented in DOPPELGANGER, not present in similar systems, is the notion of
confidence. In addition to making predictions based on the historical data, the system also
provides a confidence percentage providing the user feedback on how correct DOPPELGANGER
thinks the prediction is, for example in addition to returning a result is will say that there is a
0.9 certainty the result is correct. This confidence feature is very useful in the interpretation of
results and confidence in the system and still remains a notable omission in many systems in
the field.
- 14 -
Another novel concept DOPPELGANGER also introduced was the notion of user communities.
User communities extend the concept of stereotypes (Kobsa 1993), to categorise people
against the stereotype they best match, which may not be the one their assigned to, at the same
time as updating the model of their assigned stereotype with the acquired data. The benefit of
this approach is that the system might better represent those users that do not match the
patterns of others in their group, but as evaluation is yet to be performed these benefits are yet
to be realised.
Appendix C shows the interfaces for the DOPPELGANGER System. One important aspect to
highlight with these interfaces is that they provide srcutability in the presentation of the
results. Scrutability enhances the confidence information provided, by allowing the user to
examine how the results were calculated, view what data was used to determine the prediction,
and view the prediction at varying levels of granularity.
LaboUr (Pohl 1997, Pohl, et al.) concentrated on developing a centralised user modelling
system and treating user modelling as an open process. The components of this architecture
were similar to those in the DOPPELGANGER System, except it introduced filter mechanisms
which if necessary performed transformations on acquired data to ensure the input was
suitable for the learning algorithm being used. An example scenario on how these filters could
be applied to such systems is shown in scenario 1.
- 15 -
Scenario 1: Device A provides information using 3 data fields, while device B provides the same
information across 4 data fields, the filtering would allow the conversion of these inputs into the
same format
5.4. Determining Location and Activities
This category of application integrates the concepts of the previous three sections. These
Applications have the ability to both determine an object’s location and the context in which it
is being used.
One such application is (Lee, et al. 2002) which uses wearable sensors (See Appendix E) to
detect the location and movements of a person. This application uses a range of sensors to
determine if a person is walking, running, or standing still at the same time determining their
location. Such motion or context is measured through the rate of change in the person’s
displacement from their initial start position. Applications like these could be extended to
include additional sensors to determine different contexts; an example could include a sensor
detecting a person’s speech, and this sensor could then add richer information to many of the
motion cases.
An alternative or additional mechanism of determining activity is the association of activity
with location. (Koile, et al. 2003) discusses the idea of an activity zone, an activity zone is a
location in a building or room that has a specific activity associated with it. An example is a
meeting room that would have the activity “people meeting” associated with it. However, as
(Koile, et al. 2003) discussed there can be many different activity zones in a single
location/room, all dependent on the arrangement of the objects in that location. The use of
activity zones is the simplest way of introducing activity tracking into an already existing
location prediction system, through the association of an activity for each location readings are
taken for.
- 16 -
5.5. Modelling Environments
Most methods of prediction use some form of modelling technique. A common approach
builds up models of the entire surrounding environment and uses those models in someway.
Such models are usually built using sensor-rich environments like those systems discussed in
(Brumitt, et al. 2000) and (Kidd, et al. 1999).
Issue Description
Need for large data sets A common problem with machine learning algorithm is that they can’t build a model
with significant accuracy until it has seen a relatively large number of events to
base its decision on
Need for labelled data Some machine learning approaches require data to be labelled as part of its
calculation process, unfortunately this labelling may not be apparent from the
observations stored in the user model.
Concept Drift The behaviour of the user changes over time, a machine learning system must be
able to adjust quickly to these changes and reflect this in the results it returns.
Computational Complexity The various algorithms can require large amounts of computational power to
execute. Therefore the performance of such algorithms needs to be considered in
their implementation and use.
Table 4: Summary of issues associated with applying machine learning to user modelling (Webb, et al. 2001)
All location and activity prediction requires a data store for recording historical information.
In most systems today, this is done through user models. These can be used both to develop
behavioural models for a single user and/or to model the use of the environment as a whole.
There are many ways modelling can take place. (Glassey, et al. 2003) identified three types of
location models:
Geometric – allows the points of the physical objects to be modelled, for example the
coordinates of an object on a grid
Symbolic – describes a location or object using a predefined label, for example the
name of the room such as G62.
Hybrid – uses a combination of the previous two techniques
- 17 -
Similar definitions were also provided in (Domnitcheva 2001). The way the environment can
be represented using these techniques can be linked with the work on LaboUr. Since it applied
filters to the incoming data from sensors; similar transformations could be applied to convert
the geometric representation received from sensors into a symbolic representation used in the
models. For example a system could convert the coordinate “A2” into the room label “G62”.
Once a user model has been developed the analysis of the model is a complex process. (Webb,
et al. 2001) discuss several issues associated with applying machine learning to user models,
these have been summarised in table 4.
Technique Description
Linear Models Take weighted sums of known values to product a value of an unknown quantity.
Markov Models Based on the Markov Assumption, where the occurrence of the next event
depends on a fixed number of previous events. Given a number of observed
events, the next event is predicted from the probability distribution of the events
which have followed the observed events in the past.
Neural Networks Performed through a structure of networks, non-linear thresholds and the weights
of the edges between each node. The nodes represent the events with the weights
on each path being the probability of it occurring.
Classification Given an n-dimensional space that corresponds to the attributes under
consideration, the generated clusters or classes contain items that are close to
each other in this space and far from other clusters
Rule Induction Consists of learning sets of rules that predict the class of an observation and its
attributes.
Bayesian Networks Are directed acyclic graphs where nodes correspond to random variables, the
nodes are connected by directed arcs with each node associated with a conditional
probability distribution which assigns a probability to each possible value of this
node for each combination of its parents nodes.
Table 5: Summarising the more common techniques used in the field (Zukerman, et al. 2001)
Although the task of applying machine learning to user models is complex, there are many
techniques available, as seen by the summary of techniques used in current projects
summarised in table 1 and those summarised in table 5.
- 18 -
From the analysis of the previous work and this discussion, is can be observed that user
models are the logical choice for storing and analysing user behaviour in any location and
activity prediction system.
6. Research ProblemThere has been considerable research in the areas of location prediction, activity detection and
location detection. However, little work has occurred integrating all these issues into one
system. The aim of this research is to explore ways of determining the current location and
activity of a person and predicting future locations/activities using machine learning and user
modelling techniques on data collected from the surrounding “smart” environment.
Based on previous work described about, we can identify four main components of such
environments, summarised in figure 2. These components include
Environment – this includes the various sensors used to acquire data about the
environment and its users;
- 19 -
PDA
Machine Use
PDA
Machine Use
Conversion intoLogical
RepresentationDatabase ofLogical Data
Conversion intoLogical
RepresentationDatabase ofLogical Data
User ModelCreator - personis
Data store ofmodel info
User ModelCreator - personis
Data store ofmodel info
User
Where is X mostlikely to be?
Build Result Setusing Markov
Model
Where is X mostlikely to be?
Build Result Setusing Markov
Model
Environment
Filters/Conversion
Modelling
User
Applications
Figure 2: High Level View of intended system
Filters or Conversions – the data received by the sensors may not be in an acceptable
format for the system or may not be in a user friendly form eg. MAC address of device
could be replaced with the name of the device;
Modelling – based on the acquired data models, are generated for both the environment
and its users;
Applications – this is the software which uses the generated models. Examples include
a user prediction application or a smart meeting system application.
Aspects that will not be covered in the solution will be issues of security and scalability of the
system, both at the device end and the application end of the system.
In order to implement the various aspects of the system a variety of already available packages
will be used to provide some of the functionality; summarised in table 6.
Package Description/Use
BSpy Indoor location detection system developed by the University of Sydney,
see Appendix F for a sensor map and data view.
Elvin Messaging (Distributed
Systems Technology Centre
2003, Mantara Software
2003)
A messaging system that will be used to send the acquired data from the
devices to a central system.
Personis Software (Kay, et al.
2002)
user modelling software to be used to generate the various environment
and user models
Visualisation Software to provide meaningful feedback and srcutability of results, tools such as dot
(Ganser, et al. 2002) will be used to provide a graphical representation of
the results
Markov Modelling Packages Markov Models will be used in the prediction algorithms, packages such as
(Young, et al. 1999) will be used in the construction of these models from
the user models.
Table 6: Summary of possible packages to be used in the solution
- 20 -
Overall the aim of the research is to create a location authority that addressed some of the
functional requirements discussed in (Shafer 2003) and to provide evaluations on this type of
system.
7. Research ContributionsThe intended contributions of this research include
The exploration of Markov Models and Linear Prediction to predict the location and
activity with regards to time
The integration of activity and location detection and prediction into a single system
The visualisation of prediction models to provide srcutability to predictions.
Exploration of extensions to previous systems including the use of confidence in
predictions and the use of various user modelling techniques.
The exploration of predictions and user models using a combination of varying
granularities of data, for example the combining of daily and weekly models of a user.
- 21 -
Figure 3: Research Plan - timeline of tasks to be completed in the research process
8. Research PlanThe timeline of events and the entire process has been summarised in figure 3. From this it can
be seen that the process has been broken down into nine components summarised in table 7.
Task Description
Definition of Research
Problem
involved determining the research field and specific aspects of the field to
be investigated
Review of Literature in the
field and available packages
Gain background knowledge of previous work in the area as well as gain
experience using various packages that may be useful in the development
of a prototype system.
Experimental Design Design of the various evaluations to be performed on the prototype as well
as the development of an activity sensor to be used in the experiments and
evaluation.
Prototype Design and
Development Version 1
Development of a prototype that makes the use of one location and one
activity sensor, provides the current location and activity in addition to basic
predictions, and constructs user models to be used in future versions.
Experimental Execution Running the evaluations previously designed on a range of location and
activity sensors and basic testing/use of version 1 of the prototype.
Prototype Design and
Development Version 2
Make refinements to version 1 based on evaluation results, adjustments
made to version 1 to allow input from various activity and location sensors
and the introduction of confidence into the prediction algorithms. Test and
evaluate this version based on the data collected from the previous
experiment.
Prototype Design and
Development Version 3
Includes the development of sample applications to use the system
examples may include the smart meeting system. In addition improvements
to the interface and visualisation of results should be performed.
Execution of Final
Experiments
Evaluate the final prototype against the original experimental designs.
Writing of Thesis Perform analysis of evaluation and write up of each stage and the final
system prototype.
Table 7: Description of the various tasks shown in figure 3
9. SummaryOverall this review examined the key concepts associated with user prediction systems and
associated previous work. It identified various aspects of the research area that requires further
work including the performing of further evaluations on various systems, the integration of
ideas between systems and enhancements to systems such as adding srcutability and
- 22 -
confidence to predictions. The research problem being investigates aims to address some of
these aspects through its planned contributions to the research field and the implementation of
various prototypes to evaluate these contributions.
- 23 -
Appendix A – Lancaster Guide Project
Figure 4: Lancaster Guide Interface - giving user information about the current location (Distributed Multimedia Research Group 2004)
Figure 5: Lancaster Guide Interface - providing the user with directions and view of the surrounding area (Distributed Multimedia Research Group 2004)
- 24 -
Appendix B – Assisted Cognition Project
Figure 6: Activity Compass device developed (Patterson, et al. 2002). The devices uses a GPS transmitter connected to a PDA for the interface. The current image is giving the user directions to their home.
Figure 7: Architecture of the Assisted Cognition Project. There are three components to the system, the sensors that acquire the data from the environment, the server which performs the modelling and prediction, and the client which interacts with the models and prediction results.
- 25 -
Figure 8: The Green line shows the users movements as tracked through the Activity Compass. Based on the map layout, predictions can be made using past movements and the direction of paths that can be followed on the map.
- 26 -
Appendix C – Doppelganger Project
Figure 9: Structure of the Doppelganger System. The diagram shows the three key components of the system; the sensors, learning toolbox and the applications (Orwant 1996)
Figure 10: Screenshot of Doppelganger system. This image shows the results from a linear prediction algorithm being performed to calculate when a person is most likely to log into the computer system next and how long they will use it for (Orwant 1996)
- 27 -
Figure 11: Visualisation of results from Doppelganger using Markov Models for predicting the location state of the user. The thicker the line connecting two nodes the more occurrences have been detected of the movement between those states (Orwant 1996).
- 28 -
Appendix D – Active Badge and Bat Project
Figure 12: An Active Bat device used to emit the signal of the users current location (Harter, et al. 2002)
Figure 13: Operation of Bat location sensor system. The base stations send signals to the beacons which emit a ultrasound pulse. Readings from three base stations allows the exact position to be determined (Addlesee, et al. 2001).
- 29 -
Appendix E – Wearable Sensors
Figure 14: Sensors used to detected movement and movement type of a user (Lee, et al. 2002). The sensors in the thigh and waste determine whether the person is walking forwards or backwards. The speed at which the movements are detected determines the speed they are walking/running. Using this movement information and an initial start position the current location can be determined using the displacement of the current position from the initial position in addition to the context in which the person is currently moving.
- 30 -
Appendix F – Bspy
Figure 16: Sensor Layout of the BSpy System in the Madsen Building. Not shown here is a sensor in the Seminar room which is located in another part of the building. (Kay, et al. 2004)
- 31 -
Figure 15: Output from BSpy System. This screenshot shows the data stored by the system currently (BSpy System 2004)
References1. Addlesee, M., Curwen, R., Hodges, S., Newman, J., Steggles, P., Ward, A. and
Hopper, A. Implementing a Sentient Computing System. IEEE Computer (Auguest 2001). 2-9.
2. Anderson, C.R., Domingos, P. and Weld, D.S., Relational markov models and their application to adaptive web navigation. in Eigth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, (2002).
3. Ashbrook, D. and Starner, T. Using GPS to learn significant locations and predict movement across multiple users. Personal and Ubiquitous Computing, 7. 275-286.
4. Brumitt, B., Meyers, B., Krumm, J., Kern, A. and Shafer, S., EasyLiving: Technologies for Intelligent Environments. in 2nd International Symposium on Handheld and Ubiquitous Computing, (2000).
5. BSpy System. Search Logs, Personal Communications, 2004.
6. Csinger, A. User Models for Intent Based Authoring Computer Science, The University of British Columbia, Vancouver, 1995, 202.
7. Davies, N. and Gellersen, H.-W. Beyond Prototypes: Challenges in Deploying Ubiquitous Systems. IEEE Pervasive Computing Journal, 1 (January-March 2002). 26-35.
8. Distributed Multimedia Research Group. The Guide Project, 2004.
9. Distributed Systems Technology Centre. Elvin - Content Based Messaging Systems, 2003.
10. Domnitcheva, S., Location Modeling - State of the Art Challenges. in Ubiquitous Computing 2001, (2001).
11. Fox, D., Hightower, J., Kautz, H., Liao, L. and Patterson, D.J., Bayesian Techniques for Location Estimation. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003).
12. Ganser, E., Koutsofios, E. and North, S. Drawing Graphs with dot, 2002.
- 32 -
13. Glassey, R.J. and Ferguson, R.I., SpaceSemantics: An Architecture for Modeling Environments. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003).
14. Harter, A., Hopper, A., Steggles, P., Ward, A. and Webster, P. The Anatomy of a Context-Aware Application. Wireless Networks, 8. 187-197.
15. Hightower, J., From Position to Place. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003), 10-13.
16. Kautz, H., Etzioni, O., Fox, D. and Weld, D. Foundations of Assisted Cognition Systems, UW CSE Technical Report, 2003.
17. Kautz, H., Fox, D., Etzioni, O., Borriello, G. and Arnstein, L., An Overview of the Assisted Cognition Project. in AAAI-2002 Workshop on Automation as Caregiver: The Role of Intelligent Technology in Elder Care, (2002).
18. Kay, J., Kummerfeld, B. and Carmichael, D. Consistent Modelling of users, devices and environments in a ubiquitous computing environment, University of Sydney, 2004.
19. Kay, J., Kummerfeld, B. and Lauder, P., Personis: A Server for User Models. in Adaptive Hypertext 2002, (2002), Springer, 202-212.
20. Kidd, C.D., Orr, R., Abowd, G.D., Atkeson, C.G., Essa, I.A., MacIntyre, B., Mynatt, E., Starnet, T.E. and Newstetter, W., The Aware Home: A Living Laboratory for Ubiquitous Computing Research. in Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, (1999), Springer-Verlag Lecture Notes in Computer Science, 191-198.
21. Kobsa, A. Modeling the User's Conceptual Knowledge in BGP-MS, a User Modeling Shell System. Computational Intelligence, 6. 193-208.
22. Kobsa, A. User Modeling: Recent Works, Prospects and Hazards, University of Konstanz, 1993.
23. Kobsa, A. Generic User Modeling Systems. User Modeling and User-Adapted Interaction, 11. 49-63.
24. Koile, K., Tollmar, K., Demirdjian, D., Shrobe, H. and Darrell, T., Activity Zones for Context-Aware Computing. in Ubicomp, (2003).
25. Lee, S.-W. and Mase, K. Activity and Location Recognition Using Wearable Sensors. IEEE Pervasive Computing Journal, 1 (July-September 2002). 24-32.
- 33 -
26. Mantara Software Elvin Administrator's Guide, 2003.
27. Orwant, J. Doppelganger Goes to School: Machine Learning for User Modeling Computer Science and Engineering, Massachusetts Institute of Technology, 1991, 89.
28. Orwant, J. Machine Learning for User Modeling, Unpublished, 1992.
29. Orwant, J., Heterogeneous Learning in the Doppelganger User Modeling System. in User Modeling and User-Adapted Interaction, (1995), 107-130.
30. Orwant, J. For want of a bit the user was lost: Cheap user modeling. IBM Systems Journal, 35. 398-416.
31. Patterson, D.J., Etzioni, O. and Kautz, H., The Activity Compass. in First International Workshop on Ubiquitous Computing for Cognitive Aids, (Gothenberg, Sweden, 2002).
32. Patterson, D.J., Liao, L., Fox, D. and Kautz, H., Inferring High-Level Behavior from Low-Level Sensors. in Fifth International Conference on Ubiquitous Computing, (2003).
33. Pohl, W., Machine Learning for User Modeling. in Design of Computing Systems: Social and Ergonomic Considerations (Proceedings of the Seventh International Conference on Human Computer Interaction), (Elsevier, Amsterdam, 1997), 27 - 30.
34. Pohl, W. and Nick, A. Machine Learning and Knowledge Representation in the LaboUr Approach to User Modeling.
35. Richardson, T., Bennett, F., Mapp, G. and Hopper, A. Teleporting in an X Window System Environment. IEEE Pervasive Compunications Magazine, 1 (3). 6-12.
36. Shafer, S.A.N., Location Authorities for Ubiquitous Computing. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003).
37. Shavlik, J.W. and Dietterich, T.G., General aspects of machine learning. in Readings in Machine Learning, (San Mateo, California, 1990), Morgan Kaufmann Publishers Inc, 1-10.
38. Wang, J. and Mankoff, J., Theoretical and Architectural Support for Input Device Adaption. in ACM Conference on Universal Usability, (Vancouver, British Columbia, Canada, 2003).
39. Want, R. and Hopper, A. Active Badges and Personal Interactive Computing Objects. IEEE Transactions on Consumer Electronics (February 1992).
- 34 -
40. Webb, G.I., Pazzani, M.J. and Billsus, D. Machine Learning for User Modeling. User Modeling and User-Adapted Interaction, 11. 19-29.
41. Weiser, M. The Computing for the 21st Century. Scientific American, 265 (3). 94-104.
42. Young, S., Evermann, G., Hain, T., Kershaw, D., Moore, G., Odell, J., Ollason, D., Povey, D., Valtchev, V. and Woodland, P. The HTK Book, 1999.
43. Zukerman, I. and Albrecht, D.W. Predictive Statistical Models for User Modeling. User Modeling and User-Adapted INteraction, 11. 5-18.
- 35 -