chip project: personalized museum tour with real-time adaptation on a mobile device with multi-point...

Personalized Museum Tour with Real-

Time Adaptation on a Mobile Device

with Multi-Point Touch Interface

Master Thesis by Ivo Roes (0527673)

Graduation Committee:

Dr. L.M. Aroyo

Dr. A.J. Mooij

Prof. Dr. P.M.E. de Bra

Introduction

• Current museum tours

• Information on paper

• Audio guides

• Human tour guides

• “One size fits all” mentality

• Users want the best

experience in the available

time

/ Business Information Systems PAGE 1July 2010

Overview

• Context

• Related Work

• Problem Definition & Research Questions

• Requirements

• Design

• Adaptation Strategies

• Localization

• Demo

• Conclusion & Further Research

• Questions


Context – Project

• CHIP project

• CATCH program


Context – Existing Implementation

• Personalized Museum Tour on

a Mobile Device (PMTMD) 1

• Offline personalized PDA-tour

• User builds online profile based on

rating artworks

• Synchronization at beginning and

end of tour

• User positioning based on RFID

tags and readers


1 Master Thesis Personalized Museum Tour on A Mobile Device (PMTMD)

Van Sambeek, R and Schuurmans, . TU/e 2007

Related Work

• GUIDE

• Sotto Voce

• AgentSalon

• PEACH

• HIPPIE

• The Kubadji Project


Problem Definition

• Existing Implementation (PMTMD)

• Offline Tour

• Hardware Platform

• Static user positioning


Research Questions

1. What user, collection and spatial aspects play a

role in the real-time adaptation of a mobile museum

tour?

2. In what manner can the visitor be precisely

positioned in real-time and how can this

information be used in tour adaptation?


Requirements

• Tour

• Adaptation

• Localization

• System

• Platform independent implementation

• Non-intrusiveness to the user

• Non-intrusiveness to the environment


Design – General Scenario

• User creates online profile

• User visits physical museum

• User selects (personalized) tour

• User configures the tour to his / her preferences

• User starts tour

• During tour user can view artwork description, concepts and related

artworks.

• The user can rate artworks / concepts

• The user can add (similar) artworks

• At any moment the user can view his location on the map


Design – Use Cases

• UML Case Diagrams


Rate Artwork / Concept Use Case

Design – System Architecture

PAGE 11July 2010

Design – Graphical User Interface

• Obstacles to tackle

• Small screen

• Rough finger touch instead of

precise stylus or mouse

• Requirements: quick

response, intuitive, easy

navigation

• Design Decisions

• Button menu bar

• Small text font

• Separation of functionalities

• AJAX communication

• Adaptive Interaction


Adaptation Strategies

• CHIP mobile guide adapts

to individual user on three

levels:

• Based on time constraints

• Based on artwork preference

• Based on spatial information


Adaptation Strategies – time constraints (1)

• Usually museum visitors wish to spend a certain

amount of time in the museum

• At tour configuration users can specify #artworks and

#minutes

• Each user has a different average time for viewing an

artwork tv (continously updated)

• CHIP mobile guide guards: #artworks x tv = #minutes


• To guard the described constraint, the tour needs to

be adapted:


Adaptation Strategies – time constraints (2)

1. Order AWremain according to belief value in descending order

2. If (#current > #artworks) new AWremain = first (#artworks - #Visited) of AWremain

3. Else if (#current = #artworks) skip

4. Else new AWremain = AWremain + (#artworks - #current) recommendations

5. Order new AWremain according to belief value in descending order

Adaptation Strategies – artwork preference (1)

• Rating of artworks and concepts on a

5-star scale [-1.0, -0.5, 0.0, 0.5, 1.0]

• A belief value is calculated for all

artworks and concepts

• Belief value of concepts is built up from

child concepts

• Belief value for artworks is built up

from concepts


Adaptation Strategies – artwork preference (2)

• Rating a concept or artwork triggers:


1. Calculate Beliefc for all concepts c

2. Calculate Beliefa for all artworks a

3. Order { AWall \ AWvisited } based on belief value in descending order

4. New AWremain = first #remain elements of sorted { AWall \ AWvisited }

Adaptation Strategies – spatial information

• When adapting the tour, it is desirable that users are not

sent back to previous rooms

• This can be achieved by extending the algorithm:


1. Order AWremain according to belief value in descending order

2. If (#current > #artworks) new AWremain = first (#artworks - #Visited) of AWremain

3. Else if (#current = #artworks) skip

4. Else new AWremain = AWremain + (#artworks - #current) recommendations

(artworks from previous rooms excluded)

5. Order new AWremain according to belief value in descending order

6. Order new AWremain according to location in museum (closest first)

Adaptation Strategies – Validation (1)

• Validation of the Artwork Recommender

• Study 2 performed by CHIP team into relevant semantic

relations

• Test group rated artworks / concepts

• After presentation of recommendations, users could

rate the semantic relation used as a basis for the

recommendation

• Best performing semantic relations were based on

creator and style of an artwork

• Semantic relations based on geographic location

performed the worst


2 Semantic Relations for Content-based Recommendations, Wang, Y et al.,

In Proc. International Conference on Knowledge Capture (K-cap), 2009, Vol. September 2009


• Comparative Analysis with SpaceCHIP 3

• SpaceCHIP extends the mobile guide by implementing a

routing mechanism based on the SWI-Prolog Space package

• Main goals: minimize walking distance and take themed rooms

into account

• Introduction of connectivity graph with nodes consisting of

artworks, rooms, doors, hallways and stairs

• Set transition weights of edges to experienced distance

instead of actual distance

• Using a nearest neighbor search a suboptimal shortest tour

through the museum is calculated.

• Calculation time is exponentially related to number of artworks


3 Finding Your Way through the Rijksmuseum with an Adaptive Mobile Museum Guide

W.R. van Hage et al., Heraklion, Greece : 7th Extended Semantic Web Conference, 2010


• Both the CHIP mobile guide and SpaceCHIP aim at

minimizing user walking distance and take themed

rooms into account

• The approach for reordering the tour differs slightly

• CHIP mobile guide orders remaining tour based on the

room where artworks are located

• SpaceCHIP uses experienced distance instead of actual

distance and therefore also takes into account

unfavorable transitioning to a different floor

• Further research is needed to determine which

method results in better remaining tours


Localization

• In the exisiting implementation, localization was

achieved using RFID scanners and tags

• No real-time location information was available

• In the CHIP mobile guide we set out to see what

possibilities exist for real-time localization

• Requirements that must hold: platform-independent

implementation and non-intrusiveness.


Localization – Software

• From a literature study, a Radio Frequency

Fingerprinting technique was chosen as a basis for

localization.

• Radio Frequency Fingerprinting

• Based on Received Signal Strength Indicator (RSSI)

• Calibration phase, measure RSSI values to all access

points for predefined set of points in the space. Use

average values to generate frequency radio map

• Online phase, compare live measurement of RSSI

values to all access points and compare to map


Localization – Hardware technique

• RFID

• Bluetooth

• Infrared

• Ultra Wide Band (UWB)

• WiFi


Localization – Hardware architecture (1)


Localization – Hardware architecture (2)

PAGE 26July 2010

Localization – Test setup (1)


3x Linksys WRT54GL router

DD-WRT firmware v2.4 4

Wi-viz 2.0 5

Wireless Network Visualization

4 http://www.dd-wrt.com5 http://devices.natetrue.com/wiviz/


• Test results


Demo


Conclusion

• Focus on what aspects play a role in real-time

adaptation of a museum tour and real-time

localization of a visitor in the museum

• Adaptation in the CHIP mobile guide takes place at

three levels:

• Based on time constraints

• Based on artwork preference

• Based on spatial information

• Localization was implemented using a WiFi Radio

Frequency Fingerprinting technique, resulting in an

accuracy of up to 1,25 m


Further Research

• Graphical User Interface evaluation

• Scalability of localization technique

• Social aspects


Questions?


chip project: personalized museum tour with real-time adaptation on a mobile device with multi-point...

Technology

personalized tour user

mobile museum tour

end of tour user positioning

individual user

preferences user

user nonintrusiveness

tour configuration users

design general scenario