smArt: Open and Interactive Indoor Cultural Data

Andrea Ferracani, Daniele Pezzatini, Alberto Del Bimbo, Riccardo Del Chiaro, FrancoYang, Maurizio Sanesi

Università degli Studi di Firenze - MICCFirenze, Italy

[name.surname]@unifi.it

ABSTRACTIn this demo we present smArt1, a low-cost framework toquickly set up indoor exhibits featuring a smart navigationsystem for museums. The framework is web-based and al-lows the design on a digital map of a sensorized museumenvironment and the dynamic and assisted definition of themultimedia materials and sensors associated to the artworks.The knowledge-base uses semantic technologies and it is ex-ploited by museum visitors to get directions and to havemultimedia insights in a natural way. Indoor localisationand routing is provided taking advantage of active and pas-sive sensors advertisements and user interactions. In thisway we overcome the Global Positioning System (GPS) un-availability issue in indoor environments.

Categories and Subject DescriptorsH.4 [Information Systems Applications]: Miscellaneous;J.5 [Computer Applications]: Arts and Humanities—Ar-chitecture

General TermsTheory, Design, Cultural Data Curation, Installations

KeywordsCultural heritage, web applications, beacon, sensors, OpenLinked Data, Museum guide

1. CONTEXT AND MOTIVATIONThe problem of linking physical spaces with structured

data is urgent considering from the one hand the opportu-nities offered by the evolution of the semantic web and fromthe other the increasing adoption of the so-called Internet-Of-Things (IOT). Furthermore, managers in the culturalheritage need easy to use tools to promote, curate and pub-lish cultural data that may be exploited at several levels by

1Demo video: https://vimeo.com/miccunifi/smart

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for third-party components of this work must behonored. For all other uses, contact the Owner/Author(s). Copyright is held by theowner/author(s).MM’15, October 26–30, 2015, Brisbane, Australia.ACM 978-1-4503-3459-4/15/10.DOI: http://dx.doi.org/10.1145/2733373.2807981.

students, tourists, professionals, researchers and so on. Tothis end tools have to provide museum curators with facili-ties for artworks’ search, browsing and collection, and not atleast the opportunity to make cultural resources available aspublic structured data on the web. At the same time, nowa-days, real spaces can easily be made reactive and low-costsolutions are desirable. smArt fulfills these requirements andprovides a tool where cultural resources can be browsed andpublished, from and on the web, using a big and extendablerepository (i.e. DBPedia dataset2). Data can be enrichedwith sensors information for ad hoc deployable installations.In details, the web application allows to associate artworksin a semi-automatic way with different types of sensors andfeatures proximity sensing and routing in an indoor envi-ronment. This is not trivial: in fact while tourism electronicguides for outdoor are widespread and provide access to con-textual multimedia data relying on GPS technology, thingsare more difficult in indoor where GPS is not available. Sub-room indoor localisation is an active area of research whichincludes applications in reactive indoor spaces. smArt ex-ploits Bluetooth Low Energy (BLE) beaconing, in synergywith other tools, as an indoor positioning and routing tech-nology. Bluetooth Estimote Beacons3 and automatic gen-erated QR codes can be automatically and manually asso-ciated to artworks or locations in the knowledge-base andthen physically positioned. Cited technologies are low orwithout cost and don’t involve an infrastructural overhead:two common requirements for cultural public institutions.

2. THE SYSTEMThe system is mainly composed by two modules: 1) a

web-based application for the semi-automatic ingestion andmanagement of Open Linked Data regarding the cities ofVenice, Rome and Florence in Italy; 2) a mobile Androidapplication which exploits the data generated by the webapp and reacts to the expected signals in the real environ-ment.

2.1 Configuring the environment: the Web Ap-plication

The web application is used for the ingestion, creationand management of data concerning museums. It providesa graphical user interface for building data and componentsmashups in order to configure a sensorized environment.Users are enabled to ‘pipe’ several interface components and

2See http://bit.ly/1HxOUVI3See http://bit.ly/1d7dZdB

then set up rules for how content should be modified. Thereare three main components: 1) City Component: it allowsto choose cities from which to select public multimedia data;2) Museum Component: it is used to collect, aggregate andenrich data about museums and artworks from DBPediathrough Open Linked Data; 3) Sensor Component: it canbe applied to artwork collections in order to automaticallyassociate physical sensors. Components are managed andorganised on the interface using a drag-and-drop paradigm.A component is shown as a circle icon with a label and animage. When dragged on other components icons can makeappear contextual menus in order to apply modifiers. TheMuseum Component provides contextual panels to searchmuseums, select and associate sensors to artworks (i.e. Bea-cons or QR codes). Furthermore, for each museum the usercan: 1) interactively draw the museum map of the environ-ment to be sensorized; 2) decide the location of the artworks;3) define the access point to the museum halls and finally 4)mark out all the trajectories that visitors can use to reach theartworks. All this infos are then used by the mobile applica-tion to provide localisation and routing systems to museumvisitors. The web app has been developed in HTML5 forthe client and uses PHP and MySQL on the server for meta-data storing. Storing and communication with the seman-tic knowledge-base is performed through RDF and SparqlQueries to a self-hosted DBPedia endpoint. The knowledge-base is reachable on a self-hosted Virtuoso Server4 and ituses triple-store dumps of Wikipedia frequently and auto-matically updated.

2.2 Mobile App, Localisation and RoutingsmArt mobile application allows the visitor to localise her-

self in the indoor museum and provides a routing systemto guide the user to artworks of interest on the basis ofthe data generated by the web application. The app hasbeen designed with the aim to enhance the user experienceof a visitor approaching or searching for an artwork andprovides multimedia insights exploiting natural interactionparadigms. It is well known that GPS is not working inindoor locations due to the poor signal coverage. Further-more, indoor localisation is particularly challenging for sev-eral reasons: presence of obstacles and moving people, in-terference caused by other electronic devices etc. Standardsolutions contemplate active (QR code scanning, NFC) andpassive sensors (beacon bluetooth for proximity detection ortriangulation for exact location). smArt exploits bluetoothbeacons which are cheap and well supported and require alow level of interaction. As an alternative each artwork canbe automatically or manually associated to QR codes whichhave no cost but need more user participation.The app has been developed as an Android application anduses an SQL Lite database generated by the web applica-tion and stored on a server. The interface has been de-signed following the Google guidelines for material design.The main interface is map based and provides outdoor nav-igation. Through a sliding-up panel the user can check hislocalisation and browse all the nearby museums where aninteractive exhibit has been set up. Google directions arealso provided. When a user approaches a museum she isnotified on the app interface of the possibility to switch toan indoor map visualisation. The map is rendered in real-time on the device using canvas and vector shapes. Zoom

4See http://bit.ly/1BqKEVP

and drag gestures are enabled. Artworks equipped with asensor are visualised on the map as icons. Once the user haslocalised herself or the app has identified her location, theuser can select any artwork on the map or using the slid-ing panel in order to be suggested with the shortest pathto it. User location is acquired when the app receives theunique identifying information broadcasted by a beacon viabluetooth or when the user scans a QR code associated withan artwork or provided as a localisation hotspot in the mu-seum. A background service is always active and listens toBLE advertisements. When the device receives the signal,the user is notified and contextual artwork multimedia dataare shown.The indoor engine is in charge to draw and manage themap and the navigation system: paths are computed mod-elling the information about museum rooms and artworksas a graph of traversable spots. Spots have been definedand arranged by the web app user on the map and can besensorized artworks, path spot, door spot or museum visitorlocalisation spots. The shortest path to an artwork is esti-mated on the fly by the indoor engine using the Dijkstra’salgorithm and then visualised on the map. The engine pro-vides also a completely automatic system to calculate theshortest path in the case that the web app user has notmarked the path spots required to navigate from an artworkor a localisation in a room to another. This is achieved usingautomatic 2D polygon convex partitioning of the museummap: the Hertel-Mehlhorn algorithm is exploited which isnever worse than 2r + 1 pieces, where r is the number ofreflex vertices. Once the polygon is partitioned the centerof mass of each partition is identified and treated as a pathspot to be used in the graph by the Dijkstra’s algorithm tobuild the itinerary from the user position to the artwork ofinterest. The algorithm allows to face situations where themap is a regular polygon but paths from an artwork or alocalisation in a room to a target artwork in another roomcould cross the museum hall walls resulting in a wrong feed-back to the user.

3. CONCLUSIONS AND FUTURE WORKIn this demo we present a web-based framework designed

for museum curators to manage and set up easily interactiveexhibits in a sensorized environment. Museums, artworksand associated multimedia materials are retrieved from andsaved to the web using Open Linked Data. Artworks can beplaced on the map and associated with cheap and easy toinstall actuators to be placed in the real museum environ-ment. Exploiting these data museum visitors equipped withan ad hoc mobile app can enjoy interactive exhibits in aneffective and natural way. Future work will focus on the re-finement of methods for the reduction of localisation errorsusing BLE technology. A good starting point are the resultsin [1].

4. REFERENCES[1] P. Martin, B.-J. Ho, N. Grupen, S. Munoz, and

M. Srivastava. An ibeacon primer for indoorlocalization: Demo abstract. In Proceedings of the 1stACM Conference on Embedded Systems forEnergy-Efficient Buildings, BuildSys ’14, pages190–191, New York, NY, USA, 2014. ACM.