Location based Applications for Mobile Augmented Reality

Gerhard Reitmayr Dieter Schmalstieg

Vienna University of TechnologyFavoritenstr. 9-11/188/2,A1040 Vienna, Austria

Email: {reitmayr|schmalstieg}@ims.tuwien.ac.at

Abstract

In this work we investigate building indoor location based ap-plications for a mobile augmented reality system. We believethat augmented reality is a natural interface to visualize spa-cial information such as position or direction of locations andobjects for location based applications that process and presentinformation based on the user’s position in the real world. Toenable such applications we construct an indoor tracking sys-tem that covers a substantial part of a building. It is based onvisual tracking of fiducial markers enhanced with an inertialsensor for fast rotational updates. To scale such a system toa whole building we introduce a space partitioning scheme toreuse fiducial markers throughout the environment. Finally wedemonstrate two location based applications built upon thisfacility, an indoor navigation aid and a library search applica-tion.

Keywords: Augmented Reality, Mobile Computing,Wearable Computing, Location based Computing,Tracking.

1 Introduction

Augmented reality (AR) is a powerful user inter-face technology that augments the user’s environ-ment with computer generated entities. Azumapoints to three important aspects that define AR(Azuma 1997). It blends the real and virtual withina real environment, is real-time interactive and regis-tered in 3D. In contrast to virtual reality which com-pletely replaces the real world, augmented reality dis-plays virtual objects and information along with thereal world registered to real world locations.

Location based systems take the user’s locationinto account when processing and presenting infor-mation to the user. While augmented reality systemscan be viewed as falling into this category, locationbased systems become interesting when the supportedrange of locations expands beyond a single labora-tory room. There is a wealth of work regarding suchtypes of applications within the wearable and ubiq-uitous computing area. Both can make good use ofaugmented reality to visualize abstract and spacialinformation as described in (Starner, Mann, Rhodes,Levine, Healey, Kirsch, Picard & Pentland 1997).

To employ AR in a large environment mobile sys-tems were built that support the graphical and com-putational demands of it. Examples of such develop-ments are the Touring machine by (Feiner, MacIntyre,Hollerer & Webster 1997) or the Tinmith system by(Piekarski & Thomas 2001). The Touring machine isalso a good example of a location based application

Copyright c©2003, Australian Computer Society, Inc. This pa-per appeared at Fourth Australasian User Interface Conference(AUIC2003), Adelaide, Australia. Conferences in Research andPractice in Information Technology, Vol. 18. Robert Biddleand Bruce Thomas, Ed. Reproduction for academic, not-forprofit purposes permitted provided this text is included.

Figure 1: A user navigates through a building guidedby the heads-up display graphics providing him withdirection.

that presents the user information about buildingsthey are looking at.

Most AR based applications so far either reliedon high quality tracking with a small environmentsuch as a laboratory room with dedicated track-ing equipment or on GPS technology that deliverspositional information world wide but works onlyoutdoors and with low accuracy and update rate.There are just a few solutions such as the BAT sys-tem (Addlesee, Curwen, Hodges, Hopper, Newman,Steggles & Ward 2001) that allow position trackingindoors covering a large environment with reasonableaccuracy.

We are interested in visualizations of spacial datausing augmented reality. To investigate such applica-tions for a mobile user we developed a mobile ARsystem (Reitmayr & Schmalstieg 2001a). A natu-ral step beyond the research lab is the building weare residing in. Therefore we developed and deployeda tracking system to support building wide location

based applications and that would deliver reasonablequality. This then became the basis of two locationbased applications we present in this work.

The first application is a navigation guide calledSignpost that supports the user in finding a desiredlocation within our building. It provides her withdirectional hints and highlights exits of rooms to betaken to proceed towards the destination. The secondapplication is a a search and retrieval application thathelps a library visitor to locate a desired book or itsdesignated place in the library bookshelves.

The remainder of the paper is structured as fol-lows. The next section will describe related work inthe area of tracking and navigation applications forAR systems. Then we give a short overview of ourmobile AR platform. Next we describe our trackingsystem in detail and follow with a description of ourdemonstration applications. We finish with a discus-sion of the results and some outlook to future work.

2 Related work

2.1 Tracking

Location tracking is a prerequisite for any locationaware application. In order to be able to provide theuser with services and information related to her loca-tion, a system needs to sense or otherwise be told thecurrent location of the user. Augmented reality ap-plications require a very accurate position tracking toregister visual information accurately with the user’senvironment. Otherwise the augmented informationmay be positioned incorrectly resulting in a confusingor unusable user interface.

There is a wealth of work related to position track-ing ranging from indoor systems covering a room sizearea to outdoor systems supporting the entire planet.This diversity is also present in the accuracy of track-ing systems ranging from millimeters to several me-ters.

Outdoors, GPS provides global position with ac-curacy between several meters and centimeters de-pending on additional techniques such as broadcast-ing correction signals or using the phase informationof the received signals to improve the accuracy. How-ever, GPS requires a direct line of sight to severalsatellites and therefore is not working properly insidebuildings or in areas covered by trees or tall buildings(appropriately termed ’city canyons’).

Indoors, tethered tracking systems using mag-netic (Raab, Blood, Steiner & Jones 1979), ultrasonic(Intersense 2002) and optical technologies (Welch,Bishop, Vicci, Brumback, Keller & Colucci 2001)achieve high accuracy in the millimeter to centime-ter range. These systems are typically able to covera room and require installations of large devices ordense arrays of beacons or sensors mounted in thecovered area. Another research system (Addleseeet al. 2001) can cover a whole building but is notavailable to the public.

A large class of tracking solutions uses computervision to track the movement of a camera. Somesolutions place fiducial markers (Rekimoto 1998) inthe environment to achieve good accuracy. Thereis also experimental work that uses marker free vi-sion based tracking by selecting salient natural fea-tures in the environment (Pinz 2001) or comparingthe camera’s view with prerecorded imagery (Kourogi& Sakaue 2001).

Other approaches try to use local sensors and deadreckoning techniques to compute a user’s location(Lee & Mase 2001). However these are prone to accu-mulation of subsequent errors in their computationsunless they are synchronized with absolute position-

ing systems. Including knowledge about the environ-ment in the form of geometric models and accessibil-ity graphs (Hollerer, Hallaway, Tinna & Feiner 2001)allows increasing the accuracy of such approaches sig-nificantly.

Our contribution is relying on using a set of trainedfiducial markers together with additional knowledgeabout the environment. This allowed us to improvethe performance of the optical tracking component byreducing the number of markers required for a stableoperation of the system. A similar solution for in-door tracking has been attempted (Thomas, Close,Donoghue, Squires, Bondi, Morris & Piekarski 2000),however without an optimization of the reuse of mark-ers.

2.2 Navigation applications

Augmented reality as a user interface for mobile com-puting is particularly powerful when the computerhas access to information on location and situation,so it can provide contextual information. A promi-nent example that served as an inspiration to ourapproach is Columbia’s ”Touring Machine”, whichis used as a mobile multimedia information system(Feiner et al. 1997) and journalistic tool (Hollerer& Pavlik 1999). Other mobile applications includetourist information tools (Satoh, Hara, Anabuki, Ya-mamoto & Tamura 2001) and model constructing ap-plications for indoor (Baillot, Brown & Julier 2001)or outdoor use (Piekarski & Thomas 2001).

Examples of navigation applications are alsofound. Some of them concentrate on outdoor useand rely on GPS tracking. They display such in-formation as direction and distance to the next waypoint of a path (Thomas, Demczuk, Piekarski, ep-worth & Gunther 1998), or directions of objects thatare augmented with further information (Suomela &Lehikoinen 2000). Indoors there are applications thatprovide models and directional hints for navigation(Hollerer et al. 2001) and offer also additional infor-mation such as visualizing the path a user has taken(Newman, Ingram & Hopper 2001).

3 The mobile AR setup

While the computational power for stereoscopic ren-dering and computer vision is becoming available inmobile computer systems, the size and weight of suchsystems is still not optimal. Nevertheless, our setupis solely build from off-the-shelf hardware componentsto avoid the effort and time required for building ourown (Reitmayr & Schmalstieg 2001a). On one handthis allows us to quickly upgrade old devices or addnew ones and to change the configuration easily. Onthe other hand we do not obtain the smallest andlightest system possible.

3.1 Hardware

Our current setup uses a notebook computer witha 2Ghz processor and a powerful NVidia Quadro4graphics accelerator. It operates under Windows XP.We also added a wireless LAN network adapter to thenotebook to enable communication with our station-ary setup or a second mobile unit. The equipmentis mounted to a backpack worn by the user. As anoutput device, we use a Sony Glasstron see-throughstereoscopic color HMD. The display is fixed to a hel-met worn by the user. Moreover, an InterSense Inter-Trax2 orientation sensor and a web camera for fiducialtracking are mounted on the helmet (see Figure 2).

The main user interface consists of two self madepinch gloves and a wrist mounted touch pad. These

a) b)

Figure 2: The mobile AR setup. (a) The helmet withan HMD, a miniature camera and an inertial sensormounted to it. (b) The notebook computer and pe-ripheral devices mounted on a backpack.

are equipped with optical markers to be tracked bythe head mounted camera (see Figure 3). We con-structed a rigid frame to mount three markers aroundthe wrist to allow a wide range of hand postures tobe reliably tracked.

As gloves we use robust finger-free bicycle glovesand cover only the thumb with two layers of cottonthat have a pressure sensitive foil embedded at theinner side tip of the thumb. The pressure sensor actsas a button. Due to the relatively fixed offset fromthe back of the hand to the thumb tip when thumband index finger are pressed together, the 3D cursorcan easily be calculated. Four fingers are left free tomanipulate physical objects and to operate the touchpanel that does not respond to skin covered in cloth.This design is similar to the work of Piekarski andThomas (Piekarski & Thomas 2002). However we fo-cus on 3D interaction and require a larger range ofpostures to be tracked. We also use only one buttonas we do not rely on a complex menu system.

Figure 3: The interaction props of our setup. Thegloves are equipped with 3 markers to be able to trackthe hands in a wide range of orientations.

The touch panel is mounted to the wrist with anelastic band, and also has a marker attached. Whilethe graphical overlay of the panel is only presentedwhen it is in the users field of view, the panel opera-tion itself does not rely on the tracking of the panelmarker, and can be done while looking at somethingelse.

3.2 User interface management software

As our software platform we use Studierstube(Schmalstieg, Fuhrmann, Hesina, Szalavari, Encar-nao, Gervautz & Purgathofer 2002), a user interfacemanagement system for AR based on, but not limitedto stereoscopic 3D graphics. It provides a multi-user,multi-application environment, and supports a vari-ety of display devices including stereoscopic HMDs. Italso provides the means of 6DOF interaction, eitherwith virtual objects or with user interface elementsregistered with and displayed on the pad.

Applications are implemented as runtime loadableobjects executing in designated volumetric contain-ers, a kind of 3D window equivalent. While the orig-inal stationary Studierstube environment allowed auser to arrange multiple application in a stationarywork-space (see Figure 4 for a more elaborate exam-ple), our mobile setup allows to arrange 3D informa-tion in a wearable workspace that travels along witha user. It supports different presentation mechanismsthat have been identified as useful for AR (Feiner,MacIntyre, Haupt & Solomon 1993, Billinghurst,Bowskill, Morphett & Jessop 1998): Head-stabilized,where information is fixed to the users viewpoint,Body-stabilized, where information is fixed relative tothe users body position, and World-stabilized, whereinformation is registered with real world locations. Inthis setup we do not distinguish between the user’shead and body.

Figure 4: Two users working in a shared workspace onan application to design stage sets. They are workingwith virtual geometry as well as projections of specialviews of the environment. The Studierstube frame-work provides 3D windows and widgets and supportsmultiple applications and output devices.

3.3 Tracking support

Mobile AR requires significantly more complex track-ing than a traditional VR application. In a typicalVR or AR application tracking data passes througha series of steps. It is generated by tracking hard-ware, read by device drivers, transformed to fit therequirements of the application and send over net-work connections to other hosts. These tasks arehandled by the OpenTracker library (Reitmayr &Schmalstieg 2001b), an open software architecture forthe different tasks involved in tracking input devicesand processing multi modal input data.

The main concept behind OpenTracker is to breakup the whole data manipulation into these individual

steps and build a data flow network of the transfor-mations. The framework’s design is based on XML,taking full advantage of this new technology by allow-ing the use of standard XML tools for development,configuration and documentation.

OpenTracker uses a vision tracking library calledARToolkit (Billinghurst & Kato 1999) to implementthe tracking of the fiducial markers on the interactionprops or in the environment. It analyzes the videoimages delivered by the web camera mounted to thehelmet and establishes the position of known markersrelative to the camera. This information is then usedto either track the interaction props such as glovesand the tablet relative to the user or to track the userwithin the environment.

4 Indoor tracking

To build an environment where we could test driveour mobile AR kit, we implemented an indoor track-ing solution to cover a floor of our building. Aswe did not have access to a proprietary building-wide positioning infrastructure (such as AT&T Cam-bridge’s BAT system used by (Newman et al. 2001)),we choose to rely on a hybrid optical/inertial trackingsolution. This approach proved very flexible in termsof development of positioning infrastructure, but alsopushes the limits of what ARToolkit tracking can pro-vide.

Figure 5: This diagram shows the geometric modelof our floor. The (red) dots denote the locations ofmeasured markers used to track the user within theenvironment.

To implement a wide area indoor tracking solutionwe resolved to use a set of well-known markers thatwere distributed in the environment. Together witha geometric model of the building that includes thelocation of the well-known markers (see Figure 5) we

Figure 6: This diagram shows the different coordi-nate systems involved to compute the user’s position.U is the user’s position and orientation, R the roomcoordinate system, M a markers position and orien-tation within that room and W the world or modelcoordinate system.

can compute the user’s location as soon as a marker istracked by the optical tracking system. The model isstructured into individual rooms and the connectionsbetween these rooms called portals. The location of aroom within a world coordinate system completes themodel. These models and the location of the markerswere obtained by tape measurements.

A measurement of a marker by the optical track-ing returns the markers position and orientation TUMwithin the user’s coordinate system U . This is es-sentially the transformation to convert coordinatesfrom the system U into the system M . InvertingTUM gives the user’s position and orientation withinthe marker’s coordinate system M as TMU = T−1

UM .By combining the fixed and measured transformationTRMbetween the room coordinate system R and themarker’s M with the value TMU , we calculate theuser’s position and orientation TRU = TRM · TMUwithin the room coordinate system R. This informa-tion was then used to compute the direction of theindication arrow and to render the augmentation ofportal and the room’s geometry.

In addition to a geometric model of the environ-ment we also constructed an adjacency graph for theindividual rooms (see Figure 7). This graph was usedto provide the path finding functionality described insection 5.1 and to optimize the use of markers as de-scribed in the next section.

4.1 Marker reuse

The implemented tracking approach requires a largeset of markers. It is necessary to place a marker aboutevery two meters and to cover each wall of a sin-gle room with at least one marker. Deploying thisin our floor covering about 20 rooms and long hall-ways would require several hundred different markers.However, marking up a large indoor space with uniqueARToolkit markers is not feasible for two reasons:

• The more markers, the higher the degree of sim-ilarity of markers will be. Near rotational sym-metry can become a major problem when a largeamount of markers is used. Additionally light-ing conditions vary often from one room to an-

other. All this leads to inferior recognition accu-racy. For a larger set of markers this implies ahigher number of false recognitions.

• A large set of markers makes the search spacethat ARToolkit has to traverse larger, leading tosignificant decrease in performance.

As a consequence, it is not possible to scale the useof ARToolkit to arbitrary large marker assemblies.

Figure 7: A part of the floor plan of our buildingwith rooms colored differently. The resulting adja-cency graph is overlayed with black edges giving por-tals between rooms and grey edges giving line-of-sightconnections.

To overcome this restriction, we developed a spa-tial marker partitioning-scheme that allows reusingsets of markers within the given environment. Theidea behind this approach is that, if the tracking sys-tem knows the user’s location, it can rule out largeparts of the building because they will not be visibleto the camera. Therefore, for two areas, which arenot visible to each other, it becomes possible to usethe same set of markers. This problem is equivalent toapproaches for indoor visibility computation based onpotentially visible sets (Airey, Rohlf & Brooks 1990).

We use the room definition in the geometric modelas the basic element of this approach. Then we canbuild an adjacency graph for all rooms using roomsas nodes and portals between rooms as edges. In ad-dition to the portals representing logical connectionswe also add further edges that represent a line-of-sightbetween to rooms (see Figure 7). We call this secondgraph the extended adjacency graph.

The next step is to generate a minimal number ofdisjoint sets of markers, such that each room is as-signed to a set of markers. This is similar to graphcoloring problems where a minimal set of colors isused to color nodes in a graph with certain restric-tions. The following constraints must be fulfilled toyield a useful distribution of marker sets.

• Two nodes connected in the extended adjacentgraph must have disjoint sets of markers. Oth-erwise the camera tracking system cannot decidewhich room the marker belongs to, if both in-stances are visible from one room.

• Looking at one node all adjacent nodes in the ex-tended adjacent graph must have disjoint markersets. This constraint has to be fulfilled by allnodes in the extended adjacency graph. Other-wise the common node provides a point of view

Figure 8: This figure shows the change of the trackingsystem when a user walks from one room into another.On the left, the user is in room A and therefore thetracking system only tracks markers of room A andB. When the user moves to room B, it also starts totrack markers of room C.

that allows sighting of a marker in two differentplaces.

Starting with a given correct room position, thesystem tracks the user within the current room andinto neighboring rooms. It can detect a room changeby comparing tracked markers with the sets of mark-ers assigned to the current room and its neighbors. Ifit recognizes a marker from a set of a neighbor room itassumes that the user entered this room, which thenbecomes the current room. Figure 8 gives a smallexample of the algorithm.

The left image shows that the current node A andits adjacent node B are marked active, when the useris in room A, while node C and D are inactive (gray).As soon as the user moves to room B a room changeis triggered and the state of the tracking system isupdated. Now the current room is room B and thestatus of both adjacent rooms A and C are switchedto active. Note that room D is still inactive, becausethere is no line of sight between room B and roomD. We could also reuse markers of room A in roomD as these rooms are not common neighbors of eitherroom B or C.

We extended the scheme by placing a single glob-ally unique marker in every room, which can be usedto reinitialize the system in case of an error. It is alsosuitable for the determination of the current room atstart up. The user just has to turn around until thesystem detects the unique marker and switches intointeractive mode. This unique marker set is disjointfrom any reusable marker set.

4.2 Inertial Tracker Fusion

Our mobile AR kit is also equipped with an inertialorientation tracker providing low latency updates onthe user’s head orientation. This information is usedto update the user’s view direction in between mea-surements from the optical tracking system. How-ever the tracking information is subject to drift thatcan lead to large errors after a short period of time.Therefore we need to correct the measurements ofthe inertial tracker. This is implemented similar to(Newman et al. 2001).

Every time we receive a measurement by the opti-cal tracking system, we compute the user’s true headorientation qoptical as described in section 4. Thenwe compute a correction orientation for the measured

Figure 9: (Top) The direction is indicated by an over-layed arrow. (Bottom) The wire frame augmentationof the room and the portals can be switched on.

inertial orientation qinertial as qcorrection = qoptical ·q−1inertial. This correction is than applied to any sub-

sequent measurement of the inertial tracker to providea correct user orientation quser = qcorrection · qinertial.

5 Applications

As examples of location-based applications we chosetwo typical tasks that appear in an academic build-ing. Firstly, we implemented a navigation applica-tion called Signpost that guides the user to a selecteddestination room. Secondly, we augmented a libraryby highlighting locations of books within the book-shelves.

5.1 Signpost - navigation application

Many visitors to our institute have a hard time tofind their way through the complicated and poorlylabelled corridors of our office building. We consid-ered this situation a useful test case for a location-based service provided through our mobile augmentedreality system. The system should guide a user onthe way through the building with direction and mapbased contextual information.

User interface

Interaction with the application is attained througha user interface displayed on the wrist pad. The userhas to select the desired destination room, either byclicking into a world in miniature model or by select-ing its description from a list. Then the applicationcomputes the shortest path from the current roomto the destination room (see section 5.1) and givesappropriate feedback. The user may change the des-tination room again at any moment.

The system continuously provides the user withtwo modes of visual feedback: a heads-up display withdirectional arrows and a world in miniature model ofthe environment. These two information displays aredescribed now in more detail.

Via the HMD a wire frame model of the currentroom is superimposed on top of the real scene. Theapplication uses a shortest path search on an adja-cency graph of the building to determine the nextdoor/portal on the way to the destination. Thedoors/portals are then always highlighted in white.The wire frame overlay can optionally be turned onand off by the user. In addition, a direction arrowshows the direction to the next door or portal (seeFigure 9), indicating either to move forward, turnleft/right or to make a U-turn.

In addition to that the application presents a worldin miniature model (WIM) of the building to the useron the augmented wrist pad (see Figure 10). It al-ways shows a full miniature view of all rooms on thefloor in order to allow the user to determine his cur-rent position in the building, which is highlighted inyellow. Additionally the path to the selected destina-tion room is highlighted in cyan, while the destinationroom itself is shown in red. Using such a WIM modelto help users understand virtual environments betterwas described by (Stoakley, Conway & Pausch 1995).

Figure 10: The world in miniature model shown onthe pip. The found path to the destination room ishighlighted.

As the wrist pad is tracked, the user can bring theWIM model into her view by moving her arm with thewrist pad in front of her eyes. As a consequence, theWIM model does not permanently cover the screen,and the user can rather decide herself when she wantsthe floor plan to be displayed as well as she can ad-just the size by moving her arm nearer to her eyes orfarther away.

Path finding

The Signpost application uses a path finding compo-nent for computing the shortest path from the user’scurrent room to the desired destination. It uses theadjacency graph from the tracking system and alsothe information about the room the user is currentlyin.

Using this adjacency graph, the application is ableto compute the shortest path between two rooms.This is done by utilizing an algorithm based onthe shortest path algorithm introduced by Dijkstra(Dijkstra 1959). The information about the path isthen used to highlight the rooms along the way on theWIM display and to augment the portal that leads into the next room from the current one. The portal in-formation is also used to compute the direction of theoverlayed arrows which need to point towards thatportal.

Although the application could recalculate and up-date the shortest path several times per second, a newpath is only calculated when a room-change occurs(which is triggered whenever the tracking system rec-ognizes a marker of a neighboring room to the currentroom). As the application always adapts the shortestpath starting from the current room, even if the userwalks the wrong way, a new shortest path based onthe user’s current position is proposed.

5.2 ARLib - Library information application

ARLib aims to aid the user in stereotypical tasks thatare of use in a library by augmenting a book’s positionon a shelf. It features two main modes of use.

Assistance in searching for a book

The user is interacting with the application via a userinterface presented on the touch pad. First, one ofthe search options like searching by title, author, etc.,must be selected. Next, text is entered by using ei-ther a virtual keyboard or a graffiti pad (see Figure11). A list of books matching the search is returnedand displayed. The user then selects the book she islooking for. Now the corresponding book’s positionon the shelf and the shelf itself are highlighted in theheads-up display to aid the user in finding the book(see Figure 12). Also, all available information aboutthe publication is displayed on the wrist panel.

a) b)

Figure 11: User interface for the ARLib application.(a) The user may select one of different search modes.(b) Input of search terms using graffiti text input.

Figure 12: Augmented position of a book as seen bythe user. These images were produced by overlayingthe tracking camera’s video image by the renderedgraphics.

Assistance in returning a book to the rightshelf

In this mode, ARLib attempts to detect markers thatare attached to books. If a marked book is spotted,all available information about the publication is pre-sented on the wrist panel, and the book’s designatedposition on the shelf is highlighted to aid the user inreturning the book to its correct position. This en-ables the user to simply look at the book in her handand trigger the applications behavior.

Implementation

ARLib maintains a model of the library’s shelvesand the positions of the individual books within theshelves. Furthermore meta information about thebooks is stored such as the title, authors, category andkeywords associated with it. Using the same trackingtechnology as described in section 4 the applicationcontinuously tracks the user’s position within the li-brary. In addition to that a designated set of markersis attached to the books themselves to allow trackingof their positions as well.

Depending on the state of the application differentparts of the model are rendered. If the location of abook is augmented, its containing shelf is rendered aswell as an augmented view of the book itself within

Figure 13: Markers are mounted to the book shelvesand registered.

the shelf. If the book itself is to be returned to thelibrary the application highlights the book to signalthe user that it identified it correctly.

To use ARLib with a real-life library, some prepa-ration work had to be done. ARLib’s books databaseis an OpenInventor scene graph file automatically cre-ated from a proprietary library database. The shelfgeometry and marker positions must be surveyed af-ter attaching the markers to the shelves to registershelf information from the database with the geomet-ric model (see Figure 13).

To test ARLib, we used our in-house library forwhich a database of roughly 650 publications in 13categories on 20 shelves was available. 62 markerswere created, of which 52 were placed at shelf joints,8 on book covers and 2 large ones on the walls toimprove tracking from a distance.

6 Results

In this work we described a simple tracking solutionfor wide area indoor tracking. It tries to fill the gapbetween high accuracy tracking solutions that onlycover a relatively small area and outdoor positioningtechniques that are not applicable indoors. Further-more we implemented two example applications thatbuild upon this development.

The tracking error varies largely. As long as mark-ers were continuously visible the augmentation of theroom geometry is fairly stable and correct. Howeveras soon as all markers are obscured the tracking canonly rely on the inertial tracker and starts to becomeincorrect. In this case the augmentation is not use-able anymore, because if the overlay is off by a fewcentimeters the interpretation of the information ismore difficult for the user.

This led to the conclusion that more markers areneeded to improve the quality of the tracking. The di-rectional arrows also performed better than the high-lighting of the portals under such circumstances asthe information they provide is less prone to track-ing errors. In the future we want to automaticallyswitch between these different presentations based onthe estimated tracking error as a form of Level of Er-ror filtering (MacIntyre & Coelho 2000).

Both applications were developed as part of a labexercise by groups of two or three students. This en-couraged us that the general platform and the frame-work provided by the Studierstube simplifies the de-velopment of such applications.

7 Future work

For future work we plan to extend the area coveredby the optical tracking. Both applications were de-ployed in our building on a scale that exceeded thetypical lab environment. However they still not covera desirable area such as a whole building or a typicaluniversity library. We plan to extend the navigationalapplication to cover at least two floors of our buildingwhich amounts to about 4 times the size of the currentinstallation. Also the number of markers should beincreased to improve the quality of the tracking. Thesurveying will be redone with a more accurate profes-sional surveying tools to provide a more detailed andcorrect model of the building.

The general indoor location information will beprovided as a service within the framework so thatfuture applications can directly leverage on this in-formation and must only concentrate on their func-tionality. We also plan to cover the same area with awireless LAN network to enable research into collab-orative scenarios using multiple AR setups.

Acknowledgements

This work was sponsored by the Austrian Sci-ence Foundation under contracts no. P14470-INF,P14470-N04 and START Y193, and Vienna Univer-sity of Technology by an infrastructure lab grants(”MARDIS”). Many thanks go to Michael Kalkusch,Michael Knapp, Thomas Lidy, Eike Umlauf and Har-ald Piringer for their inspired work on the implemen-tation of the example applications.

References

Addlesee, M., Curwen, R., Hodges, S., Hopper, A.,Newman, J., Steggles, P. & Ward, A. (2001),‘A sentient computing system’, IEEE Computer:Location-Aware Computing .

Airey, J., Rohlf, J. & Brooks, F. P. (1990), Towardsimage realism with interactive update rates incomplex virtual building environments, in ‘Pro-ceedings of the 1990 Symposium on Interactive3D Graphics’, pp. 41–50. not read.

Azuma, R. T. (1997), ‘A survey of augmented real-ity’, Presence Teleoperators and Virtual Envi-ronments 6(4), 355–385.

Baillot, Y., Brown, D. & Julier, S. (2001), Authoringof physical models using mobile computers, in‘Proc. ISWC 2001’, IEEE, Zurich, Switzerland,pp. 39–46.

Billinghurst, M., Bowskill, J., Morphett, J. & Jes-sop, M. (1998), A wearable spatial conferencingspace, in ‘Proc. ISWC’98’, Pittsburgh, Penn.,USA.

Billinghurst, M. & Kato, H. (1999), Collaborativemixed reality, in ‘Proc. ISMR’99’, Springer Ver-lag, Yokohama, Japan, pp. 261–284.

Dijkstra, E. (1959), ‘A note on two problems in con-nection with graphs’, Numerische Mathematik1, 269–271.

Feiner, S., MacIntyre, B., Haupt, M. & Solomon,E. (1993), Windows on the world: 2D win-dows for 3D augmented reality, in ‘ACM Sym-posium on User Interface Software and Technol-ogy’, pp. 145–155. not read.

Feiner, S., MacIntyre, B., Hollerer, T. & Webster,A. (1997), A touring machine: Prototyping 3Dmobile augmented reality systems for exploringthe urban enviroment, in ‘Proc. ISWC’97’, Cam-bridge, MA, USA, pp. 74–81.

Hollerer, T., Hallaway, D., Tinna, N. & Feiner,S. (2001), Steps toward accommodating vari-able position tracking accuracy in a mobile aug-mented reality system, in ‘Proc. AIMS’01’, Seat-tle, WA, USA, pp. 31–37.

Hollerer, T. & Pavlik, J. (1999), Situated documen-taries: Embedding multimedia presentations inthe real world, in ‘Proc. ISWC’99’, San Fran-cisco, CA, USA, pp. 79–86.

Intersense (2002), ‘Intersense IS-900’,http://www.isense.com/ [2002, September5].

Kourogi, M. & Sakaue, T. K. K. (2001), A panorama-based method of personal positioning and orien-tation and its real-time applications for wearablecomputers, in ‘Proc. ISWC’01’, Zurich, Switzer-land, pp. 107–114.

Lee, S.-W. & Mase, K. (2001), Incremental motion-based location recognition, in ‘Proc. ISWC’01’,Zurich, Switzerland, pp. 123–130.

MacIntyre, B. & Coelho, E. M. (2000), Adapting todynamic registration errors using level of error(loe) filtering, in ‘Proc. ISAR2000’, IEEE andACM, Munich, Germany, pp. 85–88.

Newman, J., Ingram, D. & Hopper, A. (2001), Aug-mented reality in a wide area sentient environ-ment, in ‘Proc. ISAR 2001’, IEEE and ACM,New York, New York, USA, pp. 77–86.

Piekarski, W. & Thomas, B. H. (2001), Tinmith-metro: New outdoor techniques for creatingcity models with an augmented reality wearablecomputer, in ‘Proc. ISWC 2001’, IEEE, Zurich,Switzerland, pp. 31–38.

Piekarski, W. & Thomas, B. H. (2002), The tin-mith system - demonstrating new techniques formobile augmented reality modelling, in ‘Proc.AUIC2002’, ACS, Melbournce, Vic, Australia.

Pinz, A. (2001), Consistent visual information pro-cessing applied to object recognition, landmarkdefinition, and real-time tracking, in ‘Proc.VMV’01’, Stuttgart, Germany.

Raab, F., Blood, E., Steiner, T. & Jones, R. (1979),‘Magnetic position and orientation tracking sys-tem’, IEEE Transactions on Aerospace and Elec-tronic Systems AES-15(5), 709–718.

Reitmayr, G. & Schmalstieg, D. (2001a), Mobile col-laborative augmented reality, in ‘Proc. ISAR2001’, IEEE, New York, New York, USA,pp. 114–123.

Reitmayr, G. & Schmalstieg, D. (2001b), An opensoftware architecture for virtual reality interac-tion, in ‘Proc. VRST 2001’, ACM, Banff, Al-berta, Canada, pp. 47–54.

Rekimoto, J. (1998), Matrix: A realtime object iden-tification and registration method for augmentedreality, in ‘Proc. APCHI’98’.

Satoh, K., Hara, K., Anabuki, M., Yamamoto, H. &Tamura, H. (2001), TOWNWEAR: An outdoorwearable MR system with high-precision regis-tration, in ‘Proc. IEEE Virtual Reality 2001’,Yokohama, Japan, pp. 210–211.

Schmalstieg, D., Fuhrmann, A., Hesina, G., Szalavari,Z., Encarnao, L. M., Gervautz, M. & Purgath-ofer, W. (2002), ‘The studierstube augmentedreality project’, PRESENCE - Teleoperators andVirtual Environments 11(1).

Starner, T., Mann, S., Rhodes, B., Levine, J., Healey,J., Kirsch, D., Picard, R. & Pentland, A. (1997),‘Augmented reality through wearable comput-ing’, Presence Special Issue on AugmentedReality(4), 386–398.

Stoakley, R., Conway, M. J. & Pausch, R. (1995),Virtual reality on a wim: interactive worldsin miniature, in ‘Conference proceedings onHuman factors in computing systems’, ACMPress, Addison-Wesley, Denver, Colorado, USA,pp. 265–272.

Suomela, R. & Lehikoinen, J. (2000), Context com-pass, in ‘Proc. ISWC 2000’, IEEE, Atlanta,Georgia, USA, pp. 147–154.

Thomas, B., Close, B., Donoghue, J., Squires, J.,Bondi, P. D., Morris, M. & Piekarski, W. (2000),Arquake: An outdoor/indoor augmented realityfirst person application, in ‘Proc. ISWC2000’,IEEE, Atlanta, Georgia, USA, pp. 139–146.

Thomas, B. H., Demczuk, V., Piekarski, W., epworth,D. H. & Gunther, B. (1998), A wearable com-puter system with augmented reality to supportterrestrial navigation, in ‘Proc. ISWC’98’, IEEEand ACM, Pittsburgh, USA, pp. 168–171.

Welch, G., Bishop, G., Vicci, L., Brumback,S., Keller, K. & Colucci, D. (2001), ‘High-performance wide-area optical tracking - the hi-ball tracking system’, Presence: Teleoperatorsand Virtual Environments 10:1.