evaluation of a mobile projector-based indoor navigation...

© The Author 2013. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved.For Permissions, please email: [email protected]

doi:10.1093/iwc/iwt053

Evaluation of a Mobile Projector-BasedIndoor Navigation Interface

Ming Li1,∗

, Katrin Arning2, Oliver Sack

2, Jiyoung Park

3, Myoung-Hee Kim

3,

Martina Ziefle2

and Leif Kobbelt1

1Computer Graphics Group, RWTH Aachen University, Aachen, Germany2Human Computer Interaction Center, RWTH Aachen University, Aachen, Germany

3Ewha Womans University, Seoul, Korea∗Corresponding author: [email protected]

In recent years, the interest and potential applications of pedestrian indoor navigation solutions havesignificantly increased. Whereas the majority of mobile indoor navigation aid solutions visualizenavigational information on mobile screens, the present study investigates the effectiveness of a mobileprojector-based navigation aid that directly projects navigational information into the environment.A benchmark evaluation of the mobile projector-based indoor navigation interface was carried out,investigating a combination of different navigation devices (mobile projector vs. mobile screen) andnavigation information (map vs. arrow) as well as the impact of users’spatial abilities. Results showedthe superiority of the mobile screen as a navigation aid and the map as a navigation informationtype. Especially users with low spatial abilities benefited from this combination in their navigationperformance and acceptance. Potential application scenarios and design implications for novel indoor

navigation interfaces are derived from our findings.

RESEARCH HIGHLIGHTS

• Experimental benchmark evaluation of a mobile pico projector for pedestrian navigation purposes.• Multidimensional evaluation of the effectiveness of navigation information for pedestrian navigation.• User experience and acceptance of mobile pico projectors.• Inclusion of cognitive user factors (e.g. spatial abilities) and user experiences (e.g. trust, privacy) of mobile

navigation aid usage.• Outline of potential application scenarios and design implications for novel indoor navigation interfaces.

Keywords: indoor navigation; mobile projector; navigation information; distortion correction; spatialability; evaluation

Editorial Board Member: Ruven Brooks

Received 12 January 2013; Revised 9 September 2013; Accepted 16 September 2013

1. INTRODUCTION

While outdoor navigation is already widely used for car andpedestrian navigation, the development and usage of indoornavigation devices is still in its infancy. Owing to rapid technicaldevelopments in areas of localization technologies, mobiledevices, display technologies and data transfer rate, the interestin indoor navigation aids has considerably increased (May et al.,2003).

Moreover, the increased availability of handheld devices suchas personal digital assistants or smart phones allows for a mobile

use of navigation systems by ‘everyone’. Indoor navigationsystems can provide navigation support in a broad range ofapplication scenarios, e.g. in tourism navigation systems cansupport travelers in finding sights or other points of interest(POIs); in rescue services navigation systems offer help intracking lost persons or supporting the paramedics in quicklyfinding the way to a patient in a house, as well as individualnavigational aid, where people with specific needs (e.g. physicalor visual impairment) are guided to their destination (Millonigand Schechtner, 2007).

Interacting with Computers, 2013

Interacting with Computers Advance Access published November 6, 2013 by guest on N

ovember 7, 2013

http://iwc.oxfordjournals.org/

Dow

nloaded from

by guest on Novem

ber 7, 2013http://iw

c.oxfordjournals.org/D

ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from

by guest on Novem



ownloaded from

by guest on N

ovember 7, 2013


Dow

nloaded from




















2 Ming Li et al.

Figure 1. A concept sketch of mobile projector-based navigation interfaces. In the map mode (top), the circle indicates the user’s current positionand the dashed line represents the route to the destination. When the position gets updated, the circle and dashed line are refreshed as well. In thearrow mode (bottom), the arrow shows which way to go. When the user reaches a turning point, the arrow will rotate and point to the correct way.

The majority of mobile indoor navigation aid solutionsvisualize navigational information on a mobile screen (e.g.Arikawa et al., 2007; Serra et al., 2010). However, in recentyears, miniaturization introduced compact mobile projectors(so-called pico projectors) to the market which show a highapplication potential in the context of mobile navigation.Traditional applications of projectors were restricted by a fixedinstallation with very limited mobility due to constraints ofphysical size and power supply (Pinhanez, 2001; Rukzio et al.,2005). In contrast, pico projectors can be used as a mobilepersonal device in the navigation context (Rukzio et al., 2011)either as an add-on to standard, handheld devices or integratedinto smart devices (Wecker et al., 2011; Winkler et al., 2011).Combined with indoor localization techniques (Link et al.,2013; Nokia, 2010), projector-based navigation systems canaugment the environment with route information, turn-by-turninstructions, POI, etc. Compared with mobile screen-based

navigation interfaces (e.g. a 2D map on a smart phone screen),the projector directly projects the navigation information intothe environment so that there is no need to switch attentionbetween the screen and the real world (Fig. 1). Moreover, insteadof displaying information on a small (e.g. 3.5 inch) screen, picoprojectors can enlarge content up to tens of inches on an arbitrarysurface which enables group sharing navigation and helps userswith eyesight issues.

Apart from technical reliability of navigation systems andquality of navigation information, a key prerequisite in design-ing successful indoor navigation systems is the considerationof individual user demands and perceptions. The risk of marketfailure is high as long as user factors and acceptance issuesare not adequately considered in system design and users donot perceive the usefulness or benefits of a system (Kuniavsky,2010). Although pedestrian navigation systems in general and,more specifically, the use of projectors for a public display of


Evaluation of a Mobile Projector-Based Indoor Navigation Interface 3

navigation information have been discussed and researchedin recent years (Goodman et al., 2005; Rukzio et al., 2005;Wecker et al., 2011; Winkler et al., 2011), individual userfactors such as spatial abilities as well as end-users’ acceptanceof indoor navigators were not systematically included andevaluated (Downs and Stea, 2011; Hegarty et al., 2006).

One of the most relevant user factors in the context ofnavigation is the users’ spatial abilities (Dillon and Morris,1996; Downs and Stea, 2011; Hegarty et al., 2006; Rau andWang, 2003). Navigation systems provide spatial information(e.g. map and routes) to a user who has to be able tounderstand and correctly apply the given information duringwayfinding. However, people with low spatial abilities oftenexperience problems with orientation and understanding spatialinformation (Pak et al., 2008). Therefore, the design ofnavigation systems has to provide users of different spatialability proper navigation information.

2. PEDESTRIAN INDOOR NAVIGATIONTECHNOLOGY

In recent years, companies and organizations began collectingand providing indoor map and route information, e.g. Googleindoor navigation and OpenStreetMap indoor mapping. Theindoor localization service can be provided by varioustechniques (Link et al., 2013; Nokia, 2010). In this paper,we focus on the research of the mobile projector-basednavigational interface. The indoor position is provided bymanual localization. The proposed interface can be combinedwith existing indoor positioning techniques to achieve realisticnavigation solutions.

An overview of personal projector applications for pervasivecomputing is given by Rukzio et al. (2005) in their survey inwhich projector-based navigation belongs to the category ofaugmented reality (AR). Winkler et al. (2011) proposed theconcept of indoor navigation for shopping malls through amobile projector. Wecker et al. (2011) introduced Pathlight forin-museum navigation using a handheld projector. The turn-by-turn instructions were given by projecting landmark photosaugmented with directional arrows. Several research challengeswere listed to be answered, e.g. user acceptance, preferredprojection surface, visibility, etc. The concept of using a mobileprojector for indoor navigation was introduced or prototypedin these publications, but this novel interface has not beenformally evaluated: e.g. how does its performance compare toa mobile display-based navigation interface, and how well dousers perform in finding their way?

Some other researchers proposed to use a fixed projector todisplay navigational information. Pinhanez (2001) presentedan everywhere display projector, which used a fixed projec-tor and rotating mirror to overlay information on differentsurfaces of an environment. Rukzio et al. (2005) presenteda public projection-based mobile navigation approach. Apublic projector displayed a rotating arrow on the floor at an

intersection. When the arrow pointed to the direction a user hasto follow, the mobile device of the corresponding user wouldvibrate indicating that was his/her way. Compared with theseapproaches, our prototype utilizes a mobile projector that doesnot require hardware installation and can be carried easily.Therefore, it is more flexible and easier to deploy.

3. SPATIAL COGNITION AND INDIVIDUALDIFFERENCES IN NAVIGATION

Wayfinding refers to the orientation in physical spaces and tothe navigation from a point origin to a specific destination (Huntand Waller, 1999; Lynch, 1960). It is one of the very basicdaily activities humans have had to perform since the beginningof mankind. People differ largely in their wayfinding abilities,i.e. they either successfully reach their destination on the mostdirect way or they get lost and have to make detours to reachtheir destination. By the use of computer-based navigation aids,wayfinding activities can be facilitated enormously. To designeffective and useful navigation aids, the characteristics of humanwayfinding abilities and underlying spatial cognition processeshave to be considered (Ziefle, 2010).

Wayfinding is not only a physical motion in an environmentbut also a highly complex cognitive process. Spatial cognitionapproaches assume that wayfinding is based on an individual’sspatial representation (‘cognitive map’) of a real or a virtualenvironment. Hence, people use cognitive representations oftheir spatial environment to guide wayfinding. Cognitive maps,which were first described and defined in the 1940s by Tolman(1948), encompass several types of spatial knowledge (Downsand Stea, 2011; Lynch, 1960; Werner et al., 1997), i.e. landmarkknowledge (knowledge about prominent reference points), pro-cedural route knowledge (connection paths between landmarks)and survey knowledge (the ‘big picture’ of the environment).

One of the most important cognitive abilities related tospatial knowledge (the knowledge and internal or cognitiverepresentation of the structure, entities and relations of space)are spatial abilities that are defined as the ability to manipulateand organize objects in space (Downs and Stea, 2011; Hegartyet al., 2006). People with high spatial abilities are able toeffectively and efficiently visualize spatial environments andbuild an elaborate cognitive map that enables them to achievea higher navigational performance (Dillon and Morris, 1996;Kirasic, 2000; McGee, 1979; Rau and Wang, 2003). In contrast,people with low spatial abilities build up and use less efficientcognitive maps. They face higher disorientation problems whenthey navigate through novel or complex spatial environmentsbecause their cognitive map is neither sufficient nor elaborateenough. Technical navigation aid solutions might be beneficial,especially in these cases.

The acquisition of spatial knowledge happens not onlyvia direct environmental exposure but can also be basedon secondary sources such as route descriptions or maps.A map contains spatial characteristics of the environment,


4 Ming Li et al.

which support the construction of a cognitive map by thewayfinder. The map, whose effectiveness was confirmed inmany empirical studies (e.g. Dee-Lucas, 1996; Thorndyke andHayes-Roth, 1982; Ziefle and Bay, 2006), is a firmly establishedpresentation format for spatial navigation information in realenvironments as well as in computer menus. On the otherhand, maps are not always beneficial for navigation becausesome users—especially older users or those with lower spatialabilities—are not capable of processing, transforming andtransferring the spatial information given in the map totheir own navigation route (Goodman et al., 2005; Rau andWang, 2003). In these cases, landmark or route informationwas found to be the optimal navigation aid. Especiallypedestrian navigation requires different formats of navigationalinformation (Goodman et al., 2004a,b; May et al., 2003).

Interestingly, within the research field of pedestrian indoornavigation aids, the impact of individual differences, especiallyin cognitive abilities, has not yet been systematically studied.Although indoor and pedestrian navigation aid solutions existfor specific user groups with special needs, such as visuallyimpaired people (Bousbia-salah and Fezari, 2006) or older users(Goodman et al., 2005), a systematic inclusion of relevant userfactors such as spatial abilities is still missing. Research in thefield of navigation in data structures (menus or hypertext) hasshown that users with low spatial abilities face more problemsin acquiring navigational skills and interacting with computers(Arning and Ziefle, 2009; Goodman et al., 2005; Rau and Wang,2003; Ziefle et al., 2007). Moreover, they require specificallydesigned navigation aids (e.g. help functions or tutorials)that compensate for problems in perceiving, transforming andapplying spatial information (Rau and Wang, 2003; Ziefleand Bay, 2006). Therefore, we include spatial abilities asan individual user factor in the context of pedestrian indoornavigation interaction with navigation performance (n.s.) anduser perceptions in the present study.

4. TECHNOLOGY ACCEPTANCE OF NAVIGATIONDEVICES

Technology acceptance research seeks to explain and predictthe perception and adoption of technologies by end-users.Established technology acceptance theories, such as theTechnology Acceptance Model (TAM) (Davis, 1989) or theUnified Theory of Acceptance and Use of Technology (Goldinand Thorndyke, 1981), primarily focus on job-related usageof information and communication technologies (ICTs). Sincetechnology acceptance is highly context-specific (Arninget al., 2010), existing knowledge from ICT-related technologyacceptance research or the well-studied car navigation contextcannot be easily transferred to the indoor or pedestriannavigation context (Dillon and Morris, 1996). Moreover,end-user perceptions of indoor navigation systems have notbeen systematically and empirically evaluated so far.

Studies focusing on acceptance of navigation aids primarilyfocused on general satisfaction, ease of use and the perceivedusefulness of a technology. Heinroth and Buehler assessed theusefulness, ease of use in handling the device, convenienceand willingness to utilize a speech-based pedestrian navigationsystem (Arning and Ziefle, 2009). Rukzio et al. (2005) assesseddisorientation events, usability, satisfaction and workload in theevaluation of a ‘rotating compass’, a public display technique.As of yet, however, no comprehensive assessment of theacceptance of navigation aids and projectors as a displaymedium has been carried out, covering aspects such as privacy,trust and disorientation. Since the projection is not only visibleto the user but also to other people at the same location,privacy concerns are a serious issue that should be adequatelyaddressed. Greaves et al. (2009) reported a study about thesocial impact and privacy concerns regarding the usage ofmobile projectors in public spaces. In the NAVITIME study, inwhich mobile-phone-based navigation and other location-basedservices were evaluated, users perceived disorientation whenusing the device but the sample size (n = 2) was too small tointerpret and generalize this finding (Adams et al., 1992). Theassessment of a broader range of user perceptions in this studywill allow determining the most relevant factors contributing toacceptance. These factors should then be addressed as a startingpoint or fulcrum in subsequent design activities because they arethe major influential factors of the end-users’ acceptance.

5. METHODOLOGY

5.1. Research approach

Overall, three main research questions have been guiding thisstudy:

(1) Which navigation device and which type of navigationinformation are the most effective and most accepted forindoor pedestrian navigation? As a mobile projector hasbeen rarely used for indoor navigation purposes before,we wanted to conduct a benchmark evaluation to testthe usefulness of this novel interface (based on a mobileprojector) against a conventional navigation interface(based on a mobile display).

(2) How does the users’ spatial ability affect their n.s. andperceived usefulness? Which combination of navigationdevice and information type is the most beneficial forusers with low spatial abilities?

(3) What are major influential factors on n.s. and end-user perceptions of indoor navigation devices? As theinteraction with the mobile projector might be affectedby confounding factors such as visibility problems dueto changing light conditions, trust and privacy concerns,the wayfinding performance and the acceptance of theprojector as a navigation aid might be lower than that ofthe mobile display.



5.2. Experimental design

In the present study, we aimed for an evaluation of participants’wayfinding performance. Two independent variables wereexamined. The first was the navigation device, comparingmobile projector vs. mobile screen navigation. The secondindependent variable addressed two different types ofnavigation information (map vs. arrow). These two types aredescribed in more detail in the following section. To investigatelearning effects, participants had to accomplish two routesfor each condition of navigation device and information type.Accordingly, the study is based on a 2 (navigation aid) ×2 (information type) × 2 (learnability)-factorial-within-design.The following conditions were accomplished by eachparticipant: C1: map, screen; C2: arrow, screen; C3: map,projector and C4: arrow, projector. The order of conditionswas balanced. As navigation and wayfinding performance arestrongly influenced by users’ spatial abilities (Rau and Wang,2003), participants’ spatial visualization ability was assessedand treated as a between-subject variable.

5.3. Independent variables

5.3.1. Navigation devicesTwo navigation display devices were compared: a mobile screenand a mobile projector. The mobile screen is widely used todisplay digital maps, virtual guides (Google Street View) andAR navigation (Mulloni et al., 2011). The legibility is lessaffected by the light condition for indoor scenarios. Moreover, asa personal display, the screen content is only viewable to users,therefore privacy concerns concerning the displayed contentmight be less pronounced than when displaying the informationwith a projector. Compared with a mobile projector, its outputspace is limited by screen size. In non-AR navigation solutions,users have to frequently switch visual attention between thedisplay and the real world. In mobile screen-basedAR solutions,although the video-see-through interface enables environmentawareness, it is still restricted by the camera’s field of view andits viewing direction (e.g. pointing the camera in the directionone is heading to).

When the mobile projector is used forAR purposes, it requiresan adaptation to the presentation surface, i.e. pre-warping theimage for distortion-free projection. Tajimi et al. (2010) applieda stabilization approach for floor projection with a hip-mountedprojector. To improve the viewability of the projected contentduring the user’s walking, a tilting sensor was used to detectand compensate the motion of the projector. We utilize a similarapproach to deliver a distortion-free projection. The projectedimages (map/arrow) are pre-warped with reference to the devicemotion measured by a gyroscope and accelerometer (Fig. 2).The projector output data are measured in advance to create anidentical projection matrix for each rendering. The additionaltransformation applied to the rendered geometry consists ofthree rotation matrices based on the angle pitch, roll and the

heading direction (deviation from the current route): M =RrollRpitchRheading.

To keep the image projection a uniform size, like a physicallyrealistic pattern printed in the real environment, we cancompensate the size by extra scale matrices depending on thetilting angles: M = RrollRpitchRheadingSrollSpitch.

5.3.2. Navigation informationSpatial knowledge representation is based on the combination oflandmark, route and survey knowledge (Goldin and Thorndyke,1981). In our study, we contrasted two types of spatialnavigation information, a map and an arrow (Fig. 3).

The first type of spatial navigation information was a mapthat provided—at least for the visible display window—surveyknowledge. According to spatial cognition theories, surveyknowledge is the most elaborate type of spatial knowledge. Itsupports the construction of an adequate mental representationof an environment which can be used for route planning,wayfinding and the development of alternative routes in caseof getting lost (Kim and Hirtle, 1995).

The second type of spatial navigation information was anarrow that was orientation-sensitive and gave timely informa-tion about which direction to take. With regard to spatial knowl-edge theories, the arrow provided a reduced amount of routeknowledge (without landmark information or route overview).However, in the context of indoor navigation, we assume thatthe orientation-sensitive arrow would free users from spatiallyrepresenting the environment, as they would only have to followthe location-adaptive arrow orientation. This location-adaptiveorientation of the arrow allows one to overcome the main dis-advantage of route information: in case of disorientation andgetting lost, no constructive navigation information to return tothe correct route is available. In contrast, the location-adaptivearrow gives information about the correct navigation directioneven if the user deviates from the correct route.

Both types of navigation information utilized the integratedmagnetometer to detect the heading direction and align theimage to the current orientation. In our prototype setup, theindoor position was given by a manual localization methodin which the experimenter followed the participant and sentlocation updates from a host device to the participant’snavigation aid via a portable wireless network.

5.4. Hardware

We implemented the navigation prototype on an iOS platform.We used a SAMSUNG SP-H03 Pico Projector (30 ANSILumens) connected to an iPhone4 via VGA cable (Fig. 4). Theheading direction and the device motion were taken from theintegrated sensors of the iPhone4. In C1 and C2, the participantheld the iPhone4 as the navigator and the navigation informationwas displayed on the screen. In C3 and C4, the participant heldthe hardware as pictured in Fig. 4 and the information wasprojected to the floor or other surfaces that they preferred.


6 Ming Li et al.

Figure 2. Distortion-free projection: the navigation interface is pre-warped before projection. From the user’s perspective, the projected imagecontent is not distorted.

Figure 3. Navigation interfaces: map and arrow displayed on a mobilescreen (top) and projected onto the floor (bottom).

5.5. Experimental setup

The testing area was on the university campus. Based on thefloor plans of the selected buildings, the map was created asan overlay to a standard Google map view. As a within-subjectdesign was used, every participant had to go through all routes

Figure 4. Our projector-based navigator prototype (dimension: 32 ×11.5 × 7 (cm), weight: about 1 kg).

using different navigation aids accordingly. In Fig. 5, samplesof the selected routes are presented.

To evaluate the learnability of using the navigation aids, eachroute contained two sub-routes, A and B. The complexity ofall routes (route length, number of junctions and branches) wascomparable (Table 1).

The illumination on these routes was comparable as well(mainly daylight lamps). To rule out learning effects, routes



Figure 5. Samples of the floor plans and selected routes.

Table 1. Complexity of selected routes, which describes numberof junctions, number of branches, number of floors, route lengthand branch density (average number of branches per junction).

Route #Junc. #Branch #Floor Length (m) DensityR1A 3 11 1 68 3.67R1B 4 16 2 68 4R2A 3 11 1 40 3.67R2B 4 16 2 64 4R3A 2 8 1 58 4R3B 3 13 2 79 4.33R4A 3 11 1 68 3.67R4B 4 16 2 73 4

Every lobby that has more than two branches counts for onejunction. Every staircase in a lobby counts for one branch. Everyelevator counts for one branch.

were selected from different buildings. Sequence effects wereeliminated by balancing the order of the navigation aids.

During the planning phase of the experiment, we tried toselect longer routes in the testing area. However, the projection

legibility on such routes varied from time to time due tochanging light conditions (e.g. sunlight through windows) andfloor properties, and therefore longer routes were not used inthis study.

5.6. Dependent variables

As dependent variables, we assessed three categories ofvariables: (i) n.s., (ii) end-user perceptions and workload and(iii) effects of spatial abilities on n.s. and user perceptions.

5.6.1. Navigation performanceParticipants’ wayfinding performance was evaluated by thepercentage preferred walking speed (PPWS), which reflects theextent to which the use of the navigation aid disrupts normalwalking (Goodman et al., 2004a,b). Assessing individualbaselines, the participants were asked to walk a predefineddistance at their normal indoor walking speed prior to theexperiment. The preferred walking speed (PWS) of eachparticipant was calculated by the distance and completion time.In the test, their actual walking speed (AWS) was compared


8 Ming Li et al.

with their PWS. Any negative disruption due to the navigationaid will result in a decreased percentage of PWS. Highervalues indicate a better wayfinding performance (PPWS =AWS/PWS × 100%).

5.6.2. User perceptionsEnd-user perceptions (described below in alphabetical order)were collected via subjective ratings. Participants had to indicatetheir answers on a six-point Likert Scale (1 = stronglydisagree, 6 = strongly agree). Item examples are given inbrackets.

Disorientation. Navigation devices support building up amental representation of the environment (Goodman et al.,2005), e.g. knowing where to be. Participants had to rate theirperceived disorientation (‘I had difficulties orienting myself inthe building.’).

Legibility. As the legibility of the navigation informationmight be influenced by lighting conditions, we assessed thelegibility of the projected or displayed navigation information(‘The map was always legible on the display.’).

Preference. Participants had to rate their preferred way ofinformation presentation on the navigation devices.

Privacy. Participants were requested to rate the perceivedprivacy during the ‘public’projection of navigation information(‘I felt uncomfortable when other people saw me using anavigation system.’) (Goodman et al., 2005).

Satisfaction. The general satisfaction using the navigationdevice was immediately assessed after route completion andwas taken as an indicator for acceptance (‘How did you like thenavigation during the last navigation task?’).

Trust. As the perceived reliability of the given navigationinformation is of great importance for the users’ trust andacceptance (Fox, 1996), we assessed trust during navigation(‘I could trust the information of the navigation system on myway.’).

Usefulness. For the assessment of perceived usefulness(‘With the navigation system I reached my destination faster.’),a shortened version of the original items of the TAM (Davis,1989) was used.

Workload. To evaluate the perceived workload of eachcondition, participants had to answer the standard NASA-TLXquestionnaire (Hart and Staveland, 1988). The NASA TLXquestionnaire assesses the perceived workload on six differentsubscales: Mental Demand, Physical Demand, TemporalDemand, Performance, Effort and Frustration. After eachcondition, participants rated the six subscales (e.g. ‘How muchmental and perceptual activity was required?’) on a scaleranging from 0 to 100. Finally participants had to weigh whichdimensions contributed most (and least) to the workload theyexperienced during navigation. Thus, the average of the sixratings after each condition, weighted to reflect the contributionof each dimension, formed an integrated measure of the overallworkload of each navigation aid condition.

5.6.3. Spatial abilitiesTo measure spatial ability, participants completed a spatialvisualization test taken from the Kit of Factor-Referenced Cog-nitive Tests (Paperfolding test, Ekstrom et al., 1976). In this 2Dspatial task, 20 items were presented that show the illustrationof a sheet of paper, which is then folded several times andfinally punched. Participants had to select one of five drawingsto show how the punched sheet would appear after completelyunfolding the paper. To correct for guessing, the total score wascalculated by: Score = Nright − Nwrong/N − 1. Even thoughthe paperfolding test is a widely used and validated ability testin the field of human–computer interaction (e.g. Dillon andWatson, 1996; Evans and Simkin, 1989; Hegarty et al., 2003),it does not provide normative data. In a previous experimentalstudy (Bay and Ziefle, 2003), it was controlled that the paper-folding measures the same sub-abilities than other standardizedspatial tests, therefore we relied on the paper folding test toassess participants’ spatial abilities in this research. The out-comes of the spatial tests are completely in line with outcomesof other studies examining users of the same age range.

The maximum score to be reached was 20. For furtheranalyses, two groups with low and high spatial abilities wereformed by median-splitting the spatial-ability scores.

5.7. Procedure

In the beginning of the experiment, we first measured theparticipants’ PWS and spatial visualization ability. A shorttutorial was given to teach the participants how to use thenavigation aids. The participants were told to freely exploresuitable projection surfaces. After the tutorial, the participantswere led to the starting point of one testing route by theexperimenter. They had to walk to a given destination, followingthe navigation instructions presented on the navigation device.To remove the influence of area familiarity, the participant wasnot informed about the location of the destination in advance.The sequence of test routes was fixed for each participant forexperimental control reasons. The completion time of everyroute was recorded on the navigator to compute theAWS. Whenthe participants reached the destination of each route, theyrated the perceived workload (NASA-TLX) and completed apost-condition questionnaire containing acceptance items. Theparticipants had to go through four different routes and, atthe end, fill out a post-experiment questionnaire. The overallduration of one study was about 1 hour and 30 min.

5.8. Sample

A total of 24 participants, 19 women and 5 men, aged between21 and 28 (M = 23.5; SD = 2.3) took part in thestudy. Regarding the recruitment of participants, a ‘benchmarkprocedure’ was pursued. We wanted to evaluate the mobileprojector in a ‘best-case’ user group, i.e. a young, cognitivelyfit, technology-experienced sample. Since we aimed for a



comparative benchmark evaluation between the rather novelprojection-based navigation system and the established mobilescreen navigation interface, we wanted to control for effectsof cognitive aging or differing sociotechnical backgrounds.Therefore, participants were mostly students of a technicaluniversity whose participation fulfilled a course requirement.The majority of the participants (79%) were completelyunfamiliar with the testing location. Statistical analysis showedthat familiarity with the testing location had no effects onn.s. While the participants reported experience with the useof smart phones, they were all novices regarding the useof mobile projectors. Also, the participants had very limitedexperience using the broad range of navigation technologies(e.g. in-vehicle navigator or geocaching). The participantsreported minor difficulties to orient themselves in foreign cities,but no difficulties were mentioned when in cities they werefamiliar with.Assessing spatial ability revealed a heterogeneousdistribution (M = 9.4 points of 20; SD = 3.8). Participantswere divided into two groups of low and high spatial abilityusing a median split (MD = 8.8). In contrast to other studies(Arning and Ziefle, 2007; Ziefle and Bay, 2006) focusingon gender differences in spatial abilities in older adults, theyoung female and male participants did not differ significantlyregarding their spatial abilities.

5.9. Statistical analysis

Data were analyzed by ANOVAs with repeated measurement.The significance level was set at 5%. The significance of theomnibus F-Tests in the MANOVA analyses was taken fromPillai values. Greenhouse-Geisser corrected degrees of freedomand P -values were used whenever the sphericity assumptionwas violated. As a measure of effect size, partial eta-squared(η2) is reported. Non-parametric tests were used to test maineffects of subjective ratings before ANOVAs were applied.

6. RESULTS

6.1. Effects of navigation device, information type andlearnability on n.s.

Significant main effects were found in a 2 × 2 × 2 ANOVA(repeated measurement) with the factors navigation device,information type and learnability on n.s. Users’ n.s., measuredby PPWS, was significantly lower when navigating with theprojector in comparison with the screen (Mprojector = 0.83,SD = 0.95, Mscreen = 0.88, SD = 0.08; F(1, 23) = 16.6;P < 0.001, η2 = 0.42). Looking at the effects of informationtype, the PPWS was significantly reduced in the arrow conditionin contrast to the map condition (Marrow = 0.84, SD = 0.95,Mmap = 0.88, SD = 0.86; F(1, 23) = 8.8; P < 0.01,η2 = 0.28). Significant learning effects were also revealed: ThePPWS was significantly higher in the second route (MRouteB =0.91, SD = 0.86) in comparison with the first (MRouteA = 0.81,

Figure 6. Mean PPWS in the four navigation aid conditions (n = 24).Error bars show standard deviations. Higher values indicate betterwayfinding performance.

SD = 0.86; F(1, 23) = 158.6; P < 0.001, η2 = 0.87). Nointeractions between the factors were found. Summarizing sofar, users showed the best wayfinding performance, i.e. lessdeviations of their normal walking speed, when they used themap which was displayed on the screen (Mmap,screen = 0.91,SD = 0.92). The worst performance was found for wayfindingwith the projected arrow (Marrow,projector = 0.82, SD = 0.11);t (23) = 4.5; P < 0.001; see Fig. 6).

6.2. Effects of navigation device and information type onend-user perceptions

Disorientation. Perceived disorientation (e.g. ‘While navigatingwith the device I always knew where I was.’) while navigationwas comparably low (Mdisorientation = 2.27, SD = 0.73, Min =1.0, Max = 4.0). A 2 × 2 ANOVA with the factors navigationdevice and information type showed that navigating with thearrow caused higher levels of disorientation when comparedwith the map (Marrow = 2.58, SD = 0.95, Mmap = 1.96, SD =0.79; F(1, 23) = 9.8; P < 0.05, η2 = 0.30). Disorientationwas also lower when walking with the mobile screen than withthe projector (Mscreen = 2.08, SD = 0.76, Mprojector = 2.46,SD = 0.89; F(1, 23) = 5.1; P < 0.05, η2 = 0.18).

Legibility. After each condition participants were asked torate the legibility of the navigation information (map and arrow)given on the navigation device (screen and projector) on a six-point Likert scale (1 = very good legibility, 6 = very lowlegibility). In general, the legibility of all navigation aids wasrated ‘good’ (Mlegibility = 2.12, SD = 0.56, Min = 1.25,Max = 3.5). A 2 × 2 ANOVA with the factors informationtype and navigation device revealed better legibility ratings forthe screen (Mscreen = 1.44, SD = 0.54) than for the projector(Mprojector = 2.85, SD = 0.83; F(1, 23) = 67.8; P < 0.001,


10 Ming Li et al.

η2 = 0.75). No significant main effect of information type oran interaction was found.

With respect to the visual quality of displayed information,participants had to indicate which factors affected the legibilityof the projection. The majority of users (43.5%) reported thatchanging light conditions interfered the most with the legibilityof the projection, followed by patterned floor characteristics(13.0%), a reduced contrast of the projection, the changebetween different projection surfaces, too dark a floor (each8.7%) and, finally, too bright a floor (4.3%). Interestingly,13.0% reported that nothing interfered with the legibility ofthe projection.

Preferences. Participants had to indicate their preferrednavigation aid in the post-experimental questionnaire (forced-choice-format): The majority (83.3%) chose the map/screencondition, followed by the arrow/screen condition, which waschosen by 16.7%. Both projector conditions (map and arrow)were not chosen at all.

For a more detailed analysis, participants were also askedafter each experimental condition to indicate their preferenceon a six-point Likert scale (1 = very high preference, 6 =very low preference). An ANOVA with the within-factorsinformation type, navigation device and learnability revealeda significantly higher preference for the screen (Mscreen = 1.61,SD = 0.43) than for the projector (Mprojector = 1.92, SD = 0.69;F(1, 23) = 5.1; P < 0.05, η2 = 0.18) and for the map(Mmap = 1.59, SD = 0.53) in comparison with the arrowmode (Marrow = 1.94, SD = 0.75; F(1, 23) = 5.3; P < 0.05,η2 = 0.19). The preference ratings for the first and thesecond route did not differ significantly. Summarizing so far,users preferred the map displayed on the screen the most(Mmap,screen = 1.44, SD = 0.54), while the projected arrowreceived the lowest preference ratings (Marrow,projector = 2.08,SD = 0.20; t (23) = −3.2; P < 0.01).

Privacy. As the usage of the navigation projector, i.e. the‘public’ projection of navigation information, attracts more(unwanted) attention of others than the usage of conventionalnavigation devices, we included privacy issues into our analysis.Users were asked to indicate on a six-point Likert scale (1 =high privacy concerns, 6 = low privacy concerns) if they feltuncomfortable due to the fact that others could watch themusing a navigation device. Figure 7 shows bystanders watchinga participant during a wayfinding task. In general, rather fewprivacy concerns were reported by the sample (Mprivacy = 4.73,SD = 1.04, Min = 2.75, Max = 6.0). A 2×2 ANOVA with thefactors information type and navigation device yielded higherprivacy concerns when using the projector (Mprojector = 4.39,SD = 1.51) than the screen (Mscreen = 5.02, SD = 0.83;F(1, 23) = 67.8; P < 0.001, η2 = 0.72).

Satisfaction. Users’ satisfaction, which was generally high(Msatisfaction = 5.23, SD = 0.52, Min = 3.0, Max = 6.0),was significantly higher when navigating with the mobile screenthan with the projector (Mscreen = 5.39, SD = 0.54, Mprojector =5.08, SD = 0.69; F(1, 23) = 5.3; P < 0.05, η2 = 0.19).

Figure 7. Bystanders watching the ‘public’ navigation information ofthe mobile projector.

Users were also more satisfied when navigating with the mapin comparison with the arrow (Marrow = 5.4, SD = 0.52,Mmap = 5.1, SD = 0.75; F(1, 23) = 5.1; P < 0.05,η2 = 0.17).

Trust. Trust in both navigation systems was comparably high(Mtrust = 5.18, SD = 0.53, Min = 4.0, Max = 6.0).Nevertheless, participants stated to have a higher trust in thenavigation information of the mobile screen (Mscreen = 5.38,SD = 0.52) compared with the projector while navigating(Mprojector = 5.0, SD = 0.74; F(1, 23) = 7.5; P < 0.05,η2 = 0.25), as a 2 × 2 ANOVA with the factors navigationdevice and navigation type revealed. No significant main effectof information type or an interaction in trust ratings was found.

Usefulness. Usefulness ratings of navigation devices, whichwere rather high (Musefulness = 5.19, SD = 0.66, Min = 3.0,Max = 6.0), were significantly influenced by the device type:The usefulness of the mobile screen was perceived to be higherthan that of the projector (Mscreen = 5.33, SD = 0.52,Mprojector = 5.04, SD = 0.86; F(1, 23) = 59.86; P < 0.01,η2 = 0.51). The interaction between navigation device andinformation type indicated that the usefulness of the map onthe screen was the highest (Mmap,screen = 5.42, SD = 0.58;F(1, 23) = 5.3; P < 0.05, η2 = 0.18).

Workload. The usage of the navigation devices imposed arather low cognitive workload (Fig. 8).



Figure 8. Mean TLX scores in the four navigation aid conditions (n = 24). Error bars show standard deviations.

Table 2. Means and standard deviations of TLX dimensionsfor the factors ‘navigation device’ (screen vs. projector) and‘information type’ (map vs. arrow).

Screen ProjectorM SD M SD

Physical demand 7.81 6.73 11.46 9.55Effort 9.48 8.82 13.33 10.83Frustration 6.98 8.69 12.08 11.6

Map ArrowM SD M SD

Physical demand 8.65 7.66 10.63 7.98

On a scale ranging from 0 to 100 the mean general TLX scorein the study was MTLX = 12.02 (SD = 7.36, Min = 1.92,Max = 31.75). However, the overall workload was higher whenusing the projector (Mprojector = 13.81, SD = 8.54) in contrastto the screen (Mscreen = 10.22, SD = 8.38; F(1, 23) = 4.4;P < 0.05, η2 = 0.16). The highest workload was reportedwhen navigating with the projected arrow (Marrow,projector =14.48, SD = 9.79); the lowest workload was reported for themap displayed on the screen (Mmap,screen = 10.06, SD = 9.8;t (23) = 2.1; P < 0.05). Neither a main effect of informationtype nor an interaction between device type and informationtype was found. The analysis of the individual components ofthe TLX scores showed significant effects of the navigationdevice (Table 2).

Using the projector was perceived as physically moredemanding (F(1, 23) = 6.6; P < 0.05); it required more effort(F(1, 23) = 4.8; P < 0.05, η2 = 0.22) and led to a higherfrustration (F(1, 23) = 8.6; P < 0.01). Users also reporteda higher physical demand in the arrow mode than in the mapmode (F(1, 23) = 4.7; P < 0.05, η2 = 0.27). No further maineffects or interactions were found for the sub-components ofthe workload.

Figure 9. Interaction of spatial abilities and information typefor PPWS (n = 24). Higher values indicate better wayfindingperformance.

6.3. Effects of users’ spatial ability on n.s. andperceptions

Navigation performance. To investigate the influence ofusers’ spatial ability on the effectiveness of the navigationaids and information type, the between-factor spatial abilitywas included into statistical analyses. A 2(navigation aid) ×2(information type) × 2(learnability) × 2(spatial ability)ANOVA with repeated measurement revealed a significantinteraction between spatial ability and information type(F(1, 22) = 4.3; P < 0.05, η2 = 0.16). Especially users withlow spatial abilities benefited from the map information givenon the navigation device, whereas their PPWS significantlydecreased in the arrow mode (Mmap = 0.90, SD = 0.08;Marrow = 0.83, SD = 0.97; t (11) = 4.1; P < 0.01; seeFig. 9).

The PPWS of users with high spatial abilities was notsignificantly influenced by the information type (Mmap = 0.86,


12 Ming Li et al.

Figure 10. Left: preferred navigation aid for users with high and low spatial abilities, Center: interaction of spatial abilities and information type forprivacy concerns. High values indicate low privacy concerns, right: interaction of spatial abilities and information type for perceived performance.Higher values indicate lower perceived performance (n = 24).

SD = 0.08; Marrow = 0.84, SD = 1.0). No significantmain effect of spatial ability or an interaction with the factorsnavigation device and learnability was found.

Preferences. The inclusion of spatial abilities into descriptivepreference analyses showed that users with high spatial abilitiesclearly preferred the map displayed on the screen (91.7%),followed by the arrow displayed on the screen (8.3%, seeFig. 10). Even though a higher proportion of users with lowspatial abilities chose the arrow displayed on the screen (25%)as their preferred navigation aid, the majority of low spatialusers also preferred the map displayed on the screen (75%).Further effects of spatial ability on preferences were not found(see Fig. 10, left).

Privacy. Privacy concerns when using the navigation aidswere also influenced by users’ spatial abilities, as the significanteffect of the factor spatial ability (F(1, 21) = 4.7; P < 0.05,η2 = 0.05) and the interaction between navigation deviceand spatial ability indicated (F(1, 22) = 4.15; P < 0.05,η2 = 0.16; see Fig. 10, center).

Although privacy concerns were quite low for the wholesample, participants with high spatial abilities reported higherprivacy concerns than those with low spatial abilities (note:high values indicate low privacy concerns; Mlowspatials = 5.18,SD = 0.95; Mhighspatials = 4.31, SD = 0.97). Moreover, userswith high spatial abilities reported higher privacy concernswhen using the projector than the screen (Mprojector = 4.13,SD = 1.54; Mscreen = 4.58, SD = 0.97; t (11) = 1.96;P < 0.1). These privacy concerns referred to the aspect thatothers can see the navigation information as well and thereforeknow about the route someone wants to take. For participantswith low spatial abilities, it made no difference in perceivedprivacy if the navigation information was presented on thescreen or projected on the floor (Mscreen = 4.83, SD = 1.19;Mprojector = 4.88, SD = 1.21, n.s.).

Workload. General workload scores did not differ in userswith high or low spatial abilities. For the TLX sub-dimension‘perceived performance’—which was measured by the item‘How successful were you in accomplishing what you wereasked to do?’—a 2(navigation aid) × 2(information type) ×2(spatial ability) ANOVA with repeated measurement yieldeda significant interaction between spatial ability and informationtype (F(1, 22) = 5.1; P < 0.05, η2 = 0.18). In Fig. 11(right), the interaction is depicted (note: high values indicatea low perceived performance).

For users with low spatial abilities perceived performancewas higher in the map mode compared with the arrow mode(Marrow = 18.13, SD = 15.12; Mmap = 13.13, SD = 14.93;n.s.). For users with high spatial abilities, the opposite patternwas found: Perceived performance was higher in the arrowmode compared with the map mode (Marrow = 10.0, SD =7.39; Mmap = 14.79, SD = 12.99; n.s.). In both ability groupst-Tests on perceived performance differences did not showsignificance, which might be attributed to very high standarddeviations of TLX scores.

Legibility, disorientation and trust. Legibility, disorientationand trust ratings were not affected by spatial ability. Thus, userswith high and low spatial abilities judged the legibility of thenavigation information given on the navigation devices alikeand perceived similar (low) levels of disorientation and trust.

6.4. Prediction of user satisfaction

To analyze which factors contribute the most to end-usersatisfaction when using a specific navigation device, weconducted a set of stepwise multivariate linear regressionanalyses. Especially for projector usage, we assumed that userperceptions regarding navigation satisfaction were influencedby legibility problems as well as by trust or privacy concerns.



Figure 11. A concept sketch of the smart map, the map zoom level being decided by the distance to the next turning point. Left: when the user isfar away from the turn; middle: when the user is approaching the turn; right: when the user arrives at the turn.

By means of the regression analysis, we wanted to find out whichfactors influence satisfaction at all, and determine the relativepredictive power or ‘weight’ of the contributing factors. For theprediction of navigation satisfaction (as a criterion variable),we entered n.s. and user perceptions (legibility, workload, trust,usefulness, privacy, disorientation) as predictor variables. Aschanging light conditions were reported as a major interferencefactor during projector usage, we also included legibilityproblems due to changing light conditions as a predictor variableinto the projector regression analysis. To identify effects ofmulticollinearity (biases in the regression model caused byhigh intercorrelations between the predictors), appropriatecollinearity diagnosis was carried out.A variance inflation factor(VIF) >5 indicates multicollinearity problems. As all VIFs in

our regression analyses were <5, multicollinearity problemscan be ruled out.

Prediction of screen satisfaction. The regression modelfor the prediction of screen satisfaction was statisticallysignificant at α < 0.01%. Adjusted R2 values suggest that trust,perceived disorientation, trust and workload predict 51% ofthe variance in user satisfaction with the mobile screen; seeTable 3.

Satisfaction with the mobile screen was high when usersperceived high trust in navigation information, perceived a lowworkload and low levels of disorientation. Interestingly, furtheruser perceptions such as privacy concerns, perceived usefulnessof the navigation device or n.s. did not significantly affect usersatisfaction when navigating using the screen.


14 Ming Li et al.

Table 3. Stepwise regression analysis on screen satisfaction.

Adj R2 VIF Predictor Beta P t

0.51 1.27 Trust 0.65 0.001 3.961.27 Workload −0.49 0.01 −2.941.19 Disorientation −0.38 0.05 −2.37

Table 4. Stepwise regression analysis on projectorsatisfaction.

Adj R2 VIF Predictor Beta P t

0.55 1.1 Disorientation −0.64 0.001 −4.461.1 Performance 0.35 0.05 2.40

Prediction of projector satisfaction. Focusing on the end-users’ satisfaction after projector usage, the regression modelexplained 55.4% of variance in their satisfaction with themobile screen (α < 0.001). Contributing predictors of projectorsatisfaction were perceived disorientation and n.s. (PPWS); seeTable 4.

This indicates that users who experienced low levels ofdisorientation during the navigation and achieved a highern.s. (low walking interference while using the projector),were significantly more satisfied with the projector as anavigation aid. Contrary to expectations, legibility problemsdue to changing light conditions did not significantly affect usersatisfaction after projector usage. Moreover, neither concernsabout trust or privacy nor perceived workload were significantpredictors of projector satisfaction.

7. DISCUSSION

In the following, we discuss the effects of the two differentnavigation aids and information types as well as the impactof the users’ spatial ability on wayfinding performance andthe users’ preferences and perceptions. Furthermore, theimplications of our study findings for interface designers andtechnical developments as well as potential indoor navigationapplication scenarios are outlined.

7.1. The utility of navigation devices: screen vs. projector

The findings of our benchmark evaluation congruentlydemonstrated that the current projector-based navigation aidprototype was far inferior to the ‘conventional’ mobile screen-based navigation aid. Compared with the mobile screen as abenchmark, the projector apparently exerted an intrusive effect,which was deduced from the decreased PWS when navigatingwith the projector. The low utility of the projector as a navigationaid was also perceived by users themselves—they reporteda higher workload when using the projector when comparedwith the mobile screen navigation aid. The analysis of TLXsub-dimensions revealed that users perceived higher physical

demands, effort and frustration when using the projector. Ascurrently only early prototypes of pico projectors are availableand an early prototype of a projector-based navigation aid,which was bulkier and heavier than the lightweight screen-based navigation device, was used in the study, these reportsmight be mainly due to the hardware characteristics of theprojector. However, as the single navigation routes were rathershort and participants had a lot of breaks while answeringthe post-condition questionnaires, the physical demands ofholding the projector could be handled without difficulty.The latter is corroborated by the fact that none of the usersspecifically complained about the weight or size of the projector.Instead, the predominant complaint referred to suboptimalvisual ergonomics, mainly the changing lighting conditions,which reduced the legibility of the projection. Overall, however,it should be noted that both navigation devices imposed quitelow workload levels on the users. With respect to the generalutility of novel navigation devices, both tested devices seemto be basically suitable for ‘real-world’ navigation purposes.Future technical research activities should nevertheless focus onsmaller, lighter and brighter projector interfaces to (i) removehardware limitations and (ii) improve usability and convenienceof mobile projectors.

Not only in the performance and workload evaluations butalso regarding user preferences, the mobile screen was clearlypreferred over the projector. This might be explained by thehigher familiarity with conventional screen-based navigationaids and ‘typical’ renunciative reactions toward novel technicalsystems (Rogers, 2003). Privacy concerns were identified asanother barrier to projector acceptance. Some users wereconcerned that others could think they would need assistance(which was interpreted as stigmatization, even if it was onlyusing a navigation device). Also, users felt uncomfortable thatothers could see in which direction they were heading.

One potential way to reduce privacy exposure is projectingthe information only when one nears turning points or uponthe user’s request. Regarding the most effective navigationdevice for indoor pedestrian navigation, we found a clear answerbased on the results of our study: the mobile screen is stronglyrecommended for indoor navigation purposes.

7.2. The effectiveness of navigation information: map vs.arrow

In spatial cognition theories, survey knowledge is regardedas the most elaborate type of spatial knowledge (Goldin andThorndyke, 1981). Accordingly, spatial information given ina map is supposed to be the most beneficial for navigationpurposes. The findings of our study confirm this assumption: themap, which also provides survey knowledge—at least for thevisible display window—was more useful and barely interferedwith the normal walking patterns of our participants as opposedto the location-adaptive arrow, which only points into thecorrect navigation direction. Beyond this, the superiority of the



map information was confirmed by the users’ preference forthis information type. In contrast, navigating with the arrowdisrupted normal walking much more and was less preferred byusers. Our expectations that users would benefit from the arrow,as they would just have to follow the turn-by-turn instructions,were not fulfilled. Instead, users reported that they felt undercontrol and completely dependent on the arrow because theydid not receive information about the following route sequencein advance. Apparently, the current arrow mode provoked afeeling of ‘loss of control’ in the participants which couldbe due to the fact that, in the current version, the arrow justchanges direction when the user approaches the turning point.It has been shown recently that receiving information aheadof time considerably benefits the n.s. in mobile phone menus(Ziefle et al., 2007). Future studies will have to find out if theperceived utility increases if the arrow condition is amendedwith some distance information allowing users to anticipateroute changes during navigation. Another cautionary note iswith regard to the suitability of the arrow for the young andtechnology-experienced participant group, as examined here.There might be other user groups that could profit from this kindof simple navigation information. Seniors, for example, usuallyexperience more difficulties (Rau and Wang, 2003) whennavigating unfamiliar locations and are easily overwhelmed byprocessing too much information at a time. In their case, thearrow could be a helpful navigation aid as it gives precise anduncomplicated information.

Based on the current findings, we learned that navigationinformation should provide survey knowledge and advanceinformation about the route and destination. However, wethink the arrow is a valuable supplement to map-likenavigation information, as its location-adaptive informationmight facilitate reorientation in case of getting lost or easeturning decisions at junctions with many branches. Futurestudies, therefore, should focus on an arrow that is sensitiveto the distance to the turning point as well as investigate acombination of both information types. For example, an arrowshould be able to indicate the distance to the next turn/waypointby a changing shape (or color). Another possibility could bea 3D route instead of an arrow, i.e. the projection of a virtualroute on the floor. The user could also use the projector as aflashlight to reveal the ‘invisible’ route (Winkler et al., 2011).Landmarks could be displayed to provide users with more visualhints. For example, depending on the current position of the user,the nearby landmarks could be retrieved from a database anddisplayed as 2D icons or 3D objects at a position relative to thevirtual route in the projected image.

7.3. Determinants of users’ satisfaction when usingnavigation devices

The present study also uncovered major influential factors onuser satisfaction. The knowledge about the determinants ofuser satisfaction can be used to guide future R&D activities

for indoor navigation system design. Regression analysesrevealed that perceived disorientation was an important factor inpredicting users’satisfaction for both navigation devices. For themobile screen, trust and workload were even stronger predictorsof user satisfaction. Combining these three determinants ofmobile screen, user satisfaction gives a clear message regardingnavigation system design: during indoor navigation the userdemands reliable, valid and easily understandable navigationinformation that protects him/her from disorientation andfrom elevated cognitive effort while decoding, transformingand applying navigation information, and is trustworthyenough to be followed. One way to reduce perceptions ofdisorientation could be to enrich the map with landmarkinformation. Although users often reported visibility andlegibility difficulties, especially when using the projector, wecould not detect a statistically meaningful impact on the users’satisfaction. Instead, disorientation and n.s. were identifiedas key predictors of projector satisfaction. Nevertheless, toreduce legibility problems (especially in daylight conditions),a brighter projector or a miniaturized laser projector should beused in future studies.

However, it has to be noted that regression models in thisstudy explain only 51–55% of the users’ satisfaction. Thisindicates that further relevant factors need to be identified whichmight affect satisfaction after using the navigation devices(Arning et al., 2012). Interestingly, concerns regarding privacyor trust were not dominant in user perceptions. Especiallyfor projector usage, a higher importance of trust and privacyconcerns in the explanation of user satisfaction due to the ‘publicdisplay’ of navigation information (Rogers and Fisk, 2010,Wilkowska and Ziefle, 2012; Ziefle et al., 2011) could havebeen expected. However, concerns regarding privacy and trust inubiquitous computing environments should not be dropped fromthe research agenda as they might cause acceptance barriers inother user groups (frail or older users) or application contexts(e.g. emergency situations).

7.4. The impact of spatial abilities

As spatial abilities have a strong impact on navigation,we assumed that users’ spatial abilities would considerablyinfluence wayfinding performance. Moreover, as users with lowspatial abilities usually have higher disorientation problemsin navigation (Hegarty et al., 2003, 2006; Rau and Wang,2003; Ziefle and Bay, 2006), we wanted to find out ifa specific navigation aid and navigation information typewas especially suitable for this user group. Our findingsconvincingly demonstrated that users with low spatial abilitiesespecially benefit from the map information, regardless ofwhich navigation device is used. Apparently, the map providesadequate spatial information that is needed, successfullyprocessed and applied by users with low spatial abilities.They even reached the performance level of users with highspatial abilities. The design and application of the pedestrian


16 Ming Li et al.

indoor navigation devices sub-study can therefore be regarded asuccess as navigational performance differences between usersof high and low spatial abilities were diminished.

However, in order to estimate the effect of navigation devicesand to exclude an ‘expertise reversal effect’, i.e. negative effectsof the navigation devices on the n.s. of high spatial abilityusers, future studies should include a control group that doesnot receive navigational support at all. The advantage of themap was also shown by the higher performance after navigatingwith the map. Therefore, indoor navigation aids for pedestriansshould always provide map information that is useful for allusers (‘design for all’), regardless of their spatial abilities.Interestingly, users with high spatial abilities expressed higherprivacy concerns when using the projector. We assume thatother factors that are typically associated with high spatialability, such as higher technical experience or a technical self-efficacy or technical experience (Arning and Ziefle, 2009),caused privacy concerns. These users might have a higherawareness for data privacy issues or might feel ‘stigmatized’when others see them using an assistive device. Future researchactivities will have to investigate the exact causes of privacyconcerns and develop possible countermeasures.

8. LIMITATIONS OF THE PRESENT STUDY ANDDESIDERATA FOR FUTURE RESEARCH

Finally, some limitations of the current approach are described,as are desiderata for future research. Additionally, designimplications and application scenarios are outlined that couldbe pursued in further experimental studies.

8.1. Methodological issues

For future evaluation studies of indoor navigation aids, werecommend the examination of a control group without anynavigational support, and of longer and more complex routes,the analysis of further n.s. parameters such as navigationalsuccess or detouring behavior, an experimental control ofillumination (e.g. Taylor and Sokov, 1974) and the assessmentof further user factors (such as technical self-efficacy andinnovativeness). Moreover, we should note that the young usergroup examined here might not be representative for the wholegroup of potential users (frail users and those affected bycognitive aging processes) who could also benefit from thistechnology.

Future studies will therefore have to replicate the findingswith older and less technology-experienced persons (Ziefle,2010). To evaluate perceived usefulness and satisfaction in ‘real’indoor scenarios, the mobile projector should be evaluated, e.g.in an emergency scenario in which ambulance staff needs to bequickly guided to a patient, or in wayfinding situations of olderusers (when senior users need to find their requested productsin a supermarket). The quantitative evaluation of navigation

devices should be supplemented by qualitative interviewsthat could help identify user requirements in further casesof use (e.g. emergency scenario), potential frustration causesand acceptance barriers as well as further predictors of usersatisfaction.

8.2. Design implications and application scenarios

Navigation devices. According to the users’ ratings, themain influences on projection legibility were changing lightconditions and the ground texture. In future studies, a smaller,brighter projector interface should be prototyped and evaluated.Additionally, a miniaturized laser projector casting vectorgraphics that could be strong enough to be visible even indaylight should be designed and utilized. To adapt the projectedimage contrast to the ground texture, a projector–camera systemand image processing techniques should be considered (Fujiiet al., 2005; Nayar et al., 2003).

Navigation information. According to the impact of spatialability, environment information is beneficial to users of low andhigh spatial ability. Therefore, such information should alwaysbe provided. For example, a ‘smart map’can display the map andchange the zoom level depending on the distance to the turningpoint, i.e. when far away from the turn we see an overview ofthe environment; when approaching the turn we see a zoom-inview focusing on the turning junction; see Fig. 11.

This combines the advantages of map and arrow, providingan environmental overview in advance and reducing wayfindingcomplexity at a junction. Another example is a flashlight ARroute (Winkler et al., 2011) in which a 3D route is generatedfrom waypoints and rendered onto the floor (like invisible routesigns printed on the ground). The users find and follow the routeusing a virtual flashlight, the mobile device; see Fig. 12.

Environment information such as POIs and landmarks can berendered in an AR manner, which can enrich the navigationalinformation (Downs and Stea, 2011; Lynch, 1960; Werner et al.,1997).

To ensure the users’ wayfinding privacy, audio or hapticnotification could be used to inform them when to project thenavigation information, e.g. the projection is off when the useris far away from the next turning point. Only when approachingthe turning point, the projection will show and the user will benotified to look at the projected information.

Application scenarios. In the presented study, we onlyevaluated a general indoor navigation scenario, which wasa handheld navigator designed for a single user. A mobileprojector, however, could also be mounted on the user’s body(Tajimi et al., 2010) as a wearable device, which frees theuser’s hands for other activities, e.g. carrying equipment whilenavigating. A possible application scenario for a wearableprojector-based navigation device is an emergency scenarioin which the paramedic receives navigational support whilefinding his/her way to a patient in a building. As paramedicstaff usually do not have a free hand to hold an indoor navigation



Figure 12. A concept sketch of the flashlight AR route. A rendered 3D route is projected on the floor of the corridor as a navigational sign printedon the ground. The user operates a mobile projector as a flashlight to discover the turning instructions.

device while carrying their equipment (e.g. medical kit, cardiacmonitor, stretchers and boards), a projector could be attachedto their belt or uniform. We assume that our existing researchexperience of indoor navigation cannot be easily transferred toan emergency scenario as it is affected by further influentialfactors, such as time pressure in a highly dynamic situation,motion (running or climbing stairs; e.g. Vilar et al., 2013).Another possible scenario is wayfinding in group navigation, i.e.output route and environment information via a mobile projectorand share it among a group of visitors, e.g. in a museum (Weckeret al., 2011).

ACKNOWLEDGEMENTS

The authors express their thanks to participants for theirwillingness to volunteer in the study and moreover, their thanksto Simon Himmel, Julia van Heek and Chantal Lidynia forresearch support.

FUNDING

This research has been funded by the Excellence Initiativeof German state and federal governments (Project ‘PINT-Pedestrian Indoor Navigation Toolbox’, no. OPPa191).

REFERENCES

Adams, D.A., Nelson, R.R. and Todd, P.A. (1992) Perceivedusefulness, ease of use, and usage of information technology: areplication. MIS Quart., 16, 227–247.

Arikawa, M., Konomi, S. and Ohnishi, K. (2007) Navitime: supportingpedestrian navigation in the real world. IEEE Pervasive Comput.,6, 21–29. http://www.computer.org/csdl/mags/pc/2007/03/b3021-abs.html.

Arning, K. and Ziefle, M. (2007) Understanding differences in PDAacceptance and performance. Comput. Hum. Behav., 23, 2904–2927.

Arning, K. and Ziefle, M. (2009) Effects of age, cognitive, and personalfactors on PDA menu navigation performance. Behav. Inf. Tech-nol., 28, 251–268. http://www.tandfonline.com/doi/abs/10.1080/01449290701679395.

Arning, K., Gaul, S and Ziefle, M. (2010) ‘Same Same But Different’:How Service Contexts of Mobile Technologies Shape UsageMotives and Barriers. Proc. 6th Int. Conf. HCI in Work andLearning, Life and Leisure: USAB’10, pp. 34–54. Springer, Berlin,Heidelberg. http://dl.acm.org/citation.cfm?id=1947789.1947793.

Arning, K., Ziefle, M., Li, M. and Kobbelt, L. (2012) Insights into UserExperiences andAcceptance of Mobile Indoor Navigation Devices.11th Int. Conf. Mobile and Ubiquitous Multimedia. ACM, NewYork. http://dl.acm.org/citation.cfm?id=2406417.

Bay, S. and Ziefle, M. (2003) Design for All: User Characteristics tobe Considered for the Design of Devices with Hierarchical MenuStructures. In Luczak, H. and Zink, K.J. (eds), Human Factors inOrganizational Design and Management, pp. 503–508. IEA, SantaMonica.

Bousbia-salah, M. and Fezari, M. (2006) The development ofa pedestrian navigation aid for the blind, 1–5, doi:10.1109/IEEEGCC.2006.5686241.

Davis, F.D. (1989) Perceived usefulness, perceived ease of use,and user acceptance of information technology. MIS Quart., 13,319–340.


http://www.computer.org/csdl/mags/pc/2007/03/b3021-abs.html

http://www.computer.org/csdl/mags/pc/2007/03/b3021-abs.html

http://www.tandfonline.com/doi/abs/10.1080/ 01449290701679395

http://dl.acm.org/citation.cfm?id=1947789.1947793

http://dl.acm.org/citation.cfm?id=2406417

18 Ming Li et al.

Dee-Lucas, D. (1996) Effects of Overview Structure on StudyStrategies and Text Representations for Instructional Hypertext. InRouet, J.-F., Levonen, J., Dillon, A. and Spiro, R. (eds), Hypertextand Cognition, pp. 73–107. Lawrence Erlbaum, Mahwah, NJ.

Dillon, A. and Morris, M.G. (1996) User acceptance of newinformation technology: theories and models. Annu. Rev. Inf. Sci.Technol., 31, 3–32. http://hdl.handle.net/10150/10558.

Dillon, A. and Watson, C. (1996) User analysis in HCI the historicallessons from individual differences research. Int. J. Hum.-Comput.Stud., 45, 619–637.

Downs, R.M. and Stea D. (2011) Cognitive Maps and SpatialBehaviour: Process and Products. John Wiley & Sons, Ltd,Chichester, UK.

Ekstrom, R., French, J., Harman, H. and Dermen, D. (1976) Kit ofFactor-Referenced Cognitive Tests. Educational Testing Service.

Evans, H. and Simkin, M. (1989) What best predicts computerproficiency? Commun. ACM, 32, 1322–1327.

Fox, J.E. (1996) The Effects of InformationAccuracy on User Trust andCompliance. Conf. Companion on Human Factors in ComputingSystems: Common Ground, CHI ’96, pp. 35–36. ACM, New York,doi:10.1145/257089.257119.

Fujii, K., Grossberg, M. and Nayar, S. (2005) A Projector-Camera System with Real-Time Photometric Adaptation forDynamic Environments. ComputerVision and Pattern Recognition.CVPR 2005. IEEE Computer Society Conf., Vol. 2, p. 1180,doi:10.1109/CVPR.2005.42.

Goldin, S.E. and Thorndyke, P.W. (1981) U.A. R. I. for the Behavioral,S. Sciences, Spatial Learning and Reasoning Skill. R (Rand Corpo-ration), Rand. http://books.google.de/books?id=k-pTAAAACAAJ.

Goodman, J., Gray, P., Khammampad, K. and Brewster, S. (2004a)Using Land-Marks to Support Older People in Navigation. InBrewster, S. (ed.), Proc. Mobile HCI 2004, Lecture Notes inComputer Science 3160, Glasgow, UK, September 13–16, pp. 38–48. Springer, Berlin.

Goodman, J., Brewster, S and Gray, P (2004b) Using Field Experimentsto Evaluate Mobile Guides. Proc. HCI in Mobile Guides, MobileHCI 2004. September 13–16, Glasgow, UK.

Goodman, J., Brewster, S. and Gray S. (2005) How can we bestuse landmarks to support older people in navigation? Behav. Inf.Technol., 24. 3–20.

Greaves, A., Akerman, P., Rukzio, E., Cheverst, K. and Hakkila, J.(2009) Explo-ring User Reaction to Personal Projection Whenused in Shared Public Places: A Formative Study. Context-AwareMobile Media and Mobile Social Networks. http://comp.eprints.lancs.ac.uk/2264/1/cam3sn2009_greaves.pdf.

Hart, S. and Staveland, L. (1988) Development of nasa-tlx (task loadindex): Results of Empirical and Theoretical Research. In Hancock,P.A. and Meshkati, N. (eds), Human Mental Workload, pp. 139–183. Elsevier, Amsterdam.

Hegarty, M., Kriz, S. and Cate, C. (2003) The roles of mentalanimations and external animations in understanding mechanicalsystems. Cogn. Instr., 21, 325–360.

Hegarty, M., Montello, D.R., Richardson, A.E., Ishikawa, T. andLovelace, K. (2006) Spatial abilities at different scales: individual

differences in aptitude test performance and spatial-layout learning.Intelligence, 34, 151–176. http://www.sciencedirect.com/science/article/pii/S0160289605000905.

Hunt, E. and Waller, D. (1999) Orientation and wayfinding: a review.Technical Report.

Kim, H. and Hirtle, S. (1995) Spatial metaphors and disorientation inhypertext browsing. Behav. Inf. Technol., 14, 239–250.

Kirasic, K. (2000) Age differences in adults’ spatial abilities, learningenvironmental layout, and wayfinding behavior. Spat. Cogn.Comput., 2, 117–134.

Kuniavsky, M. (2010) Smart Things: Ubiquitous Computing UserExperience Design. Morgan Kaufmann. http://www.lavoisier.fr/livre/notice.asp?id=OR3W2LAKS66OWR.

Link, J.B., Smith, P., Viol, N. and Wehrle, K. (2013) Accurate map-based indoor navigation on the mobile. J. Locat. Based Serv., 7,23–43.

Lynch, K. (1960) The Image of the City. The MIT Press.

May,A.J., Ross, T., Bayer, S.H. and Tarkiainen, M.J. (2003) Pedestriannavigation aids: information requirements and design implications.Pers. Ubiquit. Comput., 7, 331–338.

McGee, M. (1979) Human spatial abilities: psychometric studies andenvironmental, genetic, hormonal, and neurological influences.Psychol. Bull., 86, 889–918.

Millonig, A. and Schechtner, K. (2007) Developing landmark-basedpedestrian-navigation systems. IEEE Trans. Intell. Transp. Syst., 8,43–49.

Mulloni, A., Seichter, H. and Schmalstieg, D. (2011) HandheldAugmented Reality Indoor Navigation with Activity-BasedInstructions. Proc. 13th Int. Conf. Human Computer Interactionwith Mobile Devices and Services, Mobile HCI’11, Stockholm,August 30–September 2, pp. 211–220. ACM, New York.

Nayar, S.K., Peri, H., Grossberg, M.D. and Belhumeur, P.N. (2003)A Projection System with Radiometric Compensation for ScreenImperfections. Proc. ICCV Int. Workshop on Projector CameraSystems. October 11–12, Nice, France.

Nokia (2010) Nokia indoor navigation. http://research.nokia.com/research.

Pak, R., Czaja, S.J., Sharit, J., Rogers, W.A. and Fisk, A.D. (2008)The role of spatial abilities and age in performance in an auditorycomputer navigation task. Comput. Hum. Behav., 24, 3045–3051.http://www.sciencedirect.com/science/article/pii/S0747563208001088.

Pinhanez, C.S. (2001) The Everywhere Displays Projector: A Deviceto Create Ubiquitous Graphical Interfaces. Proc. 3rd Int. Conf.Ubiquitous Computing, UbiComp ’01, pp. 315–331. Springer,London. http://dl.acm.org/citation.cfm?id=647987.741324.

Rau, P.-L. and Wang,Y.-J. (2003) A Study of Navigation Support Toolsfor Mobile Devices. Proc. HCI Int., Crete, Greece, June 22–27.

Rogers, E.M. (2003) Diffusion of Innovations. 5. Auflage, Free Press,New York, NY.

Rogers, W.A. and Fisk,A. D. (2010) Toward a psychological science ofadvanced technology design for older adults. J. Gerontol. Psychol.Sci., doi:10.1093/geronb/gbq065.


http://hdl.handle.net/10150/10558

http://books.google.de/books?id=k-pTAAAACAAJ

http://comp.eprints.lancs.ac.uk/2264/1/cam3sn2009_greaves.pdf

http://comp.eprints.lancs.ac.uk/2264/1/cam3sn2009_greaves.pdf

http://www.sciencedirect.com/science/article/pii/S0160289605000905


http://www.lavoisier.fr/livre/notice.asp?id=OR3W2LAKS66OWR

http://www.lavoisier.fr/livre/notice.asp?id=OR3W2LAKS66OWR

http://research.nokia.com/research

http://research.nokia.com/research



http://dl.acm.org/citation.cfm?id=647987.741324


Rukzio, E., Schmidt, A. and Krüger, A. (2005) The Rotating Compass:A Novel Interaction Technique for Mobile Navigation. CHI ’05Extended Abstracts on Human factors in Computing Systems, CHIEA ’05, Portland, OR, USA, April 2–7, pp. 1761–1764. ACM,New York.

Rukzio, E., Holleis, P. and Gellersen, H. (2011) Personal pro-jectors for pervasive computing. IEEE Pervasive Comput., 99,doi:10.1109/MPRV.2011.17.

Serra, A., Carboni, S and Marotto, V. (2010) Indoor PedestrianNavigation System Using a Modern Smartphone. Proc. 12thInt. Conf. Human Computer Interaction with Mobile Devicesand Services, MobileHCI ’10, pp. 397–398. ACM, New York.http://doi.acm.org/10.1145/1851600.1851683.

Tajimi, K., Uemura, K., Sakata, N. and Nishida, S. (2010) StabilizationMethod for Floor Projection with a Hip-Mounted Projector. Proc.20th Int. Conf. on Artificial Reality and Telexistence (ICAT2010),Aadelade, Australia, pp. 77–83.

Taylor, L. H., and Sucov, E. W. (1974) The movement of people towardlights. J. Illum. Eng. Soc., 3, 237–241.

Thorndyke, P.W. and Hayes-Roth, B. (1982) Differences inspatial knowledge acquired from maps and navigation. Cogn.Psychol., 14, 560–589. http://linkinghub.elsevier.com/retrieve/pii/0010028582900196.

Tolman, E.C. (1948) Cognitive maps in rats and men. Psychol. Rev.,55, 189–208.

Vilar, E., Rebelo, F., Noriega, P., Teles, J. and Mayhorn, C. (2013)The influence of environmental features on route selection in anemergency situation. Appl. Ergon., 44, 618–627.

Wecker, A.J., Lanir, J., Kuik, T. and Stock, O. (2011) Pathlight:Supporting Navigation of Small Groups in the Museum Context.Proc. 13th Int. Conf. Human Computer Interaction with MobileDevices and Services, Mobile HCI ’11, Stockholm, August 30–September 2, pp. 569–574. ACM, New York.

Werner, S., Krieg-Brückner, B., Mallot, H., Schweizer, K. and Freksa,C. (1997) Spatial Cognition: The Role of Landmark, Route, andSurvey Knowledge in Human and Robot Navigation. In Jarke, M.,Pasedach, K. and Pohl, K. (eds), Informatik ’97, Aachen, Germany,September 24–26, pp. 41–50. Springer, Berlin.

Wilkowska, W. and Ziefle, M. (2012) Privacy and Data Security inE-health: Requirements from Users’ Perspective. Health Inf. J., 18,191–201. http://jhi.sagepub.com/content/18/3.toc.

Winkler, C., Broscheit, M. and Rukzio, E. (2011) NaviBeam: IndoorAssistance and Navigation for Shopping Malls Through ProjectorPhones. CHI 2011, Workshop on Mobile and Personal Projection,Vancouver, BC, May 7–12.

Ziefle, M. (2010) Information presentation in small screen devices: thetrade-off between visual density and menu foresight. Appl. Ergon.,41, 719–730. http://www.sciencedirect.com/science/article/pii/S0003687010000463.

Ziefle, M. and Bay, S. (2006) How to overcome disorientation inmobile phone menus: a comparison of two different types of nav-igation aids. Hum.-Comput.-Interact., 21, 393–432. http://www.tandfonline.com/toc/hhci20/21/4#.UdkTGxZfRW8.

Ziefle, M., Schroeder, U., Michel, T. and Strenk, J. (2007)How Young and Older Users Master the Use of Hyperlinksin Small Screen Devices. Proc. SIGCHI Conf. Human Fac-tors in Computing Systems, pp. 307–316.ACM, New York,doi:10.1145/1240624.1240676.

Ziefle, M., Röcker, C. and Holzinger A. (2011) Medical Technologyin Smart Homes: Exploring the User’s Perspective on Privacy,Intimacy and Trust. The 3rd IEEE Int. Workshop on SecurityAspects of Process and Services Engineering (SAPSE’11). 35thAnnual IEEE Computer Software and Applications Conf., July 18–22, Munich, Germany, pp. 410–415, doi:10.1109/COMPSACW.2011.75.


http://doi.acm.org/10.1145/1851600.1851683

http://linkinghub.elsevier.com/retrieve/pii/0010028582900196

http://linkinghub.elsevier.com/retrieve/pii/0010028582900196

http://jhi.sagepub.com/content/18/3.toc



http://www.tandfonline.com/toc/hhci20/21/4#.UdkTGxZfRW8

http://www.tandfonline.com/toc/hhci20/21/4#.UdkTGxZfRW8

evaluation of a mobile projector-based indoor navigation...

Documents