See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/333715985

Development and Usability Analysis of a Mixed Reality GPS Navigation

Application for the Microsoft HoloLens

Chapter · June 2019

DOI: 10.1007/978-3-030-22514-8_41

CITATIONS

0READS

141

2 authors, including:

Some of the authors of this publication are also working on these related projects:

A.V.A.T.A.R. View project

Renan Luigi Martins Guarese

Universidade Federal do Rio Grande do Sul

15 PUBLICATIONS   31 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Renan Luigi Martins Guarese on 19 July 2019.

The user has requested enhancement of the downloaded file.

Development and usability analysis of a mixedreality GPS navigation application for the

Microsoft HoloLens?

Renan Luigi Martins Guarese[0000−0003−1206−5701] and AndersonMaciel[0000−0002−0780−6555]

Institute of Informatics, Federal University of Rio Grande do Sul (UFRGS).Porto Alegre, Brazil. {rlmguarese, amaciel}@inf.ufrgs.br

Abstract. Navigation in real environments is arguably one of the pri-mary applications for the mixed reality (MR) interaction paradigm. Wepropose a wearable MR system based on off-the-shelf devices as an alter-native to the widespread handheld-based GPS navigation paradigm. Oursystem uses virtual holograms placed on the terrain instead of the usualheads-up display approach where the augmentations follow the line ofsight. In a user experiment, we assessed performance and usability. Wemonitored user attention through EEG while performing a navigationtask using either the MR or the handheld interface. Results show thatusers deemed our solution to offer a higher visibility to both the on-coming traffic and the suggested route. EEG readings also exposed asignificantly less demanding focus level for our prototype. An easiness tolearn and use was also indicated for the MR system.

Keywords: Mixed and Augmented Reality · Human-Computer Inter-action · GPS Navigation

1 Introduction

When using a Personal Navigation Device (PND) while driving or riding to fol-low a suggested route, vehicle operators have to often take their eyes off the roadahead and onto the GPS navigation screen. Studies have shown that distractionby a navigation device was significantly associated with the most serious inci-dents, pointing out that among the 44 deadly incidents researched, 21 involveddistraction [5].

In this paper, we present the development of a GPS navigation system usingvirtual elements placed at the surface of the road using Mixed Reality (MR).By projecting a path onto the surface of the road, MR objects can be placed ontop of their proper geo-located analogous places, such as minimal curves of the

? This study was partly funded by the Coordenacao de Aperfeicoamento de Pessoalde Nivel Superior - Brasil (CAPES) - Finance Code 001, and partly by CNPq. Wealso acknowledge FAPERGS (project 17/2551-0001192-9) and CNPq-Brazil (project311353/2017-7) for their financial support.

2 R. L. M. Guarese and A. Maciel

road, round-abouts and corners. With the help of these virtual elements, userscan be guided throughout the extent of a route without the need to take theirfocus away from the road. Despite having an ultimate use in motorized vehicles,this preliminary development and subsequent tests were aimed at pedestriansin order to validate the technology before inserting it into more accident-pronescenarios.

2 Related Work

Using either a smartphone or an HMD (Head-Mounted Display), several AR in-door navigation studies have been accomplished. The most relevant ones presentAR-based applications that use pre-scanned environmental features to guideusers inside buildings. Rehman [7] claims that the 3D scanning during the pre-deployment stages was time consuming and complicated, which hampered theirability to conduct large scale tests. Aside from that, their study also exposesthat, when using the AR solutions, subjects presented the lowest workload oftheir tests. Bagling [1] mentions that people using the Hololens (HL) indoors, ina finding objects task, performed better than smartphone and paper maps.

Prior to commercial Head-Up Display (HUD) navigation systems, Madenicaet al. [6] compared the use of an AR HUD PND with two other common PNDmethods by using a high-fidelity driving simulator. A thorough usability studywas made. The results exposed that the HUD option provided for more visualattention at the road ahead. Their AR solution, besides being purely simulated,presented a yellow path projected above the center of the road at a height ofabout 2 meters. In our work, we reproduced their simulated AR solution in areal world MR setting.

3 Methodology

3.1 Design and Implementation

Arguably, a GPS navigator is meant to work ubiquitously, making it intrinsicnot to require markers. This can be accomplished with the HL1, an OpticalSee-through MR HMD. Since the HL does not provide all hardware necessary,a regular smartphone was used as a second device in our design, supplying itwith GPS data and access to the Google Maps Platform. Regarding the develop-ment of the mobile application per se, it creates a TCP server and broadcasts amessage containing the aforementioned information. We developed the holoNavusing the Unity 3D Engine. This MR application receives a route graph thatleads users from their current location to a requested destination, generatingvirtual holograms to guide the user along the path in a turn-by-turn manner.The latitude and longitude of these objects are mapped onto the 3D space aroundthe user, according to their real analogous points in space.

1 https://docs.microsoft.com/en-us/windows/mixed-reality/hololens-hardware-details

Mixed reality GPS navigation application for the Microsoft HL 3

Fig. 1. Examples of route holograms generated by the holoNav application.

Several methods exist to calculate distances between two pairs of coordinates,such as the Haversine Formula [4]. This method was simplified in order to reducethe overhead of the algorithm based on the notion that one degree of latitudecan be roughly approximated to 111111 meters [2]. Thus, it is possible to use theGPS position of the user as an origin point and map all points of a received routegraph onto the X and Y-axis distances away from the user, in a two-dimensionalmanner. The farther the points, the larger the error. Hologram positions areupdated occasionally to fix any misplacements done by the HMD, which mayalso happen when the depth is hard to perceive. To do so, the application requestsa new route, using the current GPS location and resets all path objects. Asidefrom fixing accumulated spatial displacements, resetting also suggests a morerelevant route in case the user goes off track.

The exact front of the user is firstly assumed to be the north direction. Inthis fashion, a yaw rotation is necessary to align it, equivalent to the number ofdegrees this direction is divergent from the factual north. The magnetic northheading received from the smartphone magnetometer is a good starting pointmeasurement. However, the deviation between the true north pole of the Earthand its magnetic equivalent is not constant across the Earth 2. A transient designchoice was made to keep this value ad hoc, using the deviation angle pertaining toour city. As to facilitate with the particular positioning of the phone, the user issuggested to place it according to a hologram in the form of a red arrow pointingahead. This process has to be done only once, since the HL’s accelerometers andgyroscope ensure that all object positions and rotations are maintained.

Relative to the holograms that indicate the suggested route, two kinds ofobjects are generated. At first, green arrows similar to Fig. 1-right are placedin each step of the route, one for each node of the graph. A different 3D arrowobject was designed for each kind of maneuver the mapping API presents, e.g.an arrow pointing right for turn-right and a roundabout figure with an arrowhead pointing left for roudabout-left. A textual description of the current turnis also displayed on the arrow body. The second type of object generated is aregular 2D blue plane parallel to the ground, as in Fig. 1-left. These planes arerectangles that link all of the turns present in the route together, thus showinga virtual path for the user to follow between arrows.

2 https://www.ngdc.noaa.gov/geomag/WMM/

4 R. L. M. Guarese and A. Maciel

3.2 Experimental evaluation

Although an MR interface has intrinsic advantages due to egocentric navigation,a user study is necessary to properly assess its practical use. Our test design com-pares the proposed MR solution with a well established navigation tool. Beingreasonable to assume it as the golden standard, we intend to assess the usabilityand performance gap between our prototype and the widely used navigation ap-plication. In the effort of accomplishing an impartial test, the order of the twotests was alternated for each subject. Tests were taken by users walking on thesidewalk, for the sake of avoiding any possible accidents.

Regarding the quantitative dependent variables, both route time and dis-placement were measured. Attention efficiency was measured by means of aBrain-Computer Interface (BCI) headset3, which reads electroencephalography(EEG) signals and filters them into focused attention data, exposing the userconcentration efficiency over time. The number of times each user checked thehandheld device was also counted.

As for the qualitative attributes, users were asked to answer a survey com-prised of three questionnaires for each test. The first two were traditional multi-dimensional assessment tools, namely the NASA TLX4 (Task Load Index) andthe SUS5 (System Usability Scale). The third set of questions was adapted basedon the Mobile or Projector questionnaire [3], since its comparison is similar toours. All three questionnaires followed the Likert scale [8].

The same route of 350 meters was chosen for both tests, a 4 minute walk withtwo 90-degree turns. One at a time, users were taken to the starting point. Thesystem was preset with the route as to not require any previous knowledge fromsubjects. Subjects were given the device to be tested and minimal instructions.Once at the destination, they were asked to answer the survey. After completion,the procedure was done again using the remaining of the two systems.

4 Results

In total, twelve subjects participated in user assessments (two female), varyingfrom 21 to 35 years of age, average being 26.23 and the standard deviation(SD) 3.21. No effect of eye condition, age or gender was found on the resultingdata. Some subjects performed their tests under very bright sun light, whichwas partially redressed by attaching a piece of dark cellophane sheet to thevisor. Five subjects had problems with the EEG headset coming loose duringthe tests, which made it present a gap amidst the readings. These EEG sampleswere discarded.

4.1 Quantitative results

Fig. 2-left displays the GPS data regarding the path test participants trod.Disregarding the second trial for each user, i.e., accounting only for the tests

3 http://neurosky.com/biosensors/eeg-sensor/biosensors/4 https://humansystems.arc.nasa.gov/groups/TLX/5 https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html

Mixed reality GPS navigation application for the Microsoft HL 5

Fig. 2. Left: Geo-located paths performed by subjects using the MR application. Right:Subject average percentage of time spent in each focused attention level.

in which the subject was not familiar with the path beforehand, the respectivetimes were 3 min 42 s (SD 31.09), and 4 min 51 s (SD 59.84). These resultsexpose a roughly 31% increase in time for the MR application.

The number of times users looked away from the route in order to check thenavigator application in their handheld device varied from 3 to 14 times. Theaverage being 8.15 (SD 3.48) or 2.71 glances away from the road every 100 meters.Regarding the EEG results, the averages of each level of focused attention wasaccounted in fig. 2-right. Joining together the three higher levels of attention,users spent an average of 55.38% of the time at a high level of focus while usingthe smartphone application. This percentage dropped to 35.64% while using theHMD solution.

4.2 Qualitative results

Fig. 3 displays the average score for each statement subjects responded to.Both raw NASA TLX and SUS results are grouped together and indicated byname. The remaining statements represent the third questionnaire used. The un-weighted TLX score was evaluated as 29.81 for the mobile navigator and 39.42for the HMD application. Meanwhile, SUS final scores were 86.6 for the smart-phone application, dropping to 66.6 regarding the MR solution, respectively anA and a C, according to SUS standards.

4.3 Discussion

Users attributed a higher score to the MR solution both in ”route attentiveness”and in ”traffic visibility”, categorizing how easy or seamless it was to perceivethe suggested route and how low it affected the capacity of users to see theoncoming traffic and aspects of the road. The substantial increase in route timecould be explained, for instance, by the ”fun to use” aspect, being arguable that

6 R. L. M. Guarese and A. Maciel

Fig. 3. Qualitative questionnaire results.

Fig. 4. Number of glances at the handheld device vs. EEG efficiency and NASA TLX

users demonstrated a considerably higher interest in the novelty aspect of theholoNav solution.

Ratings such as inconsistency, complexity and system integration could beimproved refining the coordinates mapping algorithms and increasing the screenbrightness. However, the least favorable aspects of the proposed solution arelargely related to the UX of the HMD per se. Mental and physical demand, effortand inconvenience, likeness to use again and safety are all heavily dependant onthe HMD paradigm.Nonetheless, as demand increases and the industry adjuststo it, it is customary for devices to get smaller and faster over time.

Six out of the seven subjects displayed a significantly higher level of attentionduring the handheld device trial. Disregarding the odd one out, an increase of1.94 times was measured, virtually doubling the average attention level of theuser. Arguably, the higher attention required means a greater cognitive load,which would possibly interfere in other attention seeking activities, such as driv-ing or riding a bicycle. The high number of times subjects glanced at their phonescould explain peaks in attention, since mentally translating the route seen in a2D map into the user real surroundings requires some level of concentration.However, no correlation could be observed between these two pieces of data,

Mixed reality GPS navigation application for the Microsoft HL 7

as seen in Fig. 4. Despite a couple of outliers, NASA TLX results shows somerelation to EEG efficiency results, validating user responses to a certain degree.Regardless, further studies should be held with a larger population and in moredemanding guiding conditions in order to obtain a less biased set of data.

5 Conclusion

The present study focused on comparing the map navigation task using a smart-phone versus an original proposed solution. The conception, design and develop-ment of an MR application for an HMD was illustrated, which enables users torequest, perceive and follow a geo-located route in the form of virtual holograms.

We demonstrated the performance and usability of the proposed applicationin objectively guiding subjects throughout an entire real-world route. Such an ac-complishment provides an interesting advantage, whereas mentally interpretinga two dimensional map (allocentric) into the actual streets surrounding the user(egocentric) is not a trivial task, specially in inner-city locations with complexcrossroads and traffic. The MR HMD proved to be a great interface for freeinguser attention, mainly visual, since it follows the HUD paradigm. Despite itslimitations, it was feasible to build a working prototype capable of day-to-daynavigation. This is a highly significant aspect for the current state-of-the-artboth in augmented reality and navigation, since new technologies might emergewith more capable and less effort-consuming devices.

References

1. Bagling, M.: Navigating to real life objects in indoor environments using an Aug-mented Reality headset. Master’s thesis, Umea University, Umea, Sweden (2017)

2. Beding, S.: The Christopher Columbus Encyclopedia. Palgrave Macmillan UK(2016)

3. Dancu, A., Franjcic, Z., Fjeld, M.: Smart flashlight: Map navigation using a bike-mounted projector. In: Proceedings of the SIGCHI Conference on Human Factorsin Computing Systems. pp. 3627–3630. CHI ’14 (2014)

4. Inman, J.: Navigation and Nautical Astronomy for the Use of British Seamen. C.and J.Rivington (1835)

5. Lin, A.Y., Kuehl, K., Schoning, J., Hecht, B.: Understanding ”death by gps”: Asystematic study of catastrophic incidents associated with personal navigation tech-nologies. In: Proceedings of the 2017 CHI Conference on Human Factors in Com-puting Systems. pp. 1154–1166. CHI ’17 (2017)

6. Medenica, Z., Kun, A.L., Paek, T., Palinko, O.: Augmented reality vs. street views:A driving simulator study comparing two emerging navigation aids. In: Proceedingsof the 13th International Conference on Human Computer Interaction with MobileDevices and Services. pp. 265–274. MobileHCI ’11 (2011)

7. Rehman, U.: Augmented Reality for Indoor Navigation and Task Guidance. Master’sthesis, University of Waterloo, Waterloo, Ontario, Canada (2016)

8. Robinson, J.: Likert Scale, pp. 3620–3621. Springer Netherlands, Dordrecht (2014).https://doi.org/10.1007/978-94-007-0753-51654

View publication statsView publication stats