a mobile indoor navigation system interface adapted to vision-based localization
TRANSCRIPT
A Mobile Indoor Navigation System Interface Adapted to Vision-Based Localization
Andreas Möller1, Matthias Kranz2, Robert Huitl1, Stefan Diewald1, Luis Roalter1
1Technische Universität München, Germany2Luleå University of Technology, Department of Computer Science,
Electrical and Space Engineering, Luleå, Sweden
MUM 2012, Ulm, Germany
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Outline
§ Motivation for Vision-Based Navigation§ Challenges§ Proposed User Interface Concepts§ Evaluation§ Discussion and Conclusion
2
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Background and Motivation§ Location information still the
most important contextual information§ Indoor localization is a hot topic
and useful for a lot of scenarios:□ Airports□ Hospitals□ Conference venues□ Large environments
§ Various indoor localization methods possible:□ WLAN/cell-based localization□ Sensor-based localization□ Beacon-based localization□ Vision-based localization
3
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Why vision-based localization?
§ Compare query image to reference data set
§ Use existing features in the environment
§ No additional augmentation needed
§ Modern devices are equipped with a camera
and powerful processor
4
Live features
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Challenges of Vision-Based Localization
§ Query images need to be distinctive□ Repeating structures throughout the environment (e.g. corridors)□ Little visual features (e.g. uniform walls)
§ Orientation of the user’s smartphone□ Good candidates for query images (windows, adverts, posters) are
typically found in eye height □ In a typical pose, user holds the device in a way that it points at
the floor
§ User interface needs to address those challenges
5
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
User Interface Concept
§ No 1 Challenge: Localization inaccuracy□ When insufficient number of features
in the query image
§ Solution A: Adaption of the user interface□ Augmented Reality (AR) with a live view of the environment□ Virtual Reality (VR) with panorama images
(compare to Google Street View)taken in advance
§ Solution B: Corrective actions□ The user needs to point at interesting regions with the camera
6
Ourcontribution: Address problem from a UI perspective
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Augmented Reality (AR)
• Display the live camera image
• Impose navigation instructions over the image
• Smartphone held ineye height
• Direct match of information and the real environment
7
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter 8
Virtual Reality (VR)
• Uses preloaded panorama images
• Navigation instructions are drawn in panorama
• Smartphone can be directed to the ground
• User matches environment and virtual reality
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter 9
Pointing Towards Discriminative Areas
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Highlighting Objects of Interest
10
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Research Questions
§ RQ1: Which concept (AR or VR) is preferable in terms of perceived accuracy?
§ RQ2: Which concept (AR or VR) is preferred by users?
§ RQ3: Which visualizations are appropriate to acquire sufficient visual features?
§ RQ4: Can object highlighting be improved with a soft border visualization?
11
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Methodology
§ Online Survey§ Videos and images of mockup system§ 81 participants (18-59 years) recruited via Mobileworks
12
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
RQ1: Perceived accuracy
§ Simulated error conditions□ No Error□ Location Error
□ Orientation Error□ Both Errors
13
No Error Orientation Error Location Error Loc.+Ori. ErrorResponse Std.Dev. Response Std.Dev. Response Std.Dev. Response Std.Dev.
2.5 0.9 0.8 2.0 1.7 1.5 0.6 2.01.7 1.6 1.4 1.8 1.4 1.8 1.0 1.72.4 1.0 0.2 2.1 1.2 1.8 0.4 2.11.7 1.5 1.1 1.8 1.3 1.7 0.9 1.82.3 1.0 -0.2 2.0 0.4 1.9 -0.5 1.91.8 1.4 1.4 1.6 1.4 1.7 0.9 1.8
Augmented RealityVirtual Reality
No Error Orientation Error Location Error Loc.+Ori. ErrorResponse Std.Dev. Response Std.Dev. Response Std.Dev. Response Std.Dev.
2.5 0.9 0.8 2.0 1.7 1.5 0.6 2.01.7 1.6 1.4 1.8 1.4 1.8 1.0 1.72.4 1.0 0.2 2.1 1.2 1.8 0.4 2.11.7 1.5 1.1 1.8 1.3 1.7 0.9 1.82.3 1.0 -0.2 2.0 0.4 1.9 -0.5 1.91.8 1.4 1.4 1.6 1.4 1.7 0.9 1.8
Augmented Reality 1: The system seemed to know well my location.2: The system seemed to know well my orientation.
Virtual Reality 3: I perceived the navigation instructions as correct.
No Error Orientation Error Location Error Loc.+Ori. ErrorResponse Std.Dev. Response Std.Dev. Response Std.Dev. Response Std.Dev.
2.3 1.0 -0.2 2.0 0.4 1.9 -0.5 1.91.8 1.4 1.4 1.6 1.4 1.7 0.9 1.8
Augmented Reality
Virtual Reality
Perceived Accuracy of Virtual Reality and Augmented Reality Views
my orientation.I perceived the navigation instructions as correct.
Perceived Accuracy of Virtual Reality and Augmented Reality Views
The system seemed to know well my location.The system seemed to know well
3:
Perceived Accuracy of Virtual Reality and Augmented Reality Views
1:
2:
-3: strongly disagree+3: strongly agree
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
RQ2: User Preferences
§ In total, AR more popular with users than VR
14
-3: strongly disagree+3: strongly agree
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
RQ3: Feature Indicator
§ Clarity of the indicator‘s meaning
a)b)c)d)
15
a) b)
c) d)
-3: strongly disagree+3: strongly agree
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
RQ4: Highlightingmethods
§ Objects of interest highlighting□ Potentially feature-rich□ Interaction points for
location-based services§ Two types□ Frame□ Soft border
§ Both equally raise attention, but Frame distracts more
16
-3: strongly disagree+3: strongly agree
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Discussion
§ Accuracy perception through the interface□ VR beneficial for lower localization accuracy□ AR preferred for reliable estimate
§ Feature indicators and object highlights potentially contribute to good reference images
§ Situational use□ VR for 45° angle□ Less features visible□ Less accuracy required
□ AR when holding the phone up□ Target to visual features□ Highlight objects of interaction
17
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Future andOngoing Work§ Evaluating interfaces in the
real world§ Modeling the system’s
and user’s state□ Localization accuracy□ Location/environment□ User context□ User mental model□ …
18
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Thank you for your attention!Questions?
??
19
[email protected]/team/andreas-moeller.html
This research project has been supported by the space agency of the German Aerospace Center with funds from the Federal Ministry of Economics and Technology on the basis of a resolution of the German Bundestag under the reference 50NA1107.
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Paper References
§ Please find the full paper at:http://dx.doi.org/10.1145/2406367.2406372
§ Please cite this work as follows:§ Andreas Möller, Matthias Kranz, Robert Huitl, Stefan Diewald, and Luis
Roalter. 2012. A mobile indoor navigation system interface adapted to vision-based localization. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia (MUM '12). ACM, New York, NY, USA, , Article 4 , 10 pages. DOI=10.1145/2406367.2406372 http://doi.acm.org/10.1145/2406367.2406372
§
20
Technische Universität MünchenInstitute for Media TechnologyDistributed Multimodal Information Processing Group
04.12.2012 A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter
Please use the following BibTex file:§ @inproceedings{Moller:2012:MIN:2406367.2406372, author = {M\"{o}ller, Andreas and Kranz, Matthias and Huitl, Robert and Diewald, Stefan and Roalter, Luis}, title = {A mobile indoor navigation system interface adapted to vision-based localization}, booktitle = {Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia}, series = {MUM '12}, year = {2012}, isbn = {978-1-4503-1815-0}, location = {Ulm, Germany}, pages = {4:1--4:10}, articleno = {4}, numpages = {10}, url = {http://doi.acm.org/10.1145/2406367.2406372}, doi = {10.1145/2406367.2406372}, acmid = {2406372}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {augmented reality, indoor navigation, user interface, virtual reality, vision-based localization},}
[21