27 November 2015

3D Rapid Mapping on Mobile Devices

Clemens Havas, Jakob Miksch, Nikolai Janakiev, Muhammad Salman, Bernd Resch & Thomas Blaschke

WhereCamp 2015

1

Clemens Havas 27 November 2015

� Processing Structure from Motion on standard smartphones in real time

� Navigate through a building by simply taking pictures of the rooms

2

Overview :: Intention

Clemens Havas 27 November 2015

� Smartphone takes pictures

� Comparison with database (server)

� Delivers location

3

Chadly Marouane et al. “Visual Positioning Systems - An Extension to MoVIPS". In: 5th International Conference on Indoor Positioning and Indoor Navigation (IPIN 2014) 2014, pp. 1-10. Martin Werner, Moritz Kessel, and Chadly Marouane. “Indoor positioning using smartphone camera". In: Indoor Positioning and Indoor Navigation (IPIN), 2011 International Conference on IEEE. 2011, pp. 1-6.

Related Work :: Indoor Positioning

Clemens Havas 27 November 2015

� Real time

� 3D model generation

� Processing completely on the smartphone

4

Peter Ondruska, Pushmeet Kohli, and Shahram Izadi. “MobileFusion: Real-time Volumetric Surface Reconstruction and Dense Tracking On Mobile Phones". In: International Symposium on Mixed and Augmented Reality (ISMAR). Fukuoka, Japan, 2015

Related Work :: Mobile Fusion

Clemens Havas 27 November 2015 5

Images

Structure from Motion

Point Cloud Processing

Visualisation

( Fingerprinting + Indoor Positioning)

Overview :: Workflow

Clemens Havas 27 November 2015 6

Name Open Source Lib./Soft. Support

Theia Yes Library University Project

Point Cloud Library Yes Library Companies

LIBVISO2 (No) Library University Project

OpenMVG Yes Library University Project

Aerial Vision Group No Software University Project

aSPECT 3D No Software Company

Depth Image Processing

Yes Library University Project

Visual SFM No Software University Project

SfM :: Finding the best Library/Software

Other criteria

Dependencies

Input/Output

Documentation

API

Features

Performance

Support

Clemens Havas 27 November 2015

� Written in C++

� Collection of tiny open source libraries

� Complete SfM Pipeline

� Incremental and Global Pipeline

� Support by the author: Pierre Moulon

7

https://openmvg.readthedocs.org/en/latest/

SfM :: OpenMVG

Clemens Havas 27 November 2015

� OpenMVG just for Linux and Windows � How to build OpenMVG for Android? � Usage of C++11 Standard: Not fully compatible with

Android � Many external libraries (Eigen, Ceres-Solver,

Cereal,...)

8

SfM :: Challenges

Clemens Havas 27 November 2015

� Solved the stated challenges and succeded in making OpenMVG usable for Android

� Successfully performed parts of the SfM Pipeline

9

SfM :: Results

Clemens Havas 27 November 2015

� Taking several different areas of a room and test the functionalities of OpenMVG

10

Computation :: Test OpenMVG

Clemens Havas 27 November 2015 11

Computation :: Global SfM Pipeline

http://www.theia-sfm.org/sfm.html#global-sfm-pipeline

Clemens Havas 27 November 2015 12

Normal High

Computation :: Settings

Clemens Havas 27 November 2015 13

Ultra

Computation :: Settings

Clemens Havas 27 November 2015

� Finding the best parameters for the resolution and the number of images

� Point cloud generation with a “windowing” method with overlap to reduce duration of computation

14

Computation :: Best Parameters

Clemens Havas 27 November 2015 15

Computation :: Time vs Points

Clemens Havas 27 November 2015 16

Computation :: Different Resolutions

Clemens Havas 27 November 2015

� One combined app (SfM, Visualisation)

� Point cloud merging and reduction

� Fingerprinting algorithm

¾ Indoor Positioning

17

Conclusion :: Future Work

Clemens Havas 27 November 2015

� Best point cloud reduction algorithm?

� Fingerprinting algorithm for indoor navigation?

� SURF better solution for SfM?

� Increase algorithm performance?

18

Conclusion :: Open Questions