multi-sensor as-built models of complex industrial ......remote sens. 2015, 7, 16339–16362 remote...

Article

Multi-Sensor As-Built Models of ComplexIndustrial ArchitecturesJean-François Hullo *, Guillaume Thibault, Christian Boucheny, Fabien Dory and Arnaud Mas

Received: 30 June 2015; Accepted: 23 November 2015; Published: 4 December 2015Academic Editors: Diego Gonzalez-Aguilera, Norman Kerle and Prasad S. Thenkabail

Electricité De France Lab, 1 Avenue du Général de Gaulle, 92140 Clamart, France;[email protected] (G.T.); [email protected] (C.B.); [email protected] (F.D.);[email protected] (A.M.)* Correspondence: [email protected] or [email protected]; Tel.: +33-147-653-731

Abstract: In the context of increased maintenance operations and generational renewal work,a nuclear owner and operator, like Electricité de France (EDF), is invested in the scaling-up of toolsand methods of “as-built virtual reality” for whole buildings and large audiences. In this paper, wefirst present the state of the art of scanning tools and methods used to represent a very complexarchitecture. Then, we propose a methodology and assess it in a large experiment carried out onthe most complex building of a 1300-megawatt power plant, an 11-floor reactor building. We alsopresent several developments that made possible the acquisition, processing and georeferencing ofmultiple data sources (1000+ 3D laser scans and RGB panoramic, total-station surveying, 2D floorplans and the 3D reconstruction of CAD as-built models). In addition, we introduce new conceptsfor user interaction with complex architecture, elaborated during the development of an applicationthat allows a painless exploration of the whole dataset by professionals, unfamiliar with such datatypes. Finally, we discuss the main feedback items from this large experiment, the remaining issuesfor the generalization of such large-scale surveys and the future technical and scientific challengesin the field of industrial “virtual reality”.

Keywords: 3D as-built reconstruction; multi-sensor dataset production; terrestrial laser scanner;panoramic images; 2D and 3D CAD models; virtual mock-up; complex scene visualization

1. Introduction

1.1. Industrial Context

In order to fulfill the need for as-built datasets to help workers in complex buildings intheir daily jobs, large and multi-sensor surveys now have to be considered at the whole buildingsize. Unfortunately, many current tools (including sensors, processing programs and visualizationapplications) have not been designed for such large surveys of complex indoor facilities.

Until today, the major uses with as-built data in the industry are related to the description of theactual shape of only a part of the facility, with its obstacles and free spaces with centimeter accuracy,to help maintenance planning, handling, storage, replacement or changing important components inthat specific part of the plant; see Figure 1. The next step in the field of 3D surveying of facilitiesconsists both of scaling up the current state of the art, without compromising data quality, and indedicating as-built datasets to new users, who are not experts in CAD or terrestrial laser scanner(TLS) data.

Remote Sens. 2015, 7, 16339–16362; doi:10.3390/rs71215827 www.mdpi.com/journal/remotesensing

Remote Sens. 2015, 7, 16339–16362

Remote Sens. 2015, 7 2

1. Introduction

1.1. Industrial Context

In order to fulfill the need for as-built datasets to help workers in complex buildings in their daily

jobs, large and multi-sensor surveys now have to be considered at the whole building size. Unfortunately,

many current tools (including sensors, processing programs and visualization applications) have not

been designed for such large surveys of complex indoor facilities.

Until today, the major uses with as-built data in the industry are related to the description of the actual

shape of only a part of the facility, with its obstacles and free spaces with centimeter accuracy, to help

maintenance planning, handling, storage, replacement or changing important components in that specific

part of the plant; see Figure 1. The next step in the field of 3D surveying of facilities consists both of

scaling up the current state of the art, without compromising data quality, and in dedicating as-built

datasets to new users, who are not experts in CAD or terrestrial laser scanner (TLS) data.

Figure 1. Review of three breakthrough projects (1993–2014) of as-built reconstruction

from laser scanning data of industrial facilities at Electricité de France (EDF) [1]: (a) first

water tank (1993); (b) first turbine hall (2009); (c) first reactor building (2014).

1.2. Contributions

The contributions of this paper, which is an enhanced and detailed version of [1], are the following:

Figure 1. Review of three breakthrough projects (1993–2014) of as-built reconstruction from laserscanning data of industrial facilities at Electricité de France (EDF) [1]: (a) first water tank (1993);(b) first turbine hall (2009); (c) first reactor building (2014).

1.2. Contributions

The contributions of this paper, which is an enhanced and detailed version of [1], arethe following:

‚ The state of the art of tools and methods for the acquisition, processing and georeferencing ofas-built datasets dedicated to the specific conditions of complex indoor facilities, Subsection 2.1and Subsection 2.2;

‚ The proposition of a global method for multi-sensor acquisition and processing to representcomplex architecture, Subsection 2.3 and Figure 2;

‚ Settings, processes and feedback from a large-scale multi-sensor scanning survey experimenton a whole 1300-megawatt nuclear reactor building (1000+ stations of both TLS and panoramicimages) with a highlight of the role of the human beings in the process, Section 3;

‚ A new interactive tool for pose estimation of panoramic images, Section 4;‚ Recommendations and examples for developing dedicated applications for virtual tours of

complex architectures using multiple data types in order to increase the value of the datasetand answer users’ requirements, Section 5;

‚ An overview of the remaining bottlenecks and challenges in view of the generalization of large,dense, multi-sensor scanning surveys, Section 6.

16340

Remote Sens. 2015, 7, 16339–16362


• The state of the art of tools and methods for the acquisition, processing and georeferencing of

as-built datasets dedicated to the specific conditions of complex indoor facilities, Subsections 2.1

and 2.2;

• The proposition of a global method for multi-sensor acquisition and processing to represent

complex architecture, Subsection 2.3 and Figure 2;

• Settings, processes and feedback from a large-scale multi-sensor scanning survey experiment on

a whole 1300-megawatt nuclear reactor building (1000+ stations of both TLS and panoramic

images) with a highlight of the role of the human beings in the process, Section 3;

• A new interactive tool for pose estimation of panoramic images, Section 4;

• Recommendations and examples for developing dedicated applications for virtual tours of

complex architectures using multiple data types in order to increase the value of the dataset and

answer users’ requirements, Section 5;

• An overview of the remaining bottlenecks and challenges in view of the generalization of large,

dense, multi-sensor scanning surveys, Section 6.

Figure 2. Global framework of as-built data production in the experiment. White boxes are

the initial data sources; grey boxes are intermediate data versions; and black boxes are the

datasets that composed the virtual replica of a complex building. Orange outlines represent

stronger control procedures in the data production process.

2. State of the Art in 3D Surveying for the Industry

2.1. Scanning Industrial Interiors, a 20-Year-Old Challenge

As shown in Figure 3, power facilities are a specific subset of industrial environments, in their shapes

(congested, with a huge number of objects, whose manufactured geometries correspond to a set of 3D

Figure 2. Global framework of as-built data production in the experiment. White boxes are theinitial data sources; grey boxes are intermediate data versions; and black boxes are the datasetsthat composed the virtual replica of a complex building. Orange outlines represent stronger controlprocedures in the data production process.

2. State of the Art in 3D Surveying for the Industry

2.1. Scanning Industrial Interiors, a 20-Year-Old Challenge

As shown in Figure 3, power facilities are a specific subset of industrial environments, in theirshapes (congested, with a huge number of objects, whose manufactured geometries correspond toa set of 3D primitives, like cylinders, boxes, etc.), their surfaces (objects sometimes very reflectiveor with very dark albedos) and accessibility (on-site restricted access time, difficult temperature andvibration conditions). The geometric complexity of these scenes led to the development of dedicatedacquisition and processing tools, such as terrestrial laser scanning in the 1990s [2], used very oftensince then, see Figure 1a. These methods have been used until today to help maintenance workers,by providing a description of the geometry of key areas of the facilities.

Unfortunately, and because of the specific nature of industrial environments, many developmentsintended for the 3D laser scanning of non-industrial objects have not solved all of the issues arising inthe large-scale dense scanning survey of whole industrial scenes with multiple floor heights. Thesemain issues are the speed of surveying (since the plant is only accessible a couple of weeks in theyear), the accuracy of the raw points for a large spectrum of albedos (with both mirror-like and darksurfaces in the same scene), the accuracy of registration and referencing (the 2.58σ global geometrictolerance is ˘2 cm) and, also, the productivity of 3D reconstruction (up to 100.000 objects for onesingle building).

16341

Remote Sens. 2015, 7, 16339–16362


primitives, like cylinders, boxes, etc.), their surfaces (objects sometimes very reflective or with very dark

albedos) and accessibility (on-site restricted access time, difficult temperature and vibration conditions).

The geometric complexity of these scenes led to the development of dedicated acquisition and processing

tools, such as terrestrial laser scanning in the 1990s [2], used very often since then, see Figure 1a. These

methods have been used until today to help maintenance workers, by providing a description of the

geometry of key areas of the facilities.

Figure 3. Specific issues of a power facility: (a) rooms are often very congested, (b) objects

can be very dark and (c) stainless objects can be clean and then very reflective, (d) often

coexisting in one place.

Unfortunately, and because of the specific nature of industrial environments, many developments

intended for the 3D laser scanning of non-industrial objects have not solved all of the issues arising in

the large-scale dense scanning survey of whole industrial scenes with multiple floor heights. These main

issues are the speed of surveying (since the plant is only accessible a couple of weeks in the year), the

accuracy of the raw points for a large spectrum of albedos (with both mirror-like and dark surfaces in

the same scene), the accuracy of registration and referencing (the 2.58 global geometric tolerance is

±2 cm) and, also, the productivity of 3D reconstruction (up to 100.000 objects for one single building).

In an industrial context, many users are not experienced users of laser scanner or CAD data.

Furthermore, geometry alone is not sufficient to meet all maintenance needs and to represent all of the

complexity of the architecture and the details of a power plant. These observations lead to the following

requirements for the tools and methods used in these new standards in the production and use of as-built

datasets for the maintenance of facilities:

(1) Multi-sensor datasets that describe both the geometry and appearance of a facility;

(2) Fast and accurate sensors, responsive to harsh albedos or very reflective surfaces;

(3) Automated pipelines for conversions and formatting;

(4) Large-scale efficient tools for registration and processing;

(5) User-friendly diagnostic tools to achieve high quality requirements on large and complex

datasets;

(6) Dedicated solutions for the intuitive exploration and use of multi-sensor datasets.

Figure 3. Specific issues of a power facility: (a) rooms are often very congested; (b) objects can be verydark and (c) stainless objects can be clean and then very reflective; (d) often coexisting in one place.

In an industrial context, many users are not experienced users of laser scanner or CAD data.Furthermore, geometry alone is not sufficient to meet all maintenance needs and to represent all ofthe complexity of the architecture and the details of a power plant. These observations lead to thefollowing requirements for the tools and methods used in these new standards in the production anduse of as-built datasets for the maintenance of facilities:

(1) Multi-sensor datasets that describe both the geometry and appearance of a facility;(2) Fast and accurate sensors, responsive to harsh albedos or very reflective surfaces;(3) Automated pipelines for conversions and formatting;(4) Large-scale efficient tools for registration and processing;(5) User-friendly diagnostic tools to achieve high quality requirements on large and complex datasets;(6) Dedicated solutions for the intuitive exploration and use of multi-sensor datasets.

2.2. Requirement Engineering for As-Built Datasets of Complex Architectures

In order to represent a complex facility, the requirement engineering of several jobs, throughmeetings and discussions, to estimate the optimal value of the dataset produced, has led to theconclusion that the following data types must be produced together; see Figure 4:

(1) Panoramic images, in order to be able to read each equipment tag that might be visible up to5 m from the main pathways in the facility, as well as to understand the overall architecture ofthe building;

(2) Georeferenced terrestrial laser scans, in order to deliver local sub-centimeter geometric accuracyon distance measurements, as well as global 3D locations with less than 2 cm of deviation fromthe ground truth (2.58σ tolerance);

(3) 2D floor maps, with all elements relative to the structure of the building (walls, floors, ladders,stairs, etc.), as well as landmarks to help pedestrians navigate the plant;

(4) 3D, as-built CAD models with categories, to capture empty areas or open walking spaces withless than 5 cm of error (2.58σ tolerance), as well as the shape of the main equipment of the facility.

16342

Remote Sens. 2015, 7, 16339–16362


2.2. Requirement Engineering for As-Built Datasets of Complex Architectures

In order to represent a complex facility, the requirement engineering of several jobs, through meetings

and discussions, to estimate the optimal value of the dataset produced, has led to the conclusion that the

following data types must be produced together; see Figure 4:

(1) Panoramic images, in order to be able to read each equipment tag that might be visible up to 5 m

from the main pathways in the facility, as well as to understand the overall architecture of

the building;

(2) Georeferenced terrestrial laser scans, in order to deliver local sub-centimeter geometric accuracy

on distance measurements, as well as global 3D locations with less than 2 cm of deviation from

the ground truth (2.58 tolerance);

(3) 2D floor maps, with all elements relative to the structure of the building (walls, floors, ladders,

stairs, etc.), as well as landmarks to help pedestrians navigate the plant;

(4) 3D, as-built CAD models with categories, to capture empty areas or open walking spaces with

less than 5 cm of error (2.58 tolerance), as well as the shape of the main equipment of

the facility.

Figure 4. Four types of data are required to represent a power plant facility with a high

degree of confidence (geometry and appearance): panoramic images, laser scans, floor maps

and 3D CAD model.

In order to match these requirements, one must define relevant specification criteria for each type of

data, as presented in Table 1.

Then, the tools and methods that might meet all of these criteria were selected using the experience

of survey teams that should submit sample datasets to prove their expertise (from either previous surveys

or small-scale experiments in similar conditions). The initial estimates of costs and delays were

extrapolated to the building scale thanks to a deep analysis of these previous acquisitions.

Finally, to ensure the consistency of results for initial user needs, the dataset was fully checked, using

a Level 2 control procedure and then reworked until quality standards were achieved. Finally, user

feedback showed the validity of such approaches.

Figure 4. Four types of data are required to represent a power plant facility with a high degree ofconfidence (geometry and appearance): panoramic images, laser scans, floor maps and 3D CAD model.

In order to match these requirements, one must define relevant specification criteria for eachtype of data, as presented in Table 1.

Then, the tools and methods that might meet all of these criteria were selected using theexperience of survey teams that should submit sample datasets to prove their expertise (from eitherprevious surveys or small-scale experiments in similar conditions). The initial estimates of costs anddelays were extrapolated to the building scale thanks to a deep analysis of these previous acquisitions.

Finally, to ensure the consistency of results for initial user needs, the dataset was fully checked,using a Level 2 control procedure and then reworked until quality standards were achieved. Finally,user feedback showed the validity of such approaches.

Table 1. Criteria of a multi-sensor survey of a building with complex architecture.

Type of Data Criteria

Panoramic images completeness, location of stations, field of view, resolution and noiselevel of digital images, white balance and high dynamic range(HDR), naming, approximate poses, vertical estimation and mapping

Georeferenced and cleanedlaser scans

completeness, location of stations, block structure of sub-networks,targets for registration, reference targets for georeferencing, field ofview, resolution, signal and total station traverse survey

2D floor maps completeness, geometry, naming and formatting3D as-built CAD model completeness, categories of objects, type of shapes, geometric

accuracies by category, names and structure

2.3. State of the Art of Large-Scale Dense Scanning Surveys

Recent breakthroughs offer the prospect of large-scale and multi-sensor scanning surveys thatcomply with the previously defined technical requirements and tolerances, from acquisition andprocessing to data integration in dedicated software, developed specifically for maintenance workin a power plant. As will be shown in this section, new developments in data acquisition in the pastfive years have mainly improved the speed rate, with almost no gain in accuracy. The real benefitfor scaling up dense scanning surveys consists of the improvement in data storage, visualizationand processing.

16343

Remote Sens. 2015, 7, 16339–16362

2.3.1. Panorama Acquisition

Regarding panoramic image acquisition, current 360˝ video cameras allow fast acquisition ofequirectangular images; the state-of-the-art highest resolution panoramic systems use 44 sensorsto produce videos with a resolution of 9000 ˆ 2400 pixels at a rate of 30 frames per second [3].However, static tripod-mounted motorized heads with full frame digital single lens cameras(DLSR) can generate higher resolutions (gigapixels) panoramas, for example half a billion pixelsin five minutes [4], with a high automation rate in stitching. This resolution allows reading labelsup to 5 m, and the high dynamic range (HDR) rendering can deal with high contrast and poorlightning conditions.

2.3.2. 3D Point Cloud Acquisition and Registration

Regarding 3D georeferenced point cloud acquisition, some noticeable improvements have comefrom robotic and computer vision research programs on fast 3D scanning indoors, such as rangesensors [5], backpack scanning systems [6], handheld [7] or mounted on mobile platforms invarious shapes and sizes [8,9]. However, these systems, either for local precision [5–7] or globalgeoreferencing errors [9], deal mostly with decimeter accuracy. Moreover, many indoor localizationand mapping systems require planar objects and/or straight corridors to reduce drift using elaboratealgorithms, such as “iterate closest point + inertial measurement unit + planar” proposed in [6].For labyrinthine and crowded indoors, a fusion of several techniques may be available in a fewyears, including indoor localization systems based on inertial sensors (the state of the art may befound in the EVAAL indoor positioning competition—Evaluating Ambient Assisted Living SystemsThrough Competitive Benchmarking—won in 2015 by [10]), graph-SLAM (simultaneous localizationand mapping) sensor positioning [11], efficient loop closure [12] and robust detection of features forgeo-referencing [13].

Currently, phase-based terrestrial laser scanners are better sensors for small distances, andacquisition rate and can for instance capture 50 million points of mid-range measurements inapproximately five minutes (<1 m–50 m). Regarding local accuracy, errors in depth measurementsare under 1 cm in that range on most object surfaces [14]. However, the surfaces of some industrialobjects do not correspond to the calibration standard ranges (very short distances, low albedos,high reflectance properties and low incidence angles), leading to significant consumption of theerror budget [15]. A proportion of these errors can be filtered out automatically [16] (comet-tailedeffect, outliers), although another proportion, often quite considerable, can only be removed throughmanual segmentation (multi-reflections on specular surfaces, in particular).

Regarding global accuracy, the fine registration of large laser scanner datasets is an open andactive research topic. For complex buildings, cloud-to-cloud automatic approaches, such as ICPand variants, cannot be used for several reasons. First, due to the proximity of objects, a smallrelative change in scanner position induces strong differences on point clouds. Thus, cloud-to-cloudconstraints cannot, by definition, lead to accurate and trustworthy results in complex scenes.Secondly, these approaches cannot take into account datum constraints for data referencing withouta great loss of automation. Finally, these approaches are neither efficient nor robust for very largedatasets (many hundreds of stations) [17]. Currently, only target-based registration paired with atotal station survey for datum referencing allows the fine registration of hundreds of laser scans(3σ = ˘2 cm) on the scale of 10-floor buildings.

2.3.3. 3D CAD Reconstruction

Recent improvements in processing tools for 3D reconstructions from large point clouds (tens ofbillions) have made possible the “as-built” reconstruction of a full mock-up. First, the data storagecost has increased by a factor of 25 in the last decade [18], while the data USB external transfer rate hasincreased by a factor of 20 [19], and internal SSD now reaches 500 Mb/s. However, if dozens of laser

16344

Remote Sens. 2015, 7, 16339–16362

scans requires gigabytes, thousands of laser scans require terabytes. To improve both file storage,versioning and processing, data structures have been developed [20] and implemented in manycommercial software. Other great improvements have been made for visualizing and manipulating abillion point clouds, for example using QuadTrees [21], such as implemented in the WebGL rendererPoTree [22], with real-time shaders, like Eye Dome Lightning [23], also implemented in the opensource software CloudCompare.

Looking back, 3D CAD models have been used for planning maintenance operations of industrialinstallations since the 1990s [2]. Depending on the requirements, several formats can be useddepending on the requirements of the industry. In the architectural, engineering and construction(AEC) industry, building information models (BIM) have recently attained widespread attention.These BIM models can be really valuable when they are used in the whole lifecycle of a building [24],from construction monitoring [25] to decommissioning [26], but are used for new rather than forexisting buildings [27]. In the power plant industry, PDMS from Aveva has been a leading plantdesign model for more than 30 years [28]. For power plants designed before the 1980s, only 2D planswere created.

As detailed in [29], there is an obvious need for automated or semi-automated methodsfor the production of as-built BIMs; the current process for creating parametric BIM from apoint cloud is largely a manual procedure, which is time consuming and lacks quality controls.Indeed, primitive-based 3D CAD models can still be reconstructed from point clouds with a betterproductivity than BIM [30], and most objects of a power plant can be considered as a combination ofprimitive geometries (due to a series effect of the design and part manufacturing process, except formolding large metal equipment). Though recent automatic algorithms for primitive fitting performbetter with increasing sampling resolution, they are still far from a 2.58σ tolerance of detection andfitting [31]. To achieve centimeter accuracy on more than 99% of the reconstructed objects, humaninteraction is required for either initial segmentation or picking initial points for region growing.To assess this accuracy, several tools must be used: visual inspection and a cloud to shape distancecomputation. Finally, the primitive-based CAD models can also be displayed in the usual renderingengines of virtual reality [32].

3. Experiment of a Multi-Sensor Survey in a 1300-MW Nuclear Reactor

3.1. Goals and Context of the Experiment

In the wake of several research projects that contributed to the development of tools and methodsfor dense scanning [33], reconstruction [34] or registration [17] and also tools for the visualization ofcomplex datasets [23], EDF carried out the first very large-scale experiment in the most complexbuilding in a 1300-MW nuclear power plant facility: a reactor building.

The goal was to assess the effective performance (quality, cost and speed) of current tools andbest practices through a major research project launched in 2013, in order to make a decision on thegeneralization of the multi-sensor scanning of power plants and highlight remaining bottlenecks totarget lock ups. Another goal was to maximize the benefits of this dataset to the company, especiallyby sharing it with as many users as possible, to assist them in their daily work.

Therefore, to meet the needs of many maintenance procedures and operations, a densemulti-sensor survey (total stations, laser scanning and panoramic RGB) was carried out during thesummer of 2013, using the technical specifications detailed above; Subsection 2.2; see Figure 5.

3.2. A Level 2 Procedure for the Quality Control of Large Datasets

We mentioned several definitions of tolerances and quality requirements to reach high-qualitydata production. In order to reach these requirements, it is necessary to use specific proceduresfor the quality control of large, as-built datasets (laser scans, RGB panoramic or 3D CAD models).These Level 2 procedures are a standard for quality monitoring and for detecting non-conforming

16345

Remote Sens. 2015, 7, 16339–16362

materials. Our experiment demonstrated their usefulness. To enhance dataset quality, we thereforeimplemented it in the following manner:

‚ Level 1: the data creator checks 100% of the dataset and documents it;‚ Level 2: the project data manager performs spot checks both on areas of interest and randomly

on the whole dataset.

Due to the significant number of human actions involved in the processes and the size of thedataset, at least two iterations are generally needed to reach quality levels.


Due to the significant number of human actions involved in the processes and the size of the dataset,

at least two iterations are generally needed to reach quality levels.

(a) (b)

Figure 5. (a) Scale of the reactor in a power plant facility, a cylindrical building with 11

floors. (b) 1000+ TLS stations and targets for referencing appear as a very complex 3D

network: green spheres are reference points; red spheres are TLS stations; and blue lines are

the sights of registration targets from TLS stations.

3.3. Description of the Data Acquisition

The reactor building, whose shape is cylindrical, consists of eleven floors with additional floor heights

in each of them. Moreover, the majority of the 100+ rooms in the building is particularly congested, and

many of them are only accessible through ladders. Regarding the environmental conditions, exploring

and scanning such environments is constrained by poor lighting, access restrictions and congestion due

to the equipment and the civil works of the plant itself. During this specific experiment, due to the many

maintenance operations occurring in the shutdown period, the building was exceptionally accessible for

five weeks (35 days), but we expect the available survey time to be reduced by 50% in the future

(17 days). Key numbers for the multi-sensor scanning survey are shown in Figure 6. The global

framework for data production is shown in Figure 2.

Figure 5. (a) Scale of the reactor in a power plant facility, a cylindrical building with 11 floors;(b) 1000+ TLS stations and targets for referencing appear as a very complex 3D network: green spheresare reference points; red spheres are TLS stations; and blue lines are the sights of registration targetsfrom TLS stations.

3.3. Description of the Data Acquisition

The reactor building, whose shape is cylindrical, consists of eleven floors with additional floorheights in each of them. Moreover, the majority of the 100+ rooms in the building is particularlycongested, and many of them are only accessible through ladders. Regarding the environmentalconditions, exploring and scanning such environments is constrained by poor lighting, accessrestrictions and congestion due to the equipment and the civil works of the plant itself. During thisspecific experiment, due to the many maintenance operations occurring in the shutdown period, thebuilding was exceptionally accessible for five weeks (35 days), but we expect the available surveytime to be reduced by 50% in the future (17 days). Key numbers for the multi-sensor scanning surveyare shown in Figure 6. The global framework for data production is shown in Figure 2.

16346

Remote Sens. 2015, 7, 16339–16362Remote Sens. 2015, 7 10

Figure 6. Key numbers in the multi-sensor scanning survey of a nuclear reactor building [1];

the main order of magnitude that may be highlighted is the number of stations: more than

1000 stations (for more details, see [35]). The relative time allocation is shown on the graph;

CAD reconstruction is the main budget.

Discussion on on-site acquisition: As detailed in [35], this acquisition phase demonstrated the

feasibility of such surveys thanks to some specific evolutions and developments of the tools and methods

to take into account the up scaling. To succeed, a large number of sensors and operators (five laser-

scanning surveyors with Leica HDS 6100 and Leica HDS 7000 and three photographers with Canon

EOS 5D Mark III with motorized panoramic heads) is required. However, as mentioned above, we

expect to have only half the time to carry out the survey, i.e., in the future, twice the number of sensors

and operators will have to be sent to a site, creating additional stress on current methodologies and

synchronization between operators.

Discussion on processing TLS data: This experiment also underlined some constraints on scaling up

the current post-processing tools and methods in terms of quality, time or cost (fine registration of large

laser scanner datasets, quality monitoring and validation tools, issues for formats, storage and data

sharing). To illustrate the complexity of the fine registration of laser scans in complex architectures, we

show in Figure 5a view of the 3D network used in the adjustment. This experiment, mostly performed

with a Leica Cyclone, underlined the lack of robustness of weighted least squares in such large blocks

and the need to work with 3D topographic networks instead of 2.5D. Unfortunately, even state-of-the-

art registration tools implement a basic version of the least squares algorithm and lack tools for blunder

detection, error propagation and quality monitoring.

The main issues regarding the processing of TLS data are the following:

• Several tests on TLS sensors have shown that, even today, the scanning of dark surfaces is not

efficient, and the reflective surfaces lead to billions of erroneous 3D points that cannot be filtered

by current firmware or software. These points are certainly an issue for 3D reconstructions, but

mostly for taking measurements of so-called “bubble views” or station-based views where no

feedback on the real position of 3D points is given. Our solution consisted of developing an

efficient interface to perform a manual segmentation of these outliers Figure 7.

• Without better algorithms for the fine referencing of TLS dataset and large 3D networks, the only

way to comply with a global 2.58 tolerance of 2 cm accuracy overall, the dataset consists of

creating sub-networks with fewer than 200 stations, independently referenced to the external

reference system. This constraint implies thorough planning on site for placing and surveying

targets. Recent developments have shown that we can expect better results in automation and

Figure 6. Key numbers in the multi-sensor scanning survey of a nuclear reactor building [1]; the mainorder of magnitude that may be highlighted is the number of stations: more than 1000 stations (formore details, see [35]). The relative time allocation is shown on the graph; CAD reconstruction is themain budget.

Discussion on on-site acquisition: As detailed in [35], this acquisition phase demonstrated thefeasibility of such surveys thanks to some specific evolutions and developments of the tools andmethods to take into account the up scaling. To succeed, a large number of sensors and operators(five laser-scanning surveyors with Leica HDS 6100 and Leica HDS 7000 and three photographerswith Canon EOS 5D Mark III with motorized panoramic heads) is required. However, as mentionedabove, we expect to have only half the time to carry out the survey, i.e., in the future, twice thenumber of sensors and operators will have to be sent to a site, creating additional stress on currentmethodologies and synchronization between operators.

Discussion on processing TLS data: This experiment also underlined some constraints on scalingup the current post-processing tools and methods in terms of quality, time or cost (fine registrationof large laser scanner datasets, quality monitoring and validation tools, issues for formats, storageand data sharing). To illustrate the complexity of the fine registration of laser scans in complexarchitectures, we show in Figure 5a view of the 3D network used in the adjustment. This experiment,mostly performed with a Leica Cyclone, underlined the lack of robustness of weighted least squares insuch large blocks and the need to work with 3D topographic networks instead of 2.5D. Unfortunately,even state-of-the-art registration tools implement a basic version of the least squares algorithm andlack tools for blunder detection, error propagation and quality monitoring.

The main issues regarding the processing of TLS data are the following:

‚ Several tests on TLS sensors have shown that, even today, the scanning of dark surfaces is notefficient, and the reflective surfaces lead to billions of erroneous 3D points that cannot be filteredby current firmware or software. These points are certainly an issue for 3D reconstructions, butmostly for taking measurements of so-called “bubble views” or station-based views where nofeedback on the real position of 3D points is given. Our solution consisted of developing anefficient interface to perform a manual segmentation of these outliers Figure 7.

‚ Without better algorithms for the fine referencing of TLS dataset and large 3D networks, theonly way to comply with a global 2.58σ tolerance of 2 cm accuracy overall, the dataset consistsof creating sub-networks with fewer than 200 stations, independently referenced to the externalreference system. This constraint implies thorough planning on site for placing and surveyingtargets. Recent developments have shown that we can expect better results in automation andquality by bundler snooping (moving targets) and the use of robust estimation well known fromgeodesists [36].

16347

Remote Sens. 2015, 7, 16339–16362


quality by bundler snooping (moving targets) and the use of robust estimation well known from

geodesists [36].

Figure 7. An example of blunder removal in a 3D scan. Blue: 3D points filtered

automatically. Red: 3D points manually segmented (the average segmentation time is 6.5

minutes per station).

Discussion on processing panoramic images: The level of automation and the quality of output using

modern stitching software, such as the one we used, Kolor AutoPanoPro, are very good. However, some

issues still have to be addressed:

• When dealing with 450 Mpixel images of indoor scenes, the field of view of a single picture is

quite small and can lead to a lack of feature points when only uniform objects, such as a painted

wall, are visible in the picture. The Level 2 control procedure has led to the reopening of 20% of

the panoramic stitches for editing;

• Another recurrent error in panoramic images consists of estimating verticality, based on the

images. In congested environments, default settings can lead to errors of up to 20° in that

estimate. We recommend either the use of leveled panoramic heads with custom settings for

estimating the relative position of unit images or the use of vertical definition tools through the

software interface;

• A major issue is the registration of the panoramic images on the external reference frame; see

Figure 8. As will be discussed later on, the fine registration of panoramic images is valuable in

assisting with the quality control of 3D reconstructions and offers a better user experience when

browsing the dataset. We will address this issue later in this paper.

Discussion on restructuring and updating 2D floor maps: A global model of the plant is key data for

designing the structure of the dataset, unifying the names of the objects and performing an analysis on

it. By constructing a graph of the various objects and their relationships (adjacency, verticality, inclusion,

etc.), it makes it possible to answer questions like “What are the panoramic images taken in the rooms

adjacent to a specific one?” or “Can I access a specific location in the building without taking ladders?”

This entire graph database was built using structured 2D floor plans, updated using the 3D dataset:

walls, grounds, ladders, stairs and many others; cf. Figure 9a. These as-built floor plans were then

automatically processed to extract all of the required information and to build a “topological graph”,

Figure 7. An example of blunder removal in a 3D scan. Blue: 3D points filtered automatically. Red: 3Dpoints manually segmented (the average segmentation time is 6.5 minutes per station).

Discussion on processing panoramic images: The level of automation and the quality of outputusing modern stitching software, such as the one we used, Kolor AutoPanoPro, are very good.However, some issues still have to be addressed:

‚ When dealing with 450 Mpixel images of indoor scenes, the field of view of a single picture isquite small and can lead to a lack of feature points when only uniform objects, such as a paintedwall, are visible in the picture. The Level 2 control procedure has led to the reopening of 20% ofthe panoramic stitches for editing;

‚ Another recurrent error in panoramic images consists of estimating verticality, based on theimages. In congested environments, default settings can lead to errors of up to 20˝ in thatestimate. We recommend either the use of leveled panoramic heads with custom settings forestimating the relative position of unit images or the use of vertical definition tools through thesoftware interface;

‚ A major issue is the registration of the panoramic images on the external reference frame; seeFigure 8. As will be discussed later on, the fine registration of panoramic images is valuable inassisting with the quality control of 3D reconstructions and offers a better user experience whenbrowsing the dataset. We will address this issue later in this paper.

Discussion on restructuring and updating 2D floor maps: A global model of the plant is keydata for designing the structure of the dataset, unifying the names of the objects and performingan analysis on it. By constructing a graph of the various objects and their relationships (adjacency,verticality, inclusion, etc.), it makes it possible to answer questions like “What are the panoramicimages taken in the rooms adjacent to a specific one?” or “Can I access a specific location in thebuilding without taking ladders?”

This entire graph database was built using structured 2D floor plans, updated using the 3Ddataset: walls, grounds, ladders, stairs and many others; cf. Figure 9a. These as-built floor plans werethen automatically processed to extract all of the required information and to build a “topologicalgraph”, describing several properties of the installation (shape, location, names, types, navigation,access, etc.), as well as rendering specific maps using style sheets. A previous example of this kind ofapproach can be seen in [37]. This graph is then used in the software applications that we developed,in order to answer queries on the semantic, geometric or topological properties of the building andits components; cf. Figure 9b.

16348

Remote Sens. 2015, 7, 16339–16362


describing several properties of the installation (shape, location, names, types, navigation, access, etc.),

as well as rendering specific maps using style sheets. A previous example of this kind of approach can

be seen in [37]. This graph is then used in the software applications that we developed, in order to answer

queries on the semantic, geometric or topological properties of the building and its components; cf.

Figure 9b.

Figure 8. Typical issues in panoramic images. A lack of texture can lead to glitches in the

final image (Left). The estimated vertical may have to be refined manually because horizon

estimation algorithms can fail up to 15° (Right).

Regarding the processing steps for the floor maps, the pipeline is as follows:

(1) Redraw existing floor maps in AutoCAD, following specific drawing rules and using only two

types of objects: polylines and blocks (manual);

(2) Compare the 3D as-built model and panoramic images to floor plans in a specific tool developed

in Unity3D, to update them in AutoCAD (manual);

(3) Convert polygons of floor maps to Scalable Vector Graphics (SVG) files (automatic) and export

the blocks to XLS using EXTATTBE in AutoCAD (automatic);

(4) In a dedicated C# tool, instantiate a class model of objects of the building:

a. For each floor:

i. import SVG and XLS to instantiate a relational model of the building (floors,

ladders, walls, etc.), including controls to check with respect to the drawing rules,

ii. using a Clipper Library clipping algorithm [38] and threshold, compute relations

between objects, such as adjacency, inclusion, etc.

b. For each pair of floors:

i. Connect objects, such as elevators, ladders, stairs,

ii. Using Clipper Library, compute vertical relationships between objects.

(5) Export the instantiated building model to a topological graph in XML, with a description of the

building model in XML Schema Definition (XSD).

Figure 8. Typical issues in panoramic images. A lack of texture can lead to glitches in the final image(Left). The estimated vertical may have to be refined manually because horizon estimation algorithmscan fail up to 15˝ (Right).

Regarding the processing steps for the floor maps, the pipeline is as follows:

(1) Redraw existing floor maps in AutoCAD, following specific drawing rules and using only twotypes of objects: polylines and blocks (manual);

(2) Compare the 3D as-built model and panoramic images to floor plans in a specific tool developedin Unity3D, to update them in AutoCAD (manual);

(3) Convert polygons of floor maps to Scalable Vector Graphics (SVG) files (automatic) and exportthe blocks to XLS using EXTATTBE in AutoCAD (automatic);

(4) In a dedicated C# tool, instantiate a class model of objects of the building:

a For each floor:

i import SVG and XLS to instantiate a relational model of the building (floors,ladders, walls, etc.), including controls to check with respect to the drawing rules,

ii using a Clipper Library clipping algorithm [38] and threshold, compute relationsbetween objects, such as adjacency, inclusion, etc.

b For each pair of floors:

i Connect objects, such as elevators, ladders, stairs,ii Using Clipper Library, compute vertical relationships between objects.

(5) Export the instantiated building model to a topological graph in XML, with a description of thebuilding model in XML Schema Definition (XSD).

Discussion on the 3D CAD reconstruction based on the TLS point cloud: A large dataset of40 billion 3D points was used to reconstruct an as-built 3D CAD model of the facility (see Figure 10).The main part of the 3D reconstruction was produced using Trimble RealWorks Version 8.x (80%of the final CAD model); some objects from a pre-existing CAD model were adjusted in DassaultSystems SolidWorks, as well as other equipment that could not be modeled as a combination ofprimitive shapes (see Subsection 2.3.3). As illustrated in Figure 6, the 3D reconstruction was themain line in the budget of the data production. In order to meet end users’ needs, the quality levelsof the 3D data produced had to be defined in accordance with their intended uses: maintenancetask planning (including the associated logistics: access, scaffolds, handling areas), worker safety andvirtual tour for low accessibility rooms by inexperienced professionals.

16349

Remote Sens. 2015, 7, 16339–16362Remote Sens. 2015, 7 13

(a) (b)

Figure 9. 2D as-built floor maps (a) updated using the 3D as-built dataset, laser scans and

panoramic images. (b) An interactive Virtual Reality Modeling Language (VRML)

visualization of a semantic, geometric and topological representation of the building, based

on the as-built floor maps [1].

Discussion on the 3D CAD reconstruction based on the TLS point cloud: A large dataset of 40 billion

3D points was used to reconstruct an as-built 3D CAD model of the facility (see Figure 10). The main

part of the 3D reconstruction was produced using Trimble RealWorks Version 8.x (80% of the final

CAD model); some objects from a pre-existing CAD model were adjusted in Dassault Systems

SolidWorks, as well as other equipment that could not be modeled as a combination of primitive shapes

(see Subsection 2.3.3). As illustrated in Figure 6, the 3D reconstruction was the main line in the budget

of the data production. In order to meet end users’ needs, the quality levels of the 3D data produced had

to be defined in accordance with their intended uses: maintenance task planning (including the associated

logistics: access, scaffolds, handling areas), worker safety and virtual tour for low accessibility rooms

by inexperienced professionals.

Figure 10. The dense data sources used for as-built reconstruction of the nuclear reactor

building from point clouds: (a) 1025 panoramic images with 450 million of pixels, (b) 1085

laser scans with 40 million 3D points, (c) 3D reconstruction of 25 types of objects with

specific rules (fitting tolerances and naming encodings) and (d) as-built CAD model with

100 rooms and 100,000 3D objects.

Figure 9. 2D as-built floor maps (a) updated using the 3D as-built dataset, laser scans and panoramicimages; (b) An interactive Virtual Reality Modeling Language (VRML) visualization of a semantic,geometric and topological representation of the building, based on the as-built floor maps [1].


(a) (b)

Figure 9. 2D as-built floor maps (a) updated using the 3D as-built dataset, laser scans and

panoramic images. (b) An interactive Virtual Reality Modeling Language (VRML)

visualization of a semantic, geometric and topological representation of the building, based

on the as-built floor maps [1].

Discussion on the 3D CAD reconstruction based on the TLS point cloud: A large dataset of 40 billion

3D points was used to reconstruct an as-built 3D CAD model of the facility (see Figure 10). The main

part of the 3D reconstruction was produced using Trimble RealWorks Version 8.x (80% of the final

CAD model); some objects from a pre-existing CAD model were adjusted in Dassault Systems

SolidWorks, as well as other equipment that could not be modeled as a combination of primitive shapes

(see Subsection 2.3.3). As illustrated in Figure 6, the 3D reconstruction was the main line in the budget

of the data production. In order to meet end users’ needs, the quality levels of the 3D data produced had

to be defined in accordance with their intended uses: maintenance task planning (including the associated

logistics: access, scaffolds, handling areas), worker safety and virtual tour for low accessibility rooms

by inexperienced professionals.

Figure 10. The dense data sources used for as-built reconstruction of the nuclear reactor

building from point clouds: (a) 1025 panoramic images with 450 million of pixels, (b) 1085

laser scans with 40 million 3D points, (c) 3D reconstruction of 25 types of objects with

specific rules (fitting tolerances and naming encodings) and (d) as-built CAD model with

100 rooms and 100,000 3D objects.

Figure 10. The dense data sources used for as-built reconstruction of the nuclear reactor buildingfrom point clouds: (a) 1025 panoramic images with 450 million of pixels; (b) 1085 laser scans with40 million 3D points; (c) 3D reconstruction of 25 types of objects with specific rules (fitting tolerancesand naming encodings) and (d) as-built CAD model with 100 rooms and 100,000 3D objects.

The consequences of this multiplicity of needs led to a detailed specification for 3D reconstructiontasks depending on the object type among the 25 categories used: fitting tolerances, naming andmethods (cloud least-square fitting, region growing cloud-shape fitting, cloud snapping and copies).The use of precisely georeferenced panoramic RGB images was very valuable in the reconstructionprocess to help with understanding complex areas.

In order to deal with the huge amount of work and to reduce the time of data delivery, the 3DCAD reconstruction had to be split and parallelized by sectors and then allocated to half a dozenCAD operators for almost a year (10 months). To achieve and certify the quality of the reconstructedas-built model, a Level 2 check procedure was carried out by two independent operators and ledto further iterations and re-working (approximately 10% of the total effort). This validation wasperformed using mesh-cloud distance computations, by using software originally developed byEDF R&D and Telecom ParisTech (CloudCompare [33]) and “out-of-core” technologies to display amaximum number of points for the visual inspection in Trimble RealWorks 8.x, guided by the analysisof an SQL database storing the standard deviation of each reconstructed geometry (~80,000 objects).

16350

Remote Sens. 2015, 7, 16339–16362

3.4. Summary of Tool Development during the Project

Many tools (hardware and software) were used to produce both the data and end-userapplications. Some of them were satisfactory, but many had to be improved during the project.

Acquisition and stitching of RGB panoramic images: Canon EOS 5D cameras with a Dr Claussmotorized head were used. For stitching, mapping and blending, Kolor AutoPano was used, andXNview MP was very useful for batch conversions, resampling and renaming. None of the toolswere improved.

Acquisition and processing of the TLS dataset: Leica HDS 6000 and 7000 with total stationswere used. The Z+F software was used to convert and filter scans. Specific settings were added tothat software for improve the results. For registering scans, Leica Cyclone was used. For manualsegmentation, we had to add new features in Trimble RealWorks, used to remove noise that couldnot be filtered out (mirrors).

Floor plan editing and processing: Floor plans were edited in AutoCAD, and a specific tool hadto be implemented in Unity3D to compare them to the as-built 3D CAD model. All of the processing,converting and analysis steps required to create the “topological graph” were made in a custom tooldeveloped in C#, using Clipper Library, based on a Vatti clipping algorithm [38].

3D as-built CAD reconstruction and conversions: For main walls and civils works, DassaultSystems SolidWorks was used. For all of the other objects, Trimble Realworks was used and had tobe improved to increase productivity (shortcuts, color palette, debugging fitting tools, etc.), as wellas quality, by developing a dedicated SQL plug-in to store metadata for each 3D object (“was theobject fitted or manually adjusted?”, “is the object a copy?”, etc.). For two-level control procedures,Hexagon 3DReshaper and CloudCompare were used. CloudCompare was specially enhanced for thisproject. In addition to the edition and conversion features offered by SAP Visual Enterprise Authors,we developed advanced scripts using Windows PowerShell.

4. Example of Specific Development to Reach Quality Expectations for Referencing PanoramicImages Precisely

During acquisition, panoramic images were roughly located on a map and oriented to the north.However, many usage examples have underlined the need for estimated correct pose in these images(dataset navigation, 3D reconstruction quality check through overlay, etc.). We propose severalsolutions, using either the TLS dataset, the 2D floor maps or the 3D CAD reconstruction model.

4.1. Camera Model for Panoramic Images

Using an equirectangular projection where the z axis of the reference frame is aligned with themeridians of the panorama (see Figure 11), the spherical coordinates of a pixel can be given using thefollowing relations:

θ “W ´ 2u

Wπ P r´π, πr φ “

H ´ 2v2H

π P”

´π

2,

π

2r (1)

If we represent these pixels on the surface of the sphere, the point mi = (xi, yi, zi) can be calculatedfrom its coordinates (θi, φi) using the following relations:

mi “ pcos pφq .cos pθq , cos pφq .sin pθq , sin pφqq (2)

The epipolar constraints between two images I1 and I2, with a relative pose composed of atranslation three-vector t and a 3 ˆ 3 rotation matrix R, are given for a corresponding pair of pointsfrom the two images:

mT2 Em1 “ 0 where E “ rtsˆ R is the essential matrix (3)

16351

Remote Sens. 2015, 7, 16339–16362

[t]ˆ is the anti-symmetric matrix induced by the vector product. The locus of epipolar constrainttransforms from lines (pinhole cameras) to circles in the case of spherical panoramas.


4. Example of Specific Development to Reach Quality Expectations for Referencing Panoramic

Images Precisely

During acquisition, panoramic images were roughly located on a map and oriented to the north.

However, many usage examples have underlined the need for estimated correct pose in these images

(dataset navigation, 3D reconstruction quality check through overlay, etc.). We propose several

solutions, using either the TLS dataset, the 2D floor maps or the 3D CAD reconstruction model.

4.1. Camera Model for Panoramic Images

Using an equirectangular projection where the z axis of the reference frame is aligned with the

meridians of the panorama (see Figure 11), the spherical coordinates of a pixel can be given using the

following relations: = − 2 ∈ [− , [ = − 22 ∈ [− 2 , 2 [ (1)

If we represent these pixels on the surface of the sphere, the point = ( , , ) can be calculated

from its coordinates ( , ) using the following relations: = ( ). ( ), ( ). ( ), ( ) (2)

The epipolar constraints between two images and , with a relative pose composed of a translation

three-vector and a 3 × 3 rotation matrix , are given for a corresponding pair of points from the two

images: = 0where = [ × is the essential matrix (3) [ × is the anti-symmetric matrix induced by the vector product. The locus of epipolar constraint

transforms from lines (pinhole cameras) to circles in the case of spherical panoramas.

Figure 11. Spherical image and coordinate axes.

4.2. Pose Estimate Using Constraints between Image Pixels and 3D Points from Laser Scans

For the external pose estimate, i.e., the absolute pose of the panoramic image in the global reference

frame, we can use matching feature points between pixels in the image and real world points (for

Figure 11. Spherical image and coordinate axes.

4.2. Pose Estimate Using Constraints between Image Pixels and 3D Points from Laser Scans

For the external pose estimate, i.e., the absolute pose of the panoramic image in the globalreference frame, we can use matching feature points between pixels in the image and real worldpoints (for example, from a 3D laser scan of a close station). Thus, the problem is the following,given points mi, the coordinates of the corresponding pixels on the sphere, and Mi, the homogeneouscoordinates of 3D:

aimi “ RMi ` t “ pR|tq

˜

Mi1

¸

“ PMiphomogen.q where ai is a scale factor for each point i (4)

This can be turned to a linear system problem Ah = 0 as follows, where pj is the j-th row of:

Ah “

¨

˚

˚

˚

˚

˚

˚

˚

˚

˚

˝

MT1 0 0 ´x1 0 0 0 ¨ ¨ ¨

0 MT1 0 ´y1 0 0 0 ¨ ¨ ¨

0 0 MT1 ´z1 0 0 0 ¨ ¨ ¨

MT2 0 0 0 ´x1 0 0 ¨ ¨ ¨

0 MT2 0 0 ´y1 0 0 ¨ ¨ ¨

0 0 MT0 0 0 ´x3 0 ¨ ¨ ¨

¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨ ¨

˛

‹

‹

‹

‹

‹

‹

‹

‹

‹

‚

¨

¨

˚

˚

˚

˚

˚

˚

˚

˝

pT1

pT2

pT3

a11

a22

¨ ¨ ¨

˛

‹

‹

‹

‹

‹

‹

‹

‚

“ 0 (5)

We solve this linear system by choosing h as the last column of V where A + UDVT is the singularvalue decomposition of A. Taking the 12 entries of h, we obtain Pest (estimated P), which differs fromP by a scale factor k, that can be solved through:

k “||Pest Mi||

a2i(6)

we can choose k as the average value computed from n points.In our case, panoramas are “vertical”, which simplifies the extraction of the only non-zero

rotation angle κ (using Tait–Bryan angles) and t values from P. Thus, to solve pose estimation, weneed a few reliable and well-distributed matching pairs. This can be done manually or automaticallyusing a feature point extraction algorithm and more robust estimating schemes. However, achievingstable results in matching and pose estimates for the whole dataset is tedious, and manual editing

16352

Remote Sens. 2015, 7, 16339–16362

requires a dedicated interface. In the next two sections, we propose interfaces for using either 2Dmaps or 3D as-built CAD models instead of feature point-based methods.

4.3. Partial Pose Estimate of Panoramic Images Using 2D Floor Maps

A user-oriented approach was implemented to get a first “independent” estimate of parameters(tx, ty) and (κ) of a vertical panoramic image by using 2D floor maps. Using a user interface with asynchronization of the panoramic view and its position with the camera orientation on the 2D floormap, the procedure followed by the operator is:

(1) Move (tx, ty) panoramic using near and identifiable details of the floor maps by estimating theratio of distances (doors, holes in floors, etc.);

(2) Orient (κ) using far landmarks of the floor maps;(3) Check at +90˝ and +180˝ and iterate the first two steps until the best estimate.

This method was applied to the entire dataset (1025 panoramic images), and 95% of theimages were moved or oriented (970); see Figure 12. The average processing and control time forone panoramic image is 1 min 50 s.Remote Sens. 2015, 7 17

Figure 12. User interface of the partial pose estimation tool. The camera position, orientation

and field of view of the panoramic image are synchronized with the floor map (Top). A

central cross helps the sighting of landmarks. Key numbers of the corrections using 2D maps

(1025 images) (Bottom).

4.4. Full Pose Estimate of Panoramic Images Using the 3D As-Built Dataset

Another user-oriented approach was developed to refine the whole pose (including ) of the panoramic images ( , , ) and ( ) given the hypothesis of verticality ( = 0, = 0). Using a user

interface with a synchronization of the panoramic view and its position with the camera orientation in

the 3D model and using switches for overlay and transparency, the procedure followed by the operator

consists of independently correcting the four parameters of the pose; see Figure 13:

(1) Move ( ) panoramic using landmarks of the 3D model that cross the equator of the panoramic

in the image (boxes, stairs, guardrail, etc.); front view;

(2) Align ( ) panoramic using parallel objects of the 3D model to any meridian of the panoramic in

the image (pipes, beams, etc.); zenith view;

Figure 12. User interface of the partial pose estimation tool. The camera position, orientation and fieldof view of the panoramic image are synchronized with the floor map (Top). A central cross helps thesighting of landmarks; Key numbers of the corrections using 2D maps (1025 images) (Bottom).

16353

Remote Sens. 2015, 7, 16339–16362

4.4. Full Pose Estimate of Panoramic Images Using the 3D As-Built Dataset

Another user-oriented approach was developed to refine the whole pose (including tz) of thepanoramic images (tx, ty, tz) and (κ) given the hypothesis of verticality (ω = 0, φ = 0). Using a userinterface with a synchronization of the panoramic view and its position with the camera orientationin the 3D model and using switches for overlay and transparency, the procedure followed by theoperator consists of independently correcting the four parameters of the pose; see Figure 13:

(1) Move (tz) panoramic using landmarks of the 3D model that cross the equator of the panoramicin the image (boxes, stairs, guardrail, etc.); front view;

(2) Align (κ) panoramic using parallel objects of the 3D model to any meridian of the panoramic inthe image (pipes, beams, etc.); zenith view;

(3) Move (tx, ty) panoramic using landmarks on the ceiling of the 3D model to align them to thezenith of the image; zenith view;

(4) Check horizontally for any issue at +90˝ and +180˝ and iterate the first three steps until the bestestimate; an incorrect estimate of the vertical when mapping the panorama is the main sourceof a poor estimate of the pose of the panoramic, and such a panoramic should be corrected.

This method was applied to the entire dataset after the first estimate on 2D floor maps(1025 panoramic images). During this process, we moved and oriented 100% of the images (1025).The average processing and control time for one panoramic image is 30 s for low corrections and1 min 30 s–2 min for more difficult images.

4.5. Overall Feedback on the Experiment and Discussion on Future Large Scanning Surveys

Even for a company with over 20 years’ experience in the field of as-built documentation,a multi-sensor and dense scanning survey on the scale of an entire 11-floor building remains achallenge, for which tools and methods have not yet been designed.

In addition to the several recommendations and choices that were set out in the previousparagraphs, our experiment highlighted some specific aspects of large and multi-sensor scanningsurveys in complex architectures. For these projects that respond to the growing need for as-builtdata for professionals, we recommend:

‚ specifying needs, requirements and constraints in detail. On the scale of a building, everymisunderstanding or fuzzy specification may have a severe impact on costs, quality or durations;

‚ documenting every step in the process and performing quality monitoring from the beginning,to help both fixing non-conforming data and enriching the dataset for future use;

‚ parallelizing the tasks as much as possible (acquisition and post-processing); when the dataproduction time increases, the number of non-qualities increases significantly.

Regarding remaining bottlenecks, we have mentioned several technical brakes on thegeneralization of current tools and methods for large and complex buildings, regarding raw dataacquisition and processing. We have also discussed the three areas of 3D reconstruction: costs,duration and quality. Since the real requirement is the third one, the first two areas must be viewedas secondary in achieving the dataset in industrial processes. The low level of automation and thelow quality of acquisition sensors and tools are the actual brakes on generalization. However, byperforming rigorous quality monitoring and control, each dataset can be produced with a high degreeof confidence. In such large projects, traceability is one of the keys to quality management. TheFigure 2 gives a global overview of the whole framework of as-built data production; we can see thatmany steps are required, including many tools and human interactions. As shown, also, many controlprocedures are required to reach and certify a quality level. Indeed, the main drawback for achievingexpected quality levels is the difficult inter-validation of data sources. To perform such a qualitycontrol policy, the dataset has to be merged, compared and visualized altogether. Nevertheless, due

16354

Remote Sens. 2015, 7, 16339–16362

to the limitations of current tools and formats, this fusion and exhaustive checking of multi-sourcedatasets remains very tedious.Remote Sens. 2015, 7 19

Figure 13. User interface and procedure for fully estimating the pose of panoramic images

in reference to a 3D model (Top). Equator, meridians and central dots help with the

alignment and correction of parameters. Zoom on specific details (before/results) (Middle).

Key numbers in the corrections using the 3D model (1025 images) (Bottom).

Figure 13. User interface and procedure for fully estimating the pose of panoramic images in referenceto a 3D model (Top); Equator, meridians and central dots help with the alignment and correction ofparameters. Zoom on specific details (before/results) (Middle); Key numbers in the corrections usingthe 3D model (1025 images) (Bottom).

5. Developing New Software Applications to Increase the Value of the Dataset

In the previous sections, we have detailed the challenges in producing data to represent acomplex building. Due to that complexity and the intended uses, multiple data types are required:high resolution panoramic RGB, laser scans, 3D as-built CAD models and 2D floor plans. Once the

16355

Remote Sens. 2015, 7, 16339–16362

dataset is complete, issues remain for visualization and exploration, taking into account that usersmay not be familiar with these various data types.

5.1. State of the Art of Multi-Data Visualization for Complex Architectures

By its nature, each data source represents only one aspect of the reality of the plant and onlymeets some needs among many. The co-visualization of the multiple data sources is thereforerequired to assist and help workers with finding answers in their daily jobs. In recent years,several solutions have been proposed for the problem of multi-source, as-built data visualization;see Figure 14:

‚ virtual tours with floor plans and panoramic RGB [39];‚ navigation through several spherical laser views;‚ navigation and path calculation in 3D environments, for instance in 3D video games or 2.5D

cartography services, such as Google Maps indoors.


5. Developing New Software Applications to Increase the Value of the Dataset

In the previous sections, we have detailed the challenges in producing data to represent a complex

building. Due to that complexity and the intended uses, multiple data types are required: high resolution

panoramic RGB, laser scans, 3D as-built CAD models and 2D floor plans. Once the dataset is complete,

issues remain for visualization and exploration, taking into account that users may not be familiar with

these various data types.

5.1. State of the Art of Multi-Data Visualization for Complex Architectures

By its nature, each data source represents only one aspect of the reality of the plant and only meets

some needs among many. The co-visualization of the multiple data sources is therefore required to assist

and help workers with finding answers in their daily jobs. In recent years, several solutions have been

proposed for the problem of multi-source, as-built data visualization; see Figure 14:

• virtual tours with floor plans and panoramic RGB [39];

• navigation through several spherical laser views;

• navigation and path calculation in 3D environments, for instance in 3D video games or 2.5D

cartography services, such as Google Maps indoors.

Figure 14. Examples of user interfaces that can be found today for virtual tours: Leica

TrueView, CSA VirtualTours, Kolor PanoTour, Faro WebShare.

However, none of these solutions integrates all of the data sources required to represent some complex

buildings, such as nuclear reactor buildings, and easily navigate the large dataset. Indeed, apart from the

constraints of technical integration, the complexity of the plant itself is an issue for virtual navigation

(multiple levels with dozens of rooms and vertical junctions) and requires a specific interaction design

to handle it.

5.2. Rules for the Development of Virtual Tours of a Complex Building

Our first goal is to develop an application that can be of value to many people working in nuclear

plants, taking into account that the targeted building is rarely accessible. Potential values include several

scenarios. The first scenario consists of improving the productivity of maintenance operations through a

virtual preparation stage that takes into account the spatial constraints of the environment; this requires

the collaboration of different teams based on a shared representation of the environment (e.g.,

mechanical workers defining their scaffolding needs). The second scenario aims to improve accuracy

and to reduce delays in engineering studies to prepare for modifications and revamping to the plant, with

reduced on-site time for the teams. The third and last scenario consists of helping many recently-hired

Figure 14. Examples of user interfaces that can be found today for virtual tours: Leica TrueView, CSAVirtualTours, Kolor PanoTour, Faro WebShare.

However, none of these solutions integrates all of the data sources required to represent somecomplex buildings, such as nuclear reactor buildings, and easily navigate the large dataset. Indeed,apart from the constraints of technical integration, the complexity of the plant itself is an issue forvirtual navigation (multiple levels with dozens of rooms and vertical junctions) and requires a specificinteraction design to handle it.

5.2. Rules for the Development of Virtual Tours of a Complex Building

Our first goal is to develop an application that can be of value to many people working innuclear plants, taking into account that the targeted building is rarely accessible. Potential valuesinclude several scenarios. The first scenario consists of improving the productivity of maintenanceoperations through a virtual preparation stage that takes into account the spatial constraints of theenvironment; this requires the collaboration of different teams based on a shared representation of theenvironment (e.g., mechanical workers defining their scaffolding needs). The second scenario aimsto improve accuracy and to reduce delays in engineering studies to prepare for modifications andrevamping to the plant, with reduced on-site time for the teams. The third and last scenario consistsof helping many recently-hired workers to become familiar with their working environment morerapidly, through dedicated training sessions, including tutoring courses.

These few scenarios illustrate the variety of user profiles that should be considered for thecommon view, which we wish to design. Many of them are not familiar with the handling ofas-built datasets, and their use of the tool we developed may be very occasional, which reinforcesthe need for user-friendly and simple interfaces, as well as taking into account human perception inthe design [40,41].

To address this problem, several principles have been selected for the development of navigationinteraction within our applications, developed using the game engine Unity 3D. The first principleconsists of the multi-view synchronization across a multi-source dataset, with the possibility at anytime of moving from one view to another without losing user position/orientation (shared, unique

16356

Remote Sens. 2015, 7, 16339–16362

position of the user in the environment), as shown in Figure 15. The second principle consists of theability to explore the building in its horizontal and vertical dimensions (through stairs and ladders,as well as trapdoors and removable slatted floors), which is required by tasks, such as handlingheavy equipment. The last principle consists of developing all tools and functionalities in a projectinterface for a better user data management: marks, snapshots, distance measurements with guidanceinstruction and path computations.


workers to become familiar with their working environment more rapidly, through dedicated training

sessions, including tutoring courses.

These few scenarios illustrate the variety of user profiles that should be considered for the common

view, which we wish to design. Many of them are not familiar with the handling of as-built datasets, and

their use of the tool we developed may be very occasional, which reinforces the need for user-friendly

and simple interfaces, as well as taking into account human perception in the design [40,41].

To address this problem, several principles have been selected for the development of navigation

interaction within our applications, developed using the game engine Unity3D. The first principle

consists of the multi-view synchronization across a multi-source dataset, with the possibility at any time

of moving from one view to another without losing user position/orientation (shared, unique position of

the user in the environment), as shown in Figure 15. The second principle consists of the ability to

explore the building in its horizontal and vertical dimensions (through stairs and ladders, as well as

trapdoors and removable slatted floors), which is required by tasks, such as handling heavy equipment.

The last principle consists of developing all tools and functionalities in a project interface for a better

user data management: marks, snapshots, distance measurements with guidance instruction and path

computations.

Figure 15. Multi-source data exploration and navigation in a complex building: visualization

and station-based transitions of (a) panoramic images and (b) laser scans; (c) 2D map view

of all stations and synchronized mini-maps and (d) first-person pedestrian navigation in the

3D model, including climbing ladders, taking stairs and crouching.

For instance, we estimate that in the synchronized, multi-view experience, spherical photographs need

to be positioned in the virtual environment with a 2.58 tolerance of 50 cm and 5°. This accuracy range

seems sufficient to help users to focus on their task, by not being disturbed by some inaccuracies in the

transitions when switching between views. Furthermore, the verticality of the panoramic images must

be known with an error below 3°; otherwise, navigation becomes very uncomfortable.

In addition to the navigation features, we provide users with specific interactions with the dataset.

Each feature has been designed according to the users’ needs. To implement them, we carried out several

Figure 15. Multi-source data exploration and navigation in a complex building: visualization andstation-based transitions of (a) panoramic images and (b) laser scans; (c) 2D map view of all stationsand synchronized mini-maps and (d) first-person pedestrian navigation in the 3D model, includingclimbing ladders, taking stairs and crouching.

For instance, we estimate that in the synchronized, multi-view experience, sphericalphotographs need to be positioned in the virtual environment with a 2.58σ tolerance of 50 cm and5˝. This accuracy range seems sufficient to help users to focus on their task, by not being disturbedby some inaccuracies in the transitions when switching between views. Furthermore, the verticalityof the panoramic images must be known with an error below 3˝; otherwise, navigation becomesvery uncomfortable.

In addition to the navigation features, we provide users with specific interactions with thedataset. Each feature has been designed according to the users’ needs. To implement them, wecarried out several iterations with users to overcome technical feasibility issues and create genuinelypowerful and smart tools. Among these, we can mention distance measurements on 3D CAD models(perpendicular to the normal unit vectors, vertically constrained or free), annotations and snapshotsin every data view, interactive cutaways of walls and user data management and sharing. In all ofthese developments, a large amount of feedback has been collected in a quick response developmentprocess to achieve optimum user benefit.

16357

Remote Sens. 2015, 7, 16339–16362


iterations with users to overcome technical feasibility issues and create genuinely powerful and smart

tools. Among these, we can mention distance measurements on 3D CAD models (perpendicular to the

normal unit vectors, vertically constrained or free), annotations and snapshots in every data view,

interactive cutaways of walls and user data management and sharing. In all of these developments, a

large amount of feedback has been collected in a quick response development process to achieve

optimum user benefit.

Figure 16. A procedure for improving the quality of distance measurement in laser scans.

The user must (a) choose between measurement types and (b) control his/her own

measurement by a double picking, then (c) check the result in “bubble view” and finally (d)

in the 3D model.

5.3. A Framework for Taking Measurements on Laser Scans for Non-Expert Users

So-called “bubble views” of laser scans are often used to measure distances. However, for non-

familiar users, two main issues should be avoided. Firstly, errors in range measurement (reflective

surfaces, for instance) impact the distance value; hence, many distance measurements should be checked

more than once and interpreted with caution, although most users ignore the typical error budget of a

laser scan. A manual segmentation process is highly recommended to remove as many noisy points as

possible, to comply with the definition of tolerances. Secondly, wrong picks on edges can lead to wrong

distance measurements, whereas in spherical view, no feedback can be returned. To decrease the rate of

wrong measurements, we have developed a specific procedure, as detailed below.

To reduce the occurrence of false measurements in a laser scan, even after removing outliers, we

propose the following procedure (see Figure 16):

Figure 16. A procedure for improving the quality of distance measurement in laser scans. The usermust (a) choose between measurement types and (b) control his/her own measurement by a doublepicking; then (c) check the result in “bubble view” and finally (d) in the 3D model.

5.3. A Framework for Taking Measurements on Laser Scans for Non-Expert Users

So-called “bubble views” of laser scans are often used to measure distances. However, fornon-familiar users, two main issues should be avoided. Firstly, errors in range measurement(reflective surfaces, for instance) impact the distance value; hence, many distance measurementsshould be checked more than once and interpreted with caution, although most users ignore thetypical error budget of a laser scan. A manual segmentation process is highly recommended toremove as many noisy points as possible, to comply with the definition of tolerances. Secondly, wrongpicks on edges can lead to wrong distance measurements, whereas in spherical view, no feedback canbe returned. To decrease the rate of wrong measurements, we have developed a specific procedure,as detailed below.

To reduce the occurrence of false measurements in a laser scan, even after removing outliers, wepropose the following procedure (see Figure 16):

(1) choose a type of measurement from “almost horizontal”, “almost vertical” and “oblique”(2) pick two points in the laser scan to define the distance to be measured.

Ô If the picked segment is “consistent” with the type of measurement, then:

(3) pick the two points in the laser scan again.

Ô If the measured distance is “similar” to the previous one:

(4) the measurement is displayed on the laser scan with centimeter accuracy; it can be checkedagainst the reconstructed 3D model.

16358

Remote Sens. 2015, 7, 16339–16362

5.4. An Example of Advanced Technological Features Using a Georeferenced Dataset

The quality of the entire dataset allows the development of advanced features, such as pathoverlay on panoramic images. Paths are computed on navmeshes, using Autodesk GameWare, andcan be overlaid in real-time on panoramic images; see Figure 17.


(1) choose a type of measurement from “almost horizontal”, “almost vertical” and “oblique”

(2) pick two points in the laser scan to define the distance to be measured.

If the picked segment is “consistent” with the type of measurement, then:

(3) pick the two points in the laser scan again.

If the measured distance is “similar” to the previous one:

(4) the measurement is displayed on the laser scan with centimeter accuracy; it can be checked

against the reconstructed 3D model.

5.4. An Example of Advanced Technological Features Using a Georeferenced Dataset

The quality of the entire dataset allows the development of advanced features, such as path overlay

on panoramic images. Paths are computed on navmeshes, using Autodesk GameWare, and can be

overlaid in real-time on panoramic images; see Figure 17.

Figure 17. 3D pedestrian path (using navmeshes) overlay on panoramic images.

6. Conclusions and Future Work

The professional use of as-built models has recently increased significantly, resulting in two major

challenges: scaling up dense and multi-sensor scanning surveys to a whole building and sharing this

dataset with a very large audience.

In this paper, we reviewed the state of the art of scanning tools and methods for industrial installations.

Then, we proposed a global methodology for acquisition and processing of multi-sensor datasets to

represent complex architectures. To face the lack of automation to reach high quality in modelling

multi-floor plants, we detailed the optimal contribution of human beings in the process. We then assessed

this methodology in a large-scale experiment of modelling the 11 floors of a 1300-megawatt nuclear

reactor building. From the acquisition of 1000+ co-stations (RGB panoramic and laser scans) to a 3D

Figure 17. 3D pedestrian path (using navmeshes) overlay on panoramic images.

6. Conclusions and Future Work

The professional use of as-built models has recently increased significantly, resulting in twomajor challenges: scaling up dense and multi-sensor scanning surveys to a whole building andsharing this dataset with a very large audience.

In this paper, we reviewed the state of the art of scanning tools and methods for industrialinstallations. Then, we proposed a global methodology for acquisition and processing of multi-sensordatasets to represent complex architectures. To face the lack of automation to reach high quality inmodelling multi-floor plants, we detailed the optimal contribution of human beings in the process.We then assessed this methodology in a large-scale experiment of modelling the 11 floors of a1300-megawatt nuclear reactor building. From the acquisition of 1000+ co-stations (RGB panoramicand laser scans) to a 3D as-built reconstruction, the project has involved many contributors for almosta year and a half. At every step, quality requirements have put stress on tools and methods and ledto the implementation of Level 2 quality controls. We finally presented new tools that have beendeveloped to help many professionals in their daily jobs by allowing them to explore a completedigital plant easily using different types of data.

We can summarize the contribution made by our work in three points:

‚ We assessed the feasibility, as well as underlined the current complexity of tools and methodsused in a multi-sensor scanning of industrial environments (1000+ stations). Issues remain toreduce the effort in the production of such models.

‚ We have highlighted human beings’ contribution in the data production (from interactive toolsto quality management and control).

‚ We have shown how the value of such datasets can be increased by developing multi-sourcedata visualization and navigation applications in multi-floor plants with recommendations forgeneral principles applied to virtual tours of complex architectures.

16359

Remote Sens. 2015, 7, 16339–16362

Based on these findings and experiments, future work is already planned, with a view towardreducing the significance of the remaining issues:

‚ Improve data referencing procedures and algorithms to ensure confidence in quality levels,across all of the datasets.

‚ Develop new tools for data cross-validation and consistency across a multi-source dataset.

However, some serious bottlenecks seem farther removed from the current state of the art:

‚ How could we significantly increase (benefit >50% of current cost) the productivity of 3D as-builtCAD (or BIM) models from very large point clouds (hundreds of billions)?

‚ Which procedures and resources should be employed to update a large multi-sensor dataset anddetect errors or inconsistencies between two epochs?

Acknowledgments: The authors would like to thank the reviewers and copy editors, whose helpful commentsled to a better paper overall. The authors also want to thank all the involved teams in this project, with a specialmention to survey teams (photographers and surveyors), CAD operators and software developers.

Author Contributions: Jean-François Hullo and Guillaume Thibault conceived and designed the tools and themethods of data acquisition and processing. Christian Boucheny and Arnaud Mas conceived and designed theend-user application. Fabien Dory developed tools for 3D path computations and panoramic pose estimation.

Conflicts of Interest: The authors declare no conflict of interest.

References

1. Hullo, J.-F.; Thibault, G.; Boucheny, C. Advances in multi-sensor scanning and visualization of complexplants: The utmost case of a reactor building. ISPRS Arch. Photogramm. 2015, 1, 163–169. [CrossRef]

2. Pot, J.; Thibault, G.; Levesque, P. Techniques for CAD reconstruction of “as-built” environments andapplication to preparing for dismantling of plants. Nucl. Eng. Des. 1997, 178, 135–143. [CrossRef]

3. Depraz, F.; Popovic, V.; Ott, B.; Wellig, P.; Leblebici, Y. Real-time object detection and tracking inomni-directional surveillance using GPU. In Proceedings of the 2015 SPIE Target and BackgroundSignatures, Toulouse, France, 21 September 2015.

4. Kopf, J.; Uyttendaele, M.; Deussen, O.; Cohen, M. Capturing and viewing gigapixel images. ACM Trans. Gr.2007, 26, 93. [CrossRef]

5. Khoshelham, K.; Elberink, S.O. Accuracy and resolution of kinect depth data for indoor mappingapplications. Sensors 2012, 12, 1437–1454. [CrossRef] [PubMed]

6. Chen, G.; Kua, J.; Shum, S.; Naikal, N.; Carlberg, M.; Zakhor, A. Indoor localization algorithms for ahuman-operated backpack system. In Proceedings of the 5th International Symposium 3D Data Conferenceon Visualization, and Transmission, Paris, France, 17–20 May 2010.

7. Bosse, M.; Zlot, R.; Flick, P. Zebedee: Design of a spring-mounted 3-d range sensor with application tomobile mapping. IEEE Trans. Robot. 2012, 28, 1104–1119. [CrossRef]

8. Adán, A.; Quintana, B.; Vázquez, A.S.; Olivares, A.; Parra, E.; Prieto, S. Towards the automatic scanning ofindoors with robots. Sensors 2015, 15, 11551–11574. [CrossRef] [PubMed]

9. Zlot, R.; Bosse, M. Efficient large-scale three-dimensional mobile mapping for underground mines.J. Field Robot. 2014, 31, 758–779. [CrossRef]

10. Li, Y.; Zhang, P.; Lan, H.; Yuan, Z.; Niu, X.; El-Sheimy, N. Real-time indoor navigation using smartphones.In Proceedings of IPIN 2015 Conference on Evaal Indoor Navigation Competition, Banff, AB, Canada,13–16 October 2015.

11. De la Puente, P.; Rodriguez-Losada, D. Feature based graph-SLAM in structured environments.Auton. Robot. 2014, 37, 243–260. [CrossRef]

12. Labbe, M.; Michaud, F. Online global loop closure detection for large-scale multi-session graph-basedslam. In Proceedings of THE 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems,Chicago, IL, USA, 14–18 September 2014.

13. Ji, S.; Shi, Y.; Shan, J.; Shao, X.; Shi, Z.; Yuan, X.; Ynag, P.; Wu, W.; Tang, H.; Shibasaki, R. Particle filteringmethods for georeferencing panoramic image sequence in complex urban scenes. ISPRS J. Photogramm.2015, 105, 1–12. [CrossRef]

16360

http://dx.doi.org/10.5194/isprsarchives-XL-5-W4-163-2015

http://dx.doi.org/10.1016/S0029-5493(97)00211-2

http://dx.doi.org/10.1145/1276377.1276494

http://dx.doi.org/10.3390/s120201437

http://www.ncbi.nlm.nih.gov/pubmed/22438718

http://dx.doi.org/10.1109/TRO.2012.2200990

http://dx.doi.org/10.3390/s150511551


http://dx.doi.org/10.1002/rob.21504

http://dx.doi.org/10.1007/s10514-014-9386-z

http://dx.doi.org/10.1016/j.isprsjprs.2015.03.005

Remote Sens. 2015, 7, 16339–16362

14. Barras, V.; Delley, N.; Chapotte, G. Analyses aux limites des scanners laser terrestres. XYZ Rev. Assoc.Fr. Topogr. 2013, 137, 19–26.

15. Voegtle, T.; Wakaluk, S. Effects on the measurements of the terrestrial laser scanner HDS 6000 (Leica) causedby different object materials. In Proceedings of the 2009 ISPRS Work on Laser Scanning, Paris, France,1–2 September 2009.

16. Tang, P.; Huber, D.; Akinci, B. A comparative analysis of depth-discontinuity and mixed-pixel detectionalgorithms. In Proceedings of the IEEE Sixth International Conference on 3-D Digital Imaging andModeling, Montreal, QC, Canada, 21–23 August 2007.

17. Hullo, J.-F.; Thibault, G.; Grussenmeyer, P.; Landes, T.; Bennequin, D. Probabilistic feature matchingapplied to primitive based registration of TLS data. In Proceedings of the XXII ISPRS Congress onAnnals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Melbourne, Australia,25 August–1 September 2012.

18. Komorowski, M. A History of Storage Cost. Available online: http://www.mkomo.com/cost-per-gigabyte-update (accessed 30 October 2015).

19. Wikipedia. List of Device Bit Rates. Available online: https://en.wikipedia.org/wiki/List_of_device_bit_rates (accessed 4 October 2015).

20. Elseberg, J.; Borrmann, D.; Nüchter, A. One billion points in the cloud—An octree for efficient processingof 3D laser scans. ISPRS J. Photogramm. 2013, 76, 76–88. [CrossRef]

21. Scheiblauer, C. Interactions with Gigantic Point Clouds. Ph.D. Thesis, Institute of Computer Graphics andAlgorithms, Vienna, Austria, 2014.

22. Schütz, M. Potree, A Free Open-Source WebGL Based Point Cloud Renderer for Large Point Clouds.SCANOPY Project. Available online: https://github.com/potree/potree (accessed 30 October 2015).

23. Boucheny, C. Interactive Scientific Visualization of Large Datasets: Towards a Perceptive-Based Approach.Ph.D. Thesis, Université Joseph Fourier, Grenoble, France, 2009.

24. Azhar, S. Building information modeling (BIM): Trends, benefits, risks, and challenges for the AEC industry.Leadersh. Manag. Eng. 2011, 11, 241–252. [CrossRef]

25. Golparvar-Fard, M.; Peña-Mora, F.; Savarese, S. Automated progress monitoring using unordered dailyconstruction photographs and IFC-based building information models. J. Comput. Civil. Eng. 2012, 29,04014025. [CrossRef]

26. Jung, I.; Kim, W. Analysis of the possibility of required resources estimation for nuclear powerplant decommissioning applying BIM. In Proceedings of the KNS 2014 Spring Meeting, Jeju, Korea,28–30 May 2014.

27. Volk, R.; Stengel, J.; Schultmann, F. Building Information Modeling (BIM) for existing buildings—Literaturereview and future needs. Automat. Constr. 2014, 38, 109–127. [CrossRef]

28. Trickett, K.G.; Chaney, J.C. PDMS: Plant Layout and Piping Design; Gulf Publishing Company: Houston, TX,USA, 1982.

29. Tang, P.; Huber, D.; Akinci, B.; Lipman, R.; Lytle, A. Automatic reconstruction of as-built buildinginformation models from laser-scanned point clouds: A review of related techniques. Automat. Constr.2010, 19, 829–843. [CrossRef]

30. Son, H.; Kim, C.; Kim, C. 3D reconstruction of as-built industrial instrumentation models from laser-scandata and a 3D CAD database based on prior knowledge. Automat. Constr. 2015, 49, 193–200. [CrossRef]

31. Kang, Z.; Li, Z. Primitive fitting based on the efficient multiBaySAC algorithm. PLoS ONE 2015, 10,e0117341. [CrossRef] [PubMed]

32. Whyte, J.; Bouchlaghem, N.; Thorpe, A.; McCaffer, R. From CAD to virtual reality: Modelling approaches,data exchange and interactive 3D building design tools. Automat. Constr. 2000, 10, 43–55. [CrossRef]

33. Girardeau-Montaut, D.; Roux, M.; Marc, R.; Thibault, G. Change detection on points cloud data acquiredwith a ground laser scanner. ISPRS Arch. Photogramm. 2005, 36, 30–35.

34. Bey, A.; Chaine, R.; Marc, R.; Thibault, G.; Akkouche, S. Reconstruction of consistent 3D CAD modelsfrom point cloud data using a priori CAD models. In Proceedings of the 2011 ISPRS Workshop on LaserScanning, Calgary, AB, Canada, 29–31 August 2011.

35. Hullo, J.-F.; Thibault, G. Scaling up close-range surveys: A challenge for the generalization of as-built datain industrial applications. ISPRS Arch. Photogramm. 2014, 1, 293–299. [CrossRef]

16361

http://dx.doi.org/10.1016/j.isprsjprs.2012.10.004

http://dx.doi.org/10.1061/(ASCE)LM.1943-5630.0000127

http://dx.doi.org/10.1061/(ASCE)CP.1943-5487.0000205

http://dx.doi.org/10.1016/j.autcon.2013.10.023



http://dx.doi.org/10.1371/journal.pone.0117341


http://dx.doi.org/10.1016/S0926-5805(99)00012-6

http://dx.doi.org/10.5194/isprsarchives-XL-5-293-2014

Remote Sens. 2015, 7, 16339–16362

36. Wicki, F. Robust estimator for the adjustment of geodetic networks. In Proceedings of the First InternationalSymposium on Robust Statistics and Fuzzy Techniques in Geodesy and GIS, Zurich, Switzerland,12–16 March 2001.

37. Whiting, E. Geometric, Topological & Semantic Analysis of Multi-Building Floor Plan Data. Master’s Thesis,Massachusetts Institute of Technology, Deptment of Architecture, Cambridge, MA, USA, 2006.

38. Vatti, B.R. A generic solution to polygon clipping. Commun. ACM 1992, 35, 56–63. [CrossRef]39. Koehl, M.; Scheider, A.; Fritsch, E.; Fritsch, F.; Rachebi, A.; Guillemin, S. Documentation of historical

building via virtual tour: the complex building of baths in Strasbourg. In Proceedings of the XXIVInternational CIPA Symposium on Archives of the Photogrammetry, Remote Sensing and SpatialInformation Sciences, Strasbourg, France, 2–6 September 2013.

40. Thibault, G.; Pasqualotto, A.; Vidal, M.; Droulez, J.; Berthoz, A. How does horizontal and verticalnavigation influence spatial memory of multifloored environments? Atten. Percept. Psychophys. 2013,75, 10–15. [CrossRef] [PubMed]

41. Dollé, L.; Droulez, J.; Bennequin, D.; Berthoz, A.; Thibault, G. How the learning path and the very structureof a multifloored environment influence human spatial memory? Adv. Cogniti. Psychol. 2015, in press.

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an openaccess article distributed under the terms and conditions of the Creative Commons byAttribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

16362

http://dx.doi.org/10.1145/129902.129906

http://dx.doi.org/10.3758/s13414-012-0405-x


multi-sensor as-built models of complex industrial ......remote sens. 2015, 7, 16339–16362 remote...

Documents