INTRODUCTION

In the AH the traditional analysis, conservation, preservation, management, rehabilitation, exploitation and communication process is complex, driven from multidimensional data and approaches, fragmented, high-cost and still limited to major Monuments.These activities are then based on a continuous collaboration between architects, historians, engineers, researchers, managers and specialists, who work together for solutions to a complex process that includes the entire lifecycle: knowledge, use, communication and management. This implies the need for a platform to promote a real collaborative work between all parties involved. Finally, the process of conservation and restoration requires an increasing degree of automation.This situation leads to having only one database and one Information System for all of the different phases of the lifecycle and not for different and detached subsystems. Overall, the model deals with global knowledge about AH, which could be shared and made available at any time, in any place, to any user: researchers, professional operators, students, and city-users.Unfortunately these requirements and this methodological model are today simply a dream. Currently we see a total lack of accessibility to the entire corpus of information that should be shared by the specialists and the breakdown of the process into discontinuous isolated parts.The main reason of this deficit lies not only in the large amount of heterogeneous data (3D models, images, photos, drawings, written documents, etc.) required by the process, which prevents the immediate usability and an easy transfer of information, but also in the complexity and partiality of the systems developed to provide an answer to these problems.A first key step to overcome these lacks and deficiencies is to recognize, as a central moment of the entire building lifecycle, the conservation and maintenance stages, whose design plans are substantially the active parts of the process, in which shape, appearance, functionality and efficiency of the building are determined, being therefore the most important features of interventions.A further improvement towards a better AH process management is to exploit the BIM intrinsic capabilities well bounded by William J. Mitchell “Building Information Modeling (BIM) databases ... is opening up new ways to think about designing and producing buildings and - as we are beginning to see - new formal and functional possibilities.” (Mitchell, 2009).The use of BIM software in the AH lifecycle field, as reported by recent researches (Apollonio, Gaiani & Sun, 2012; Dore & Murphy., 2014; Oreni et al., 2014; Barazzetti et al., 2015), has many advantages such as semantic object-oriented modeling which allows for the classification of heritage objects, automatic lists of objects and material and automated conservation documents. However BIM techniques present in the AH field some limitations which prevented an effective use until today. It is easy to notice that one of the current limitations of BIM in the AH field is the lack of parametric library objects within BIM software that could be used for historical buildings or heritage sites. In addition, BIM as a tool of new design generally are not capable of modeling non-ideal state such as deviation, damage and deterioration, which are of prime concern when documenting AH and, more in general, the integration of low-level captured geometry with 3D parametric BIM objects is so hard. Also the activation of direct manufacturing processes for all objects BIM in the AH field is practically impossible. We think, however, that a more accurate analysis of failing factors is needed today in order to go further in the use of BIM in the AH and to embed the BIM in the AH lifecycle process. We propose an in-depth approach based on the following issues:

BIM methodology & AH

Problems related to the 3D capture techniques focusing on image-based modeling

Knowledge-based modeling in BIM platforms

Structuring acquired information for BIM processes.

BACKGROUND

BIM methodology & AH

BIM enables accurate object-oriented parametric modeling, and inherently incorporates semantic data pertaining to structural, material and additional information (Eastman, Eastman, Teicholz & Sacks, 2011). Apart from collection of geometry, BIM allows temporal representation showing various construction phases (Fai, Graham, Duckworth, Wood & Attar, 2011). Associative modeling among internal documents improves the efficiency of as-built documentation by real time model adaption and automatic clash detection. BIM can enhance information exchange for improved maintenance among stakeholders in life cycle (Motawa & Almarshad, 2013). All these characteristics make BIM an ideal platform for documenting and sharing the information of AH, but the as-built BIM is still in infancy (Hichri, Stefani, De Luca & Véron, 2013).The problem is first of all methodological. BIM processes are established for workflows as-planned instead of as-built (Eastman, Eastman, Teicholz & Sacks, 2011; Volk, Stengel & Schultmann, 2014). Although the developed data acquisition techniques have been linked to the as-built BIM workflow by commercial software, the gap between raw survey data and intelligent BIM objects is still to be bridged. The current limits of BIM in AH exist in the following aspects:Semantic object Vs unsegmented massThe semantic structure of BIM allows 3D models to be built, enriched and exchanged in object level (Eastman, Eastman, Teicholz & Sacks, 2011). The data acquisition techniques, however, generate huge amount of unsegmented data. Although various algorithms of object recognition have been addressed, little or none of them are applied to AEC industry especially to AH yet (Tang, Huber, Akinci, Lipman & Lytle, 2010; Hichri, Stefani, De Luca & Véron, 2013). It leads to a series of problems ranging from tedious and labor-intensive manual modeling to inaccurate documentation. Standardization Vs Irregularity BIM is a highly standard platform in the light of shape of component and the way they are organized. Therefore, it retains limited ability to represent the irregularity of AH caused by active factors (handcraft ornament and order variation from the same base) and passive factors (deviation, deformation, damage and weathering). Parametric intelligence Vs Geometric accuracyA typical BIM object has ambiguous geometry but explicit rules involving internal constraints and external adaption which represent the construction logic. As-built model of AH, however, usually contains millions of points, each represented by precise values without any relationships beyond geometry. The key point is how to transform the as-built model to parametric model without oversimplifying its geometry.

3D capture techniques focusing on image based modeling

A key step to establish an AH BIM is the shape and color data acquisition of the artifact. Basically, there are two approaches to the problem: active sensors (like terrestrial laser scanner (TLS) or structured light projectors) and image-based reconstruction. Active optical sensors (Blais, 2004; Vosselman & Maas, 2010) provide directly 3D range data and can capture relatively accurate geometric details, although still costly, usually bulky, not easy to use, requiring stable platform and affected by surface properties. They can acquire millions of points, even on perfectly flat surfaces, often resulting in over-sampling, but it is likely that corners and edges are not well captured. These sensors have also limited flexibility, since a range sensor is intended for a specific range/volume and generally lack of good texture information. Different technologies are used to overcome this problem: Time of Flight (ToF) used for longer range with accuracy in the single point measurement ~6 to 10 mm; Phase-based used for shorter ranges (~1 to 50 m) with an accuracy in the single point measurement ~0.5 to 5 mm; triangulation-based used for short range (~0.1 to 1 m) and high accuracy (~0.05 to 2 mm).The range-based modeling pipeline (Callieri, Dellepiane, Cignoni & Scopigno, 2011) is nowadays quite straightforward but there are some drawbacks: TLS are not part of the standard documentation procedure

in archaeology and serve only a very special purpose (Barber & Mills, 2007). Besides that, technically trained personnel are needed to operate the device and data acquisition can be tiresome and time consuming. Scanning above or below these ranges should be avoided so as to prevent inaccurate data capture. Some laser scanning equipment can have problems with reflectance from certain materials, such as marble or gilded surfaces. But, most importantly, they are still expensive to be used widely and problems generally arise in case of huge data sets and complex objects. Lastly, because these range sensors have been developed from an industry-oriented perspective, only a few are really useful for 3D AH applications (Blais & Beraldin, 2006). They do not provide unlimited geometric accuracy and completeness over objects and landscapes of all sizes at a low cost. Laser scanners are not as versatile as cameras with regard to capturing data, as they require time to scan the object, whereas a camera can capture a scene almost instantaneously. Moreover, they require line of sight to the object being recorded, meaning that it cannot see through objects (including dense vegetation), and it cannot see around corners. Scanning systems have minimum and maximum ranges over that they operate.Image-based methods (Remondino & El‐Hakim, 2006), circumvent these drawbacks, allowing surveys at different levels and in all possible combinations of object complexities, with high quality outputs, easy usage and manipulation of the final products, few time restrictions, good flexibility and low cost (Bryan, Blake & Bedford, 2013). 3D modeling from images provides sparse or dense point clouds, according to the employed measurement methodology (manual or automated), project requirements and aims. For simple structures (e.g. buildings) interactive approaches are satisfactory, but for complex and detailed surfaces need automated measurement approaches. Recent developments in automated and dense image matching (Furukawa & Ponce, 2010; Hirschmuller, 2008; Remondino, El-Hakim, Gruen & Zhang, 2008; Hiep, Keriven, Labatut & Pons, 2009), allows getting dense and well-calibrated point clouds semi-automatically from images.According to (Remondino, 2011), the choice of the 3D data capture approach depends on the required accuracy, object dimensions, location constraints, the instrument’s portability and usability, surface characteristics, the working team experience, the project budget and the final goal of the survey.To author an accurate and realistic 3D model the way previously mentioned, single capturing techniques are not able to give satisfactory results in all situations (i.e. high geometric accuracy, portability, automation, photo-realism, low costs, flexibility, efficiency). Image and range data could be combined to fully exploit the intrinsic potentialities of each approach (De Luca, Véron & Florenzano, 2006; Guarnieri, Remondino & Vettore, 2006; Stamos et al., 2008; Gašparovic & Malarić, 2012). However this is a complex solution not fitting the conservation needs but only the render of a very accurate representation of the current state of the building. Using a BIM 3D model as final output, the most part of the data will be lost during the geometric conversion, and the result rarely is satisfactory.Similarly to the mature and long-established 3D capture pipeline (Bernardini & Rushmeier, 2002), automated photogrammetric techniques, emerged in the last years from the collaboration between Computer Vision and Photogrammetric communities, are able to output shape and color, but at considerably lower costs, using a nearly standardized workflow: a) images acquisition, b) feature detection, c) feature matching, d) sparse 3D reconstruction, e) dense 3D reconstruction, f) coordinate transformation, g) mesh generation (Snavely, Seitz & Szeliski, 2008; Frahm et al., 2010; Agarwal et al., 2011).Main drawback in the image-based methods is in that images contain all the useful information to derive 3D geometry and texture at low cost, but require a mathematical formulation (perspective or projective geometry) to transform 2D image observation into 3D information. Furthermore, the recovering of a complete, detailed, accurate and realistic 3D textured model from images is still a difficult task, in particular for large and complex sites and if uncalibrated or widely separated images are used.A second problem is that these approaches were developed for consumer cameras and cannot provide resolution as high as and above 10 Mp, which modern cameras can readily provide. This limited resolution, coupled with Bayer pattern sensor demosaicking from a single matrix of red, green, and blue pixels, can severely limit the color accuracy. Performances sometimes are unclear and often low

reliability and repeatability (Remondino, Del Pizzo, Kersten & Troisi, 2012). Moreover, a deep and metric evaluation of the different (hidden) steps is still missing.Other unsolved issues, as always in the photogrammetric pipeline, involve:

Efficiency of photogrammetric processing algorithms that can drop out for limited image quality, or certain surface materials to be acquired, resulting in noisy point clouds or difficulties in feature extraction;

Known distance or Ground Control Points (GCP), required in order to derive metric 3D results;

Variations that arise from the use of various cameras by different working groups, which can further affect many photo-consistency-based reconstruction algorithms (Zhao, Zhou & Wu, 2012);

Color capture, management and rendering (i.e., OpenGL graphics).

The use of 3D acquired data inside a BIM workflow introduces further problems, mainly linked to the lack of point clouds proper editing commands, making BIM production a highly manual, time-consuming process. Filling the gap between unstructured acquired data and semantic objects in BIM is a challenging task. Automatic methods for structural and semantic analysis of point clouds are essential. Although robust methods have been proposed, the application in AEC industry is still to be developed (Tang, Huber, Akinci, Lipman & Lytle, 2010; Hichri, Stefani, De Luca & Véron, 2013).

Knowledge-based modeling in BIM platform

In the light of information organization, BIM is an ideal platform for representation and management of AH. Inherently BIM is semantically structured as a system of information. The information comes from prior knowledge and reality-based data acquisition. The prior knowledge could be extracted from Architectural Treatises that are served as a knowledge system. Before the advent of digital media, Treatises are one of the most important approaches for architectural knowledge broadcasting. The organization of BIM is highly analogous to the nature of architectural Treatises in terms of semantic structure and parametric relationships. Classical architecture has always depended on precedents and therefore on Treatises. Vitruvius himself indebted to ancient authors, and the later architects and theorists such as Alberti, Palladio and Claude Perrault to a great extent depended on Vitruvius. Instead of merely imitating the contents of Architecture Treatises using digital 3D model, recent research tends to focus on extracting the shape grammar of classical architecture developed by (Stiny & Mitchell, 1978; Mitchell, 1990). Parametric modeling and knowledge extraction from architectural Treatises have been proposed in (De Luca, Véron & Florenzano, 2007; Murphy, McGovern & Pavia, 2011). They aims at constructing parametric library of architectural component and integrate it with hybrid data acquisition techniques.BIM is an ideal platform for knowledge-based modeling not only for its semantic structure, but also for its potential to integrate as-built information. Among the many features of BIM software, object-based modeling is a huge gain when used for as-built BIM in terms of semantic organization and data segmentation. Instead of the traditional CAD pipeline, in which operators begin to work from scratch or they use drawing templates, BIM users define at first a family or a class (Eastman, Eastman, Teicholz & Sacks, 2011). The family or class is essentially the category for architectural component (window, column, beam, etc.), which retains geometry, relationships and structural information corresponding to its semantic logic. E.g. Autodesk Revit 2015 represents architectural objects with three types of family varying in potentials of as-built modeling:

system families, hard-coded into the software parametric engine, and slightly editable by users;

loadable families, authored starting from templates and can be used in several different projects;

in-place families, allowing user to customize but used primarily in single projects.

System family retains the most intelligence but limited possibility to integrate with captured data. System families consist of components such as wall, floor, and ceiling whose distinctions are generally quantitative (e.g. thickness, length, width), and therefore should be defined numerically rather than geometrically. Loadable families are the most commonly used family in Revit. They are created from templates where semantic behaviors have been pre-defined. Loadable families provide a more flexible modeling environment than system family, so the distinctive components such as Order (column), Entablature (beam), window and door could be made in loadable families with parametric intelligence for potential adaption and variation generation. Extracted features from captured data could be integrated to loadable families by nesting a cross section on which a 3D shape is built by extrusion (Entablature), revolve (Order) or sweep (window and door). In-place family does not retain any pre-defined constraints, for it is created as instance drawing like in traditional CAD. The drawing is not compulsory to obey the construction logic of any category. Hence it is in in-place family that irregular objects could be made (e.g. an inclined wall) with the loss of parametric intelligence and semantic information. A totally different approach in Revit is the "mass modeling" family authoring, in which many editing commands are available. Summarizing it’s possible to state that today BIM solutions are rich in potential but many problems emerge for an effective use.

Structuring acquired information for BIM

Structuring acquired information is a critical step towards as-built BIM. The unstructured point clouds from laser scanning or photogrammetry cannot be directly handled or recognized by current BIM software.Categorizing the acquired data into semantic objects and extracting the topological data on object level is mandatory. The need for semantically rich 3D model has long been existed in the field of architectural heritage (De Luca, Véron & Florenzano, 2007; Apollonio, Gaiani & Corsi, 2010) for improved information organization and representation. BIM inherently structures the 3D model semantically, hence it is on object level that acquired information should be extracted, enriched and converted to as-built BIM.In conventional approaches, as-built BIM is primarily manually created from point clouds, which is labor-intensive, costly, and time consuming. In spite of the growing literatures in automatic 3D modeling from point clouds, there are little or no automated process of semantic extraction applied to AEC industry currently (Tang, Huber, Akinci, Lipman & Lytle, 2010). The complex geometry of AH makes the process even more difficult (Hichri, Stefani, De Luca & Véron, 2013). On the other hand, the robustness of process is affected by the accuracy, convergence and density of point clouds especially when the point clouds is generated by image-based techniques (Haala & Kada, 2010). Some of the current researches rely on assumptions of planar surface and volumetric primitives, hence they have most been applied to indoor environments (Xiao & Furukawa, 2012; Oesau, Lafarge & Alliez., 2013; Xiong, Adan, Akinci & Huber, 2013; Turner & Zakhor, 2014; Mura, Mattausch, Villanueva, Gobbetti & Pajarola, 2014) with relative simple geometry such as walls, ceilings, floors and doorways. Other recent solutions (Wang, Cho & Kim, 2015) aiming at automatically extracting building geometries from unorganized point clouds are limited to planar elements and typical elements (e.g. planar walls and roofs, doors, windows, etc.).Extending the automated semantic recognition to more complex scene requires prior knowledge such as topological description of architectural components (Pu & Vosselman, 2009; Vanegas, Aliaga & Benes, 2012) or pre-defined formal grammar rules (Yu, Helmholz, Belton & West, 2014). Another promising approach of semantic extraction is embedding RFID tags on demanded objects (Hajian & Becerik-Gerber, 2009; Valero, Adan & Cerrada, 2012), while most current applications are not addressed for AH, but for virtual reality, gaming, navigation and simulation applications (Turner & Zakhor, 2014). Unfortunately, manual segmenting the captured data is still a necessity for AH in most cases and the process is time-consuming and labor-intensive. BIM by default allows users to filter the point clouds by leveling and

sectioning, but the process is error-prone when high accuracy is expected. Best fitting algorithms could be executed via commercial software, but the robustness have been limited to primitives such as plumbing (Scan to BIM) and simple interior environment (Kubit VirtuSurv). Starting from this framework we developed a complete solution from data acquisition to project delivery based on low-cost tools and instruments, easy-to-use, and perfectly integrated in current BIM practices to extend BIM techniques also to the lifecycle of AH.The key point of our solution is that it tries to overcome the problem of structuring acquired information for BIM starting from a different point of view. Rather than trying to find an automatic solution we focused the problem explaining the different steps and the complexity observing the AH construction and logic rules.Mainly we noted that for some steps we have a direct solution, while for other no. For some situation we need a robust structuring of the acquired data, for other it is possible to use as-built raw models as a simple low-level bounded geometry. E.g. if you do not need change shapes as in the most decorative parts (where it is possible just a cleaning of the existing element) you could put the as-built model directly in the BIM system without structuring. Otherwise you could use the points as template trying to fit the points to a library of parametric objects related to the classical architecture as described and parameterized in the Treatises.The developed solution is based on three ideal steps and their integration at BIM level:

Step 1: Knowledge-based modeling

Step 2: Data capture and as-built model construction

Step 3: Ideal and as-built model integration.

In the stage of data acquisition, we combine knowledge-based modeling with survey data. The knowledge-based modeling converts geometry and relationship of architectural components from architectural Treatises to BIM. The resulting BIM is used as a template to which survey data are enriched. In return, BIM is adapted to the survey data by controlling parameter instead of intensive drawing as in traditional CAD. Our data capture approach is based on image-based modeling, which combines automation with accuracy. The results show that both the accuracy and completeness of the resulting 3D model are apparently improved compared to the original solution. The ideal model from parametric library and as-built model from captured data are semantically integrated. The deviation between them is evaluated to determine if feature extraction from point clouds is essential to improve the accuracy of as-built BIM.Overall our solution speeds-up the today process and does not inhibit subsequent improvement using automatic techniques to structure point clouds. On the contrary, by creating a classification between the elements it is possible to recognize specific classes which can be managed through specific appropriate algorithms.We know that our solution is theoretically complex but we experienced that it is practically easy and without ambiguities for the operator that need just a basic knowledge about AH to take the right solution.

MATERIALS AND METHODS

The proposed solution is basically a framework aimed at the integration of newly developed digital technology in data acquisition and data representation with the semantic-based knowledge of AH extracted from architectural Treatises.Treatises were founded since Renaissance on the observation (measure) of ancient ruins on one hand, and set up patterns and grammars (draw) on the other. Elements, patterns, rules and variants are well defined in Treatises. We extend treatise technique allowing the integration of as-built data with ideal data to the BIM domain integrating ideal model with as-built model.

As said above, developed solution is based on three ideal steps and their integration at BIM level:

knowledge-based modeling;

image-based modeling;

integrating ideal model with as-built model.

We illustrate each step separately.

BIM-based knowledge-based modeling – A parametric library after Palladio

BIM knowledge-based modeling transforms the semantic composition and their ratio to 3D model. This technique presents a fundamental background in the Treatises of architecture. We exploited extensively this background. Stemmed from the ideal Antiquity, the Treatises served as a knowledge system that set up the grammar of classical language and guided the rules of architectural practice. Treatises such as Andrea Palladio’s I Quattro Libri dell'Architettura (1570) exerted profound influence in Europe since its publication in 1570 and later in North America. Palladio’s treatment on villas’ plan was studied for its similarity with parametric shape grammar (Stiny & Mitchell, 1978). I Quattro Libri dell'Architettura could be regarded as a system of knowledge from which rules of formal composition are employed to generate variations. Palladio’s Treatise is an ideal source for knowledge-based modeling in BIM. It is possible to build BIM components strictly based on Palladio's description on tangible elements, to make parametric models using constraints based on Palladio's module, and to generate parametric façade. Instead of drawing a real shape, we can replace dimensions by variables, and set formulas among them. In any case, the dimension adjustments are automatic. One modification will reflect to the others in real time. The work is essentially a process of information translation between two systems of knowledge.Lack of parametric library is one of the current challenges for as-built BIM in AH. However, architectural Treatises serves as a source from which architectural components could be converted to BIM object. The parametric BIM objects retain the potential for qualitative adaption and quantitative adaption to captured data. In the former case, a new family is built with different geometry to the template; while in the latter, only a new type is generated with different dimension to the template. Both of them are reusable and enrich the parametric library.We developed a method based on Autodesk Revit 2015 to put in place the whole knowledge-based design system proposed by the classical architecture illustrated in the Treatises. It retains parametric features without any embedded language, and architectural components are inherently semantic. The BIM platform supports additional attributes assigned to the components. The simulation and interoperability are manipulated in object level instead of model level. Revit API (Application Programming Interface) provides access to customize and optimize the modeling process and the management of database. Multiple information are supposed to translate among different interfaces to better preserve and share the 3D model. In order to be sufficiently concise, but to give a clear exposition in this part, we demonstrate the parametric modeling from the canon of orders to a parametric unit of façade based on the Palladian grammar.Palladio described the semantic structure of orders and their ratios by texts and illustrations. In the case of Doric orders, Palladio wrote: “. . . The capital ought to be in height half the diameter of the column, and is to be divided into three parts. The upper part is given to the abaco and cimacio. The cimacio is two of the five parts thereof, which must be divided into three parts; with the one the listello is made, and with the other two the goal. The second principal part is divided into three equal parts, one to be given to the anelli or annulets, or gradetti, which three are equal. The other two remain for the ovolo, which projects two thirds of its height. The third part is for the Collarino . . . ” (Palladio, 1570).

Figure 1. Transforming Palladio’s Treatise on Doric Order to BIM

Based on the construction logic above, more parametric objects can be built after Palladio and even after other architectural theorists such as Scamozzi and Vignola (figure 2). The translation of the Doric Order after Palladio into the other developed after Scamozzi, for example, is only a matter of formula reset - their discrepancies lie in the ratio among semantic nodes (e.g. abaco/cimacio). Moreover, the parametric modeling to the level of building façade can be extended by the rearrangement of parametric objects based on the shape grammar extracted from Palladio's Treatises.

Figure 2. BIM family of Doric orders after Treatises of Palladio (left), Scamozzi (middle) and Vignola (right)

A basic unit that compose Palladio's façade is usually articulated in four rules: 1) A pair of pilasters whose capitals could be either Ionic or Corinthian, the shafts are cubes or cylinders, the height dominates one or two floors; 2)A window with top molding’s cornice either triangular or curve (If the pilasters dominate two floors in rule one, then a rectangular window is added either above or below the window on the noble plan); 3) A balcony with molding and an array of balusters, and 4) Two segments of Entablatures on the bottom and top of the façade unit. In this case, we set only variables in the first and the second rules, while keep the third rule and fourth rule constant. The second rule is articulated with a “If, then” sentence. The result of the first rule determines if the part after “then” will be executed. The application of the first rule and the second rule lead to the generation of façade’s unit (figure 3).

Figure 3. Parametric façade based on grammar of Palladio’s Treatise

If we keep in mind that the capital of the Ionic and Corinthian orders are composed by members as well as sub-members, we realize the Bottom-Up process from atom units to the whole façade. We can apply the process of Order to other members and their sub-members down to the atom units. In this way, we create all the other possibilities from the same compositional rule.

In most cases, classical architecture does not employ new geometries, but different combinations and regroups of elements based on certain rules. We classified these rules allowing a quick interpretation and modification of the elements of the family using parametric techniques.

DATA ACQUISITION TECHNIQUES

Automated image-based 3D reconstruction is affected from its development in the Computer Vision community by a trend that tends to focus more on the automation than on the accuracy. Metric accuracy and completeness in complex scene are of prime concern when automated photogrammetry is applied to data acquisition in AH. Therefore we developed techniques to fill the current gap of automated techniques to make them appropriate to our use (figure 4), namely:

a number of techniques and schemes of shooting;

several improvements to the automatic photogrammetry chain to make it efficient and effective in our case; mainly: an efficient approach for the identification of tie developed starting from a careful analysis and calibration of different operators of interest; a series of images pre-processing techniques easy to use also by non-expert operators and without the need of sophisticated equipment (Apollonio, Ballabeni, Gaiani & Remondino, 2014);

techniques to assist the automatic camera parameter finding (Ballabeni, Apollonio, Gaiani & Remondino, 2015);

a solution to the problem of faithful color mapping (Apollonio, Fallavollita, Gaiani & Sun, 2013b).

We developed our data acquisition techniques in two case studies. The first case study, the façade of Palazzo Albergati, is a 55 meter-long façade built by bricks and decorated with stone in Entablature and Cornice. The second case study is represented by a five-bay portico, mostly covered by plaster. The two buildings were selected for their different architectural typology and material, both of which lead to different strategies in camera network, image enhancement, bundle adjustment, feature descriptor and dense reconstruction. The façade is considered as a 2D planar surface where subtle distortion caused by various factors (e.g. camera network, feature descriptor, etc.) are shown by comparing to laser scanned model. The main challenges in portico come from the dramatically altered illumination and low-texture surfaces, which caused difficulties in image orientation and model completeness.

Figure 4. The developed automatic photogrammetry pipeline

Façade

(a) Camera network The camera network for façade, as suggested by various tutorials of commercial software, requires perpendicular lens orientation and parallel baseline to the façade. But such camera network tends to cause distorted 3D model according to our experiment. From a photogrammetric point of view, such camera network dose not benefit to metric accuracy because of low baseline to depth ratio (B:D). Large baseline to depth ratio explicitly leading to convergent camera network could improve the metric accuracy (Remondino & El‐Hakim, 2006) and to be necessary for on site targetless camera calibration (Barazzetti, Scaioni & Remondino, 2010).Our developed camera network is an integration of both: it employs a global parallel-axis network for coverage, and a local convergent network with orthogonal camera rotation for accuracy. The image orientation and 3D recovery will initialize from the local convergent camera network (depicted in 2.2.1 (c)). The results show that our proposed camera network (bottom group in figure 5) yields better accuracy than the other two groups: one provides only parallel-axis images to the facade (top group in figure 5), the other one also based one parallel-axis but provides redundant images (middle group in figure 5).

Figure 5. Influence of camera network on metric accuracy (The laser scanned point clouds are obtained via Leica C10 with one shot/low resolution). Top: camera network with baseline parallel to façade; Middle: camera network with redundant images; Bottom: our developed camera network

(b) Image enhancementThe raw image set without image enhancement is not an ideal source for image-based modeling, as they could contain noise, chromatic aberration and limited distinctive features on low-texture surfaces. Therefore, we applied a set of image enhancement such as noise reduction, color calibration (figure 6) to the image set. A detailed description could be found in (Ballabeni, Apollonio, Gaiani & Remondino, 2015).

Figure 6. Comparison between a 8-bit photograph as shot (top), and its version calibrated using our color workflow (bottom)

(c) Bundle adjustmentCamera orientation automated techniques are based on Structure from Motion (SfM) concept, firstly introduced by (Ullman, 1979), i.e. the automated and simultaneous determination of camera parameters together with scene’s geometry. SfM systems recover the camera positions by incrementally triangulating successfully matched features and related cameras. Initializing from a well-matched image pair, 3D model is reconstructed and grown by repeatedly triangulating new cameras until no new cameras reach the minimum inlier matches. The incremental procedure employs Bundle Adjustment (Triggs, McLauchlan, Hartley & Fitzgibbon, 2000) and outliers filter (typically a RANSAC algorithm (Fischler &

Bolles, 1981)) to reduce projection errors, but the misalignment may still accumulate as the model grows. Therefore, initializing from a solid camera network would improve the accuracy. We initialize the 3D reconstruction from the local convergent network containing lens rotation, then execute bundle adjustment, and finally resume the process orienting the other images.

(d) Feature descriptorsMost SfM systems use SIFT operator (Lowe, 2004) to extract features from image, while the Computer Vision community prevails efficiency to accuracy by constraints of maximum features (Wu, 2013). SIFT descriptors have also proved to be robust to a wide family of image transformations, such as slight changes of viewpoint, noise, blur, contrast changes, scene deformation, while remaining discriminative enough for matching purposes. As reported in literature (Morel & Yu, 2009; Remondino, Del Pizzo, Kersten & Troisi, 2012; Zhao, Zhou & Wu, 2012; Apollonio, Fallavollita, Gaiani & Sun, 2013b), the typical failure cases of the SIFT algorithm are changes in the illumination conditions, reflecting surfaces, object / scene with strong 3D aspect, highly repeated structures in the scene and very different viewing angle between the images. To ensure strong efficiency to the operator it’s fundamental a correct calibration of the different parameters embedded in it’s algorithmic chain. Among the parameters of SIFT, ‘peak’ is a crucial one that closely relate to accuracy by altering image contrast threshold (May, Turner & Morris, 2010). We improve the accuracy of feature detection specifically calibrating SIFT parameters using the open implementation Vlfeat (http://www.vlfeat.org) (figure 7).

Figure 7. Influence of different feature operators and different values on metric accuracy (The laser scanned point clouds are obtained via Leica C10 with one shot/ low resolution)

Portico

(a) Camera networkThe quality of input image is fundamental in image-based modeling. It has to ensure the essential coverage of all the demanded surfaces on the one hand, and avoid redundant images acquisition on the other since it not only increases work intensity but also reduces the metric accuracy due to small Baseline/Depth ratio (Barazzetti, Scaioni & Remondino, 2010). As an architectural typology, the portico is more challenging than the façade or piazza to design the camera network. This is mainly because it is not a fully convex or concave space but a mixture of both. Then, unevenly exposed, portico often has a dramatically altered illumination and limited angle of view as well as low textured areas (plasterwork such as vault). According to our experience, the image orientation tends to fail in vaults and columns because such surfaces are either low-texture or isolated from the rest of the portico. We propose a semantic-based approach that (1) designs the camera network for each semantic part (façade, column, floor and vault) and (2) it integrates all the semantic parts, ensuring that the transition in-between has been sufficiently covered (figure 8).The final camera network includes 212 images.

(b) Image enhancement Due to the characteristics of portico's configuration, unlike what was seen in 2.2.1 (b), we applied the Wallis filter (Wallis, 1976) to the image set, since it is able to overcome the difficulties of portico such as uneven exposure of different parts and low-texture surface. The vault of portico, built in plaster, is extremely uniform and this leads to failure of feature extraction and image orientation. Besides, the illumination varies dramatically between the surfaces under the portico and surfaces outside the portico. Using Wallis filter, the features on low-texture areas are extracted in a well-distributed way, and the uneven exposure is flattened (figure 9). A more detailed description of this step could be found in (Ballabeni, Apollonio, Gaiani & Remondino, 2015). This step leads to an obvious improvement in the completeness of resulting model. (figure 10).

Figure 8. Semantically designed and obtained camera network for portico

Figure 9. The image enhancement process using Wallis filter. Top: the original color image; in the middle: greyscale converted; bottom: the image wallised

Figure 10. Comparison between original image set and wallised image on modeling results

(c) Dense reconstructionThere are various multi-view-stereo algorithms that generated dense point clouds from oriented images and sparse points. Compared to the CMVS+PMVS2 process (Furukawa et al., 2010), we use SURE (Rothermel, Wenzel, Fritsch & Haala, 2012) for its better completeness in the resulting point clouds. CMVS first decomposes the oriented images into clusters, and PMVS2 uses a patch-based algorithm to reconstruct dense 3D points for each clusters independently. First of all, PMVS2 tends to automatically discard redundant images to avoid computational effort and clean the noise (Furukawa et al., 2010). When the structure of architecture is highly repetitive, such as portico, it is prone to discover redundant images improperly and cause holes on the 3D model. Secondly, PMVS2 initializes from sparse cloud of points carrying the typical problem due to the shooting position, from street level, that is, the excessive number of points in the lower part of the portico and the near absence in the high part of the façade.Different from PMVS2, SURE employs semi-global matching and depth map, which are proved to be more robust and avoid the holes on 3D model caused by redundant image removal. More in general other studies, demonstrated it is efficient (Remondino, Spera, Nocerino, Menna & Nex, 2014).

A semantic approach towards as-built AH BIM

Our semantically structured framework integrating ideal model with as-built model is a balance between parametric intelligence with accuracy in BIM, and keeps manual modeling within a reasonable degree. This approach is based on the fact that full automation from point clouds to as-built BIM is not available in AEC industry yet, and documenting all the as-built data such as irregular shape with BIM is neither possible nor a necessity to most situations. The process contains following steps (figure 11):(a) Adjusting BIM object to point cloudsKnowledge-based modeling provides a parametric template to which captured data of as-built state is enriched. A semi-automated mapping process is addressed in (Dore & Murphy, 2014), but it is applied to building façade but not to more complex situations. The BIM objects from parametric library are adapted to the point clouds. If the typology already exists, a new type is made by adapting the parameters of existed family to point clouds; if not, a new typology is made and added to the parametric library for future use. The research on modeling effort and impact of different Level-of-Detail (LoD) in BIM (Leite, Akcamete, Akinci, Atasoy & Kiziltas, 2011; Fai & Rafeiro, 2014) shows that more detailed modeling does not necessarily mean more modeling work, and potential simulations would benefit of higher precision as well.(b) Evaluation of deviation The components of AH are prone to be irregular because of damage, deterioration and deformation during their life cycle. Although the BIM objects are mapped on the captured point clouds, the primitives in BIM may not be sufficiently precise when the deviation reaches a certain threshold. The evaluation of deviation between BIM objects and captured data becomes a necessity. (Quattrini, Malinverni, Clini, Nespeca & Orlietti, 2015) propose a method to evaluate such deviation in complex architecture and how to preserve the diagram as attributes of objects by 3rd party software. We use the plug-in of Revit 2015-Autodesk Point Layout 2015-to automatically carry out the process, and then decide which objects do not meet the demanded accuracy. Such objects will be modeled from the extracted features of captured point clouds.

Figure 11. The developed workflow towards as-built BIM

(c) Feature extraction

Various approaches topologically bridging the gap between point clouds and BIM exist, such as converting point clouds to BIM via NURBS (Oreni et al., 2014), extracting feature points (Garagnani, 2013) and extracting cross sections (Coutinho et al., 2013). Cross section is an effective way to extract features from point clouds of AH to BIM, since classical architecture are closely related to moldings. Most classical architecture elements can be modeled by sweeping certain profile along a straight line or a curve. It is not a coincidence but closely analogous to the craftsman’s technique of using a template to make a profile on a block of stone and then cut off the extra parts (Mitchell, Mateus, Duarte, Ferreira & Kruger, 1995). Cross sections could be transformed into 2D drawings and saved as profiles in BIM. The profile is a reusable family that could be nested into other families.

RESULTS AND DISCUSSION

We demonstrate our workflow and techniques with two case studies.

Case study 1

Image-based modeling of Bologna’s porticoes (Bocchi, 1997) was launched to cater its application of World Heritage of UNESCO. Range-based approach was discarded for low flexibility to traffic and pedestrians in the city center given the large scale of 42km-long portico. As the most distinctive feature of the portico, vault is a problem for image orientation due to the low-texture surface and dramatic transition of illumination. Another challenge is to construct as-built BIM models based on the captured point clouds.Our developed photogrammetry pipeline reconstructed the five-bay portico as described in 2.2.2. The results show an apparent improvement in completeness compared to the original process contributed by Computer Vision community (Wu, 2013; Furukawa & Ponce, 2010) (figure 12).

Figure 12. Comparison between original process and our developed process in terms of completeness

As-built BIM is based on the resulting point clouds in a semantic sequence. Walls, floor, windows and doors are imported from the parametric library and adjusted to the point clouds by modifying dimensional parameters. Then the semantic objects are compared to point clouds respectively to evaluate their accuracy. In spite of the irregular shape of real objects (e.g. unflatten surface of wall), the accuracy of as-built BIM is acceptable to most cases (figure 13). Feature extraction is carried out for semantic objects that beyond the acceptable range of accuracy such as the vault. We could know even from prior knowledge that vault is by no means Boolean operations of primitives - as it appears - but compromise of irregular disposition of building plan, unflatten site and joints of problematically constructed column and wall. Firstly the point clouds of vault is segmented from the global model and transformed into NURBS in Rhino. Then generative tools (grasshopper) is used to extract splines from the NURBS model of vault. The procedure of feature extraction is automated and reversible, because it is based on following parameters: level of detail of resulting NURBS model, number of extracted cross sections and number of extracted points on each cross section (figure 14). Level of detail determines the accuracy of the resulting BIM model: the larger the threshold, the less deviation between point clouds and BIM model (figure 15). Finally in Revit the surface of vault is generated from the splines via Revit’s plug-in (Garagnani, 2013) (figure 16). The model is generated in in-place family, so metadata should be assigned to the model of vault by embedding IFC (Industry Foundation Classes) information including semantic data, material, deviation map, etc.

Figure 13. Deviation evaluation between semantic object and point clouds

Figure 14. Parametrically extracting splines from point clouds. Meaning of parameters: LoD of surface - level of detail of resulting NURBS; No. of sections - Number of extracted cross sections; Pts. per sections - Number of extracted points on each cross section

Figure 15. Accuracy evaluation of LoD threshold in extracting splines from point clouds

Figure 16. Pipeline from image-based modeling to as-built BIM

Case study 2

Palazzo Barbaran is one of the remains of Palladio’s Palazzo whose drawings are collected in the third part of his Treatise, The Four Books on Architecture (Battilotti, 1999).Although the building was expanded with two extra bays on façade after Palladio’s death, the existing part was completed under his full supervision. We reconstructed the 3D model of this façade following two approaches - knowledge-based modeling and image-based modeling - with the aim to find the discrepancies between the ideal model and the as-built model and integrate them into a BIM database.The knowledge-based approach yields to the ideal model as it is strictly based on Palladio's grammar. Palladio set up rules of architectural components such as orders and explained how the rules could be applied to practice with flexibility. Columns, windows and molding are created in Family environment of Revit 2015 based on such rules, and then globally composed to façade coupled with temporal evolution (figure 17).

Figure 17. Knowledge-based approach based on Palladio’s Treatise integrated with temporal information (semi-transparent façade showing the expanded bays)

The as-built model is reconstructed from 172 images by automatic photogrammetric techniques. Then a series of processing steps are carried out to transform the captured data to semantic organized objects in BIM (figure 18).

Figure 18. Image-based approach and post-processing towards as-built BIM

In order to compare the discrepancies between the ideal model and the as-built model, point cloud are extracted from the polygonal surfaces of the ideal model, and registered to the points cloud from survey with further refinement to ensure the accuracy. For each point in the ideal model, the closest point is found in the as-built model and the distance in-between is calculated. Discrepancies are observed in the Entablature of Ionic Order on the ground floor and Entablature of Corinthian Order on the first floor. The distinction exists in the molding of Entablature: lines in real construction replace the curves in Palladio’s Treatise (figure 19). In BIM the two models could be defined as two options of single component: when one option is active, the other is filtered. The ideal model and the as-built one in each semantic level (e.g. Order, Capital, etc.) could exist in parallel. The approach could be further applied to AH where discrepancies between different source of information such as original drawing, draft and Treatises exist (Apollonio, Gaiani & Sun, 2013a).

Figure 19. Integrating the ideal model and as-built model into a single BIM system by its options

The significance of our approach includes: the developed photogrammetric approach is robust to complex scene and generates point clouds

with high accuracy and completeness. The techniques are supposed to be versatile to AH in use given the superiority in cost, portability and duration of on-site work;

architectural Treatises could serve as a source of knowledge from which patterns and grammars of AH are extracted to enrich the missing parametric library.

the integration of ideal model and as-built model not only reveals their difference, but also provides reference to feature extraction for accurate as-built BIM which depends on the purpose, use and time restriction of the work.

FUTURE RESEARCH DIRECTIONS

Future developments could be addressed in 2 aspects: 1) The application of proposed method in AH will benefit from the fast-growing capability of image sensing and its integration with UAV. As the camera sensor develops, it is expected that better resolution and accuracy could be achieved in resulting model while less images capture is needed. The low-altitude photogrammetry is very efficient in modeling large-scale AH and components beyond terrestrial capture angle. Our approach could be based not only on terrestrial image input, but also on aerial images. Hence more types of AH could be efficiently modeled by proposed method; 2) Effective segmentation algorithms will fill the gap between captured data and BIM. Currently only simple geometry could be accurately extracted from point clouds, making the segmentation of AH a labor-intensive process. Although our case study use a set of codes to partially solve this problem, it is still a complicated process which needs time and training. Once this transition become smooth, the reverse-modeling workflow from images to BIM would be widely applied to more types of AH.

CONCLUSION

The paper presents a complete process from image-based data acquisition approach to as-built BIM for AH. Automated workflow from images to 3D model has been established by computer vision community, while its metric accuracy in AH is questionable. We demonstrated that our optimized image-based approach generates high accuracy comparable to that of laser scanning. In spite of the recent development of laser scanning (cheaper, lighter and integration with external camera), generally camera-based data acquisition approach is still more versatile than that of laser scanning when applied to AH. Hence the proposed data acquisition approach could not only be applied to monumental architecture, but also to historic building in city center such as the case study. This solution is low-cost, easy to use and combines accuracy with automation and then is more in line with the BIM approach. Knowledge-based modeling serves as component library and provides prior knowledge for data segmentation and feature extraction. In contrast to traditional CAD, operators builds not only shapes in BIM but also parametric relationships extracted from shape grammar, e.g. architectural treatises. Object-oriented modeling of BIM is inherently close to the nature of classical architecture. Semantic structure, types of profiles and mathematical constraints from architectural treatises could be translated into BIM, and adapted to several different projects integrated with as-built data.The parametric family in BIM allows dimension modification, joints adjustment and further data enrichment such as temporal information, material and uncertainties. The as-built models generated by image-based modeling and ideal models based on architectural Treatises are integrated in BIM. BIM objects from parametric library are adapted to as-built data if the typology exists, otherwise a new typology is created and added to the parametric library for potential use. The as-built BIM objects are compared to point clouds for evaluation of deviation. If the deviation is beyond acceptable threshold, features are extracted from point clouds for more accurate modeling.

REFERENCES

1. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., & Szeliski, R. (2011). Building Rome in a day. Communications of the ACM, 54(10), 105-112.

2. Apollonio, F. I., Gaiani, M., & Corsi, C. (2010). A semantic and parametric method for 3D models used in 3D cognitive-information system. In Proceedings of the eCAADe Conference (pp. 863-872). Zurich, Switzerland: ETH Zurich.

3. Apollonio, F. I., Gaiani, M., & Sun, Z. (2012). BIM-based modeling and data enrichment of classical architectural buildings. SCIRES-IT, 2(2), 41-62.

4. Apollonio, F. I., Gaiani, M., & Sun, Z. (2013a). 3D modeling and data enrichment in digital reconstruction of architectural heritage. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (Vol. XL-5/W2, pp. 43-48).

5. Apollonio F.I., Fallavollita, F., Gaiani, M., & Sun Z. (2013b). A colour digital survey of arcades in Bologna. In Colour and Colorimetry Multidisciplinay Contributions (vol. IXB, pp. 58-68). Rimini, Italy: Maggioli Editore.

6. Apollonio, F. I., Ballabeni, A., Gaiani, M., & Remondino, F. (2014). Evaluation of feature-based methods for automated network orientation. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (Vol. XL-5, pp. 47-54).

7. Ballabeni, A., Apollonio, F. I., Gaiani, M., & Remondino, F. (2015). Advances in image pre-processing to improve automated 3D reconstruction. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, (Vol. XL-5/W4, pp. 315-323).

8. Barazzetti, L., Scaioni, M., & Remondino, F. (2010). Orientation and 3D modelling from markerless terrestrial images: combining accuracy with automation. The Photogrammetric Record, 25(132), 356-381.

9. Barazzetti, L., Banfi, F., Brumana, R., Gusmeroli, G., Oreni, D., Previtali, M., ... Schiantarelli, G. (2015). BIM from laser clouds and finite element analysis: combining structural analysis and geometric complexity. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (Vol. XL-5/W4, pp. 345-350).

10. Barber, D., & Mills, J. (2007). 3D Laser Scanning for Heritage: Advice and guidance to users on laser scanning in archaeology and architecture. Swindon, England: English Heritage Publishing.

11. Battilotti, D. (1999). Palazzo Barbarano. In L. Puppi, Andrea Palladio (Updated edition by D. Battilotti). Milano, Italy: Electa.

12. Bernardini, F., & Rushmeier, H. (2002). The 3D model acquisition pipeline. Computer graphics forum, 21(2), 149-172.

13. Blais, F. (2004). Review of 20 years of range sensor development. Journal of Electronic Imaging, 13(1), 231-243.

14. Blais, F., & Beraldin, J. A. (2006). Recent developments in 3D multi-modal laser imaging applied to cultural heritage. Machine Vision and Applications, 17(6), 395-409.

15. Bocchi, F. (1997). Bologna e i suoi portici: storia dell'origine e dello sviluppo. Bologna, Italy: Grafis.

16. Bryan, P., Blake, B., & Bedford, J. (2013). Metric survey specifications for cultural heritage. Swindon, England: English Heritage Publishing.

17. Callieri, M., Dellepiane, M., Cignoni, P., & Scopigno, R. (2011). Processing sampled 3D data: reconstruction and visualization technologies. In Stanco F., Battiato S., Gallo G. (Ed.), Digital Imaging for Cultural Heritage Preservation: Analysis, Restoration, and Reconstruction of Ancient Artworks (pp. 105-136). Boca Raton, FL: CRC Press.

18. Coutinho, F., Mateus, L., Duarte, J. P., Ferreira, V., & Kruger, M. (2013). From Point Cloud to Shape Grammar to Grammatical Transformations. In Proceedings of the eCAADe Conference (pp. 655-664). Delft, Holland.

19. De Luca, L., Véron, P., & Florenzano, M. (2006). Reverse engineering of architectural buildings based on a hybrid modeling approach. Computers & Graphics, 30(2), 160-176.

20. De Luca, L., Véron, P., & Florenzano, M. (2007). A generic formalism for the semantic modeling and representation of architectural elements. The Visual Computer, 23(3), 181-205.

21. Dore, C., & Murphy, M. (2014). Semi-automatic generation of as-built BIM facade geometry from laser and image data. ITcon, 19, 20-46.

22. Eastman, C., Eastman, C. M., Teicholz, P., & Sacks, R. (2011). BIM handbook: A guide to building information modeling for owners, managers, designers, engineers and contractors. Hoboken, NJ: John Wiley & Sons.

23. Fai, S., Graham, K., Duckworth, T., Wood, N., & Attar, R. (2011). Building information modelling and heritage documentation. In Proceedings of XXIIIrd International CIPA Symposium. Retrieved November 29, 2015, from http://cipa.icomos.org/index.php?id=69

24. Fai, S. & Rafeiro, J. (2014). Establishing an Appropriate Level of Detail (LoD) for a Building Information Model (BIM) – West Block, Parliament Hill, Ottawa, Canada. In ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences (vol. II-5, pp. 123-130).

25. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381-395.

26. Frahm, J. M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., ... Pollefeys, M. (2010). Building Rome on a cloudless day. In Computer Vision–ECCV 2010 (pp. 368-381). Berlin/Heidelberg, Germany: Springer

27. Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362-1376.

28. Garagnani, S. (2013). Building Information Modeling and real world knowledge: A methodological approach to accurate semantic documentation for the built environment. In Proceedings of Digital Heritage International Congress (DigitalHeritage) (vol. 1, pp. 489-496). Piscataway, NJ: IEEE.

29. Gašparović, M., & Malarić, I. (2012). Increase of readability and accuracy of 3D models using fusion of close range photogrammetry and laser scanning. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (vol. XXXIX-B5, pp. 93-98).

30. Guarnieri, A., Remondino, F., & Vettore, A. (2006). Digital photogrammetry and TLS data fusion applied to Cultural Heritage 3D modeling. In ISPRS International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, (vol. XXXVI-part5).

31. Haala, N., & Kada, M. (2010). An update on automatic 3D building reconstruction. ISPRS Journal of Photogrammetry and Remote Sensing, 65(6), 570-580.

32. Hajian, H., & Becerik-Gerber, B. (2009). A research outlook for real-time project information management by integrating advanced field data acquisition systems and building information modeling. In Proceedings of Computing in civil engineering, (ed. Caldas, C. H., O’Brien, W.J., pp. 83-94). Austin, TX: American Society of Civil Engineers.

33. Hichri, N., Stefani, C., De Luca, L., & Véron, P. (2013). Review of the «as-built BIM» approaches. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, (vol. XL-5/W1, pp. 107-112).

34. Hiep, V. H., Keriven, R., Labatut, P., & Pons, J. P. (2009). Towards high-resolution large-scale multi-view stereo. In Proceedings of Computer Vision and Pattern Recognition, 2009. CVPR 2009 (pp. 1430-1437). Piscataway, NJ: IEEE.

35. Hirschmüller, H. (2008). Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2), 328-341.

36. Leite, F., Akcamete, A., Akinci, B., Atasoy, G., & Kiziltas, S. (2011). Analysis of modeling effort and impact of different levels of detail in building information models. Automation in Construction, 20(5), 601-609.

37. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2), 91-110.

38. May, M., Turner, M. J., & Morris, T. (2010). Scale invariant feature transform: a graphical parameter analysis. In Proceedings of the BMVC.

39. Mitchell, W. J. (1990). The logic of architecture: Design, computation, and cognition. Cambridge, MA: MIT press.

40. Mitchell, W. J., & McCullough, M. (1995). Digital design media. Hoboken, NJ: John Wiley & Sons.

41. Mitchell, W. J. (2009). Thinking in BIM. A+U, special issue-Architectural Transformations via BIM, 10-15.

42. Morel, J. M., & Yu, G. (2009). ASIFT: A new framework for fully affine invariant image comparison. SIAM Journal on Imaging Sciences, 2(2), 438-469.

43. Motawa, I., & Almarshad, A. (2013). A knowledge-based BIM system for building maintenance. Automation in Construction, 29, 173-182.

44. Mura, C., Mattausch, O., Villanueva, A. J., Gobbetti, E., & Pajarola, R. (2014). Automatic room detection and reconstruction in cluttered indoor environments with complex room layouts. Computers & Graphics, 44, 20-32.

45. Murphy, M., McGovern, E., & Pavia, S. (2011). Historic building information modelling-adding intelligence to laser and image based surveys. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (vol. XXXVIII-5/W16, pp. 1-7).

46. Oesau, S., Lafarge, F., & Alliez, P. (2013). Indoor scene reconstruction using primitive-driven space partitioning and graph-cut. In Proceedings of the Eurographics Workshop on Urban Data Modelling and Visualisation (pp. 9-12). Eurographics Association.

47. Oreni, D., Brumana, R., Della Torre, S., Banfi, F., Barazzetti, L., ... Previtali, M. (2014). Survey turned into HBIM: the restoration and the work involved concerning the Basilica di Collemaggio after the earthquake (L’Aquila). In ISPRS International Annuals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, (vol. II-5, pp. 267-273).

48. Palladio, A. (1570). I Quattro Libri dell'Architettura. Venezia, Italy.

49. Pu, S., & Vosselman, G. (2009). Knowledge based reconstruction of building models from terrestrial laser scanning data. ISPRS Journal of Photogrammetry and Remote Sensing, 64(6), 575-584.

50. Quattrini, R., Malinverni, E. S., Clini, P., Nespeca, R., & Orlietti, E. (2015). From TLS to HBIM. High Quality Semantically-Aware 3D Modeling of Complex Architecture. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, (vol. XL-5-W4, pp. 367-374).

51. Remondino, F., & El‐Hakim, S. (2006). Image-based 3D Modelling: A Review. The Photogrammetric Record, 21(115), 269-291.

52. Remondino, F., El-Hakim, S., Gruen, A., & Zhang, L. (2008). Turning images into 3-D models. Signal Processing Magazine, IEEE, 25(4), 55-65.

53. Remondino, F. (2011). 3D recording for Cultural Heritage. In D.Cowley (Ed.), Remote Sensing for Archaeological Heritage Management, EAC Symposium (pp. 107-116).

54. Remondino, F., Del Pizzo, S., Kersten, T. P., & Troisi, S. (2012). Low-cost and open-source solutions for automated image orientation–A critical overview. In Progress in Cultural Heritage Preservation (pp. 40-54). Berlin/Heidelberg, Germany: Springer.

55. Remondino, F., Spera, M. G., Nocerino, E., Menna, F., & Nex, F. (2014). State of the art in high density image matching. The Photogrammetric Record, 29(146), 144-166.

56. Rothermel, M., Wenzel, K., Fritsch, D., & Haala, N. (2012, December). Sure: Photogrammetric surface reconstruction from imagery. In Proceedings of LC3D Workshop (pp. 1-9). Berlin.

57. Snavely, N., Seitz, S. M., & Szeliski, R. (2008). Modeling the world from internet photo collections. International Journal of Computer Vision, 80(2), 189-210.

58. Stamos, I., Liu, L., Chen, C., Wolberg, G., Yu, G., & Zokai, S. (2008). Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes. International Journal of Computer Vision, 78(2-3), 237-260.

59. Stiny, G., & Mitchell, W. J. (1978). The palladian grammar. Environment and planning B, 5(1), 5-18.

60. Tang, P., Huber, D., Akinci, B., Lipman, R., & Lytle, A. (2010). Automatic reconstruction of as-built building information models from laser-scanned point clouds: A review of related techniques. Automation in construction, 19(7), 829-843.

61. Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000). Bundle adjustment—a modern synthesis. In Vision algorithms: theory and practice (pp. 298-372). Berlin/Heidelberg, Germany: Springer.

62. Turner, E., & Zakhor, A. (2014). Floor plan generation and room labeling of indoor environments from laser range data. In GRAPP 2014 - Proceedings of the 9th International Conference on Computer Graphics Theory and Applications (pp. 22-33). Setubal, Portugal: SciTePress.

63. Ullman, S. (1979). The interpretation of structure from motion. In Proceedings of the Royal Society of London B: Biological Sciences (vol. 203/1153, pp. 405-426).

64. Valero, E., Adan, A., & Cerrada, C. (2012). Automatic construction of 3D basic-semantic models of inhabited interiors using laser scanners and RFID sensors. Sensors, 12(5), 5705-5724.

65. Vanegas, C. A., Aliaga, D. G., & Benes, B. (2012). Automatic extraction of Manhattan-world building masses from 3D laser range scans. IEEE Transactions on Visualization and Computer Graphics, 18(10), 1627-1637.

66 Vlfeat. Retrieved November 29, 2015, from http://www.vlfeat.org

67. Volk, R., Stengel, J., & Schultmann, F. (2014). Building Information Modeling (BIM) for existing buildings-Literature review and future needs. Automation in Construction, 38(3), 109-127.

68. Vosselman, G. V., & Maas, H. G. (Ed.). (2010). Airborne and terrestrial laser scanning. Caithness, Scotland, UK: Whittles.

69. Wallis, R.H. (1976). An approach to the space variant restoration and enhancement of images. In Proceeding of symposium on current mathematical problems in image science, naval postgraduate school (pp. 107-111). Monterey, CA: Naval Postgraduate School.

70. Wang, C., Cho, Y. K., & Kim, C. (2015). Automatic BIM component extraction from point clouds of existing buildings for sustainability applications. Automation in Construction, 56(8), 1-13.

71. Wu, C. (2013). Towards linear-time incremental structure from motion. In Proceedings of 3D Vision-3DV 2013, 2013 International Conference (pp. 127-134). Piscataway, NJ: IEEE.

72. Xiao, J., & Furukawa, Y. (2012). Reconstructing the world’s museums. In Proceedings of ECCV 2012 (pp. 668-681). Berlin/Heidelberg, Germany: Springer.

73. Xiong, X., Adan, A., Akinci, B., & Huber, D. (2013). Automatic creation of semantically rich 3D building models from laser scanner data. Automation in Construction, 31(5), 325-337.

74. Yu, Q., Helmholz, P., Belton, D., & West, G. (2014). Grammar-based Automatic 3D Model Reconstruction from Terrestrial Laser Scanning Data. In ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (Vol. XL-4, pp. 335-340).

75. Zhao, X., Zhou, Z., & Wu, W. (2012). Radiance-based color calibration for image-based modeling with multiple cameras. Science China Information Sciences, 55(7), 1509-1519.