uav photogrammetry for cultural heritage ......uav photogrammetry for cultural heritage preservation...

UAV PHOTOGRAMMETRY FOR CULTURAL HERITAGE PRESERVATION MODELING AND MAPPING VENETIAN WALLS OF BERGAMO

P. Azzola 1, A. Cardaci 1, G. Mirabella Roberti 1, V.M. Nannei 1

Department of Engineering and S.A., University of Bergamo, Bergamo, Italy ([email protected]; [email protected]; [email protected]; [email protected])

Commission II

KEY WORDS: Venetian Fortress of Bergamo, photogrammetric survey, planned maintenance, deterioration mapping ABSTRACT: In July 2017, the system of Bergamo City Walls was registered in the UNESCO World Heritage List, together with other Venetian fortresses in Italy, Croatia and Montenegro. This important historical document needs continuous care and caution in order to be preserved complete, despite the continuous atmospheric and anthropic attacks, giving rise to dangerous deterioration processes on the artefact. The UAV photogrammetry is a suitable surveying method for such an extended system, able to collect all the geometrical, material and deterioration information needed for an effective maintenance program, also quick enough to allow for a repeated monitoring of the entire wall circuit. This paper presents the UAV survey campaign planned, and partially already completed, in order to test the methodological choices and solve all the operational difficulties to use drones in an urban environment, mainly because short distance shooting of nearly vertical surfaces was required.

1. INTRODUCTION

The system of Bergamo City Walls was built by the ‘Serenissima’ Republic of Venice as the last bulwark of defence against Milan duchy in the second half of 16th century. Born as a military structure, the city walls were eventually used mainly as a custom boundary in 18th and 19th centuries, becoming finally a terrace for a panoramic promenade on the southern side and assuming definitely a landmark role in the transformation of the urban fabric. Bergamo remains an exceptional example of fortified city, very well preserved, characterized by an articulate

defensive work, perfectly adapted to the hilly formation of the site. The contrast between the medieval city, made by the ancient town with its urban shape, and the renaissance town, that express herself by means of military and symbolic elements (city gates, the wall layout, the typical shape of platforms and bulwark) defines the identity of the town, impressing strongly the memory of the visitors (figure 1). The insertion of the Walls in the WHL of UNESCO confirmed the importance of the site, stressing the need of elaborating a maintenance program, in addition to the need of collecting the different studies developed during the last years and share them with the citizens. The digital media can

Figure 1. The Bergamo city walls in a winter view from South, facing St. James’ bulwark.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W9, 2019 8th Intl. Workshop 3D-ARCH “3D Virtual Reconstruction and Visualization of Complex Architectures”, 6–8 February 2019, Bergamo, Italy

This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-2-W9-45-2019 | © Authors 2019. CC BY 4.0 License.

45

allow indeed the use of the same material by different types of users, from the tourist to the scholars. In this context the project of walls survey by means of drones was born, as a tool for mapping the consistency of the entire perimeter of the fortress.

1.1 The Venetian Fortress: history background.

In the second half of 16th century, Bergamo represented the western boundary of Venetian possession on the mainland, in front of the Milan dukedom, assuming a strategic role in the defence of the commercial ways towards North, just when the Republic of Venice started its unavoidable decline in the domination of maritime trading. The choice of fortifying Bergamo – and not another town in the lowland, possibly more militarily efficient – was achieved with determination and some misunderstanding under the lead of count Sforza Parravicino, who imposed his project also in contrast with other military expert of the time. The initial budget provisions were soon exceeded, imposing also the demolition of many private houses, but also churches and public monuments. Wall construction started in 1561 and finished only in 1588, with the ‘Fara’ bastion, encountering huge difficulties from economi¬cal and practical point of view; but during centuries were never attacked. Instead, in Christmas 1796 The French troops entered in Bergamo without one single shooting and city walls finished definitely their defensive role; but the civil usage of earthworks and moats by the citizens was already diffused. In 1825 the Municipality become owner of the top terraces facing south, and a boulevard was created increasing the height of the wall and adding trees and green areas (Figure 2).

1.2 The Venetian Fortress: maintenance and upgrading.

Starting from the Seventies, the city walls have undergone several occasional maintenance interventions, mainly facing temporary emergencies. Works that are more systematic have been carried out between 2001 and 2008 by the Municipality together with the Monument Office, but only on limited portion of the wall circle. Defining a maintenance program, meaning a daily care process of the different architectonical elements that constitute the ‘stone landscape’ of Venetian Walls, appears now mandatory. This process was started in the last years by the impulse of Bergamo Municipality, that signed a convention with the State Property Administration (owner of the majority of the walls itself) and ‘Orobicambiente’, a private volunteer association that collaborate in eliminating weeds from the walls. In July 2017, Bergamo City Walls were registered in the UNESCO World Heritage List, together with other Venetian fortresses in Italy, Croatia and Montenegro, as Venetian Works

of Defence between 16th and 17th Centuries: ‘Stato da tera’ – Western ‘Stato da Mar’. In the meantime, the Municipality started a maintenance planning of the walls aimed to fighting the decay phenomena. The conservation works go alongside to a continuous analysis of the state of the sites, which includes direct inspections and systematic controls. In the framework of the project Five km of stone landscape to take care of and keep in health, set up by Bergamo Municipality in cooperation with the University of Bergamo and ‘Orobicambiente’ association, a cyclic monitoring of the state of decay plays a primary role, using both 3D laser scanning and aerial photogrammetry by means of drones. The extension of the artefact to be surveyed, having a perimeter of around 5,200 meters with a maximum height of 22 m from the ground, for some aspects put the surveyor face to the problems typical of the urban or territorial scale. However, the scope of supporting the maintenance program draft request a ‘level of detail’ belonging more to the architectural survey. Indeed, in order to adopt an effective maintenance program for such an extended system, a first problem to be solved is the choice of an adequate surveying method, able to collect all the geometrical, material and deterioration information needed, but in the same time quick enough to allow for a repeated monitoring of the entire wall circuit. Therefore, the first phase of the survey campaign by UAV (Unmanned aerial vehicle), ending within spring 2019, had an experimental character in order to define the best procedure for programming, performing and representing the repeated survey. For speediness reasons, a complete restitution in 1:200 scale was chosen, comprising the whole perimeter of the walls, exept St. Peter’s bastion, not accessible for the moment. Only in selected parts of the wall circle, the survey was extent to a 1:50 detail, as a first step of a detailed survey to be repeated in the next years. Using drones in an urban environment required to act with proper caution, mainly because short distance shooting of nearly vertical surfaces was required, and performing zenithal shooting, flying at high altitude, was not sufficient. For this reason, a DJI Phantom IV drone was employed (kindly supplied by EcoGeo – Orobicambiente), so handy that the pilot could drive the camera at short distance, despite the obstacles that could be found on site.

1.3 Survey technology.

The phases of survey require the simultaneous presence of the UAV pilot and an assistant operator, who manages the positioning of high visibility paper targets, supports the pilot with indications on the flight and on the presence of possible people and obstacles, a frequent condition in the urban environment. The choice to acquire data by dynamic UAV flight with multiple horizontal passages at different altitudes (rather than from the ground with a camera on a tripod) has been more effective and faster because it provided the quality of the survey appropriate to established objectives. Furthermore, it allowed the 3D complex modelling even where it was impossible to make a traditional static survey downstream of the walls. The achievement of the expected accuracy, in addition to seeking maximum efficiency in carrying out both campaign and data processing operations, have been essential conditions in this process. The entire process comprise first the definition of a topographic support network, taking into account the subdivision of the survey and the photogrammetric modelling of the walls perimeter into various independent sectors, then the union of these parts based on the topographic references and at last the verification of the obtained results. This verification process will be fulfilled at the end of the entire model reconstruction. A portion of the photogrammetric model will be analysed comparing it with an high precision laser-

Figure 2. The city of Bergamo (the high town) at the beginning of 19th century (Manzini 1816).



46

scanner geometric model and with a more accurate photogrammetric model taken statically from the ground. This decision to conduct an aerial dynamic survey has involved the impossibility of capturing images with the HDRI technique. For this reason, the imperfections and the less readable parts of the obtained orthophoto are finally improved by a post-processing in order to obtain the correct layout, where information such as the dimensions, the metric scale and any further information are integrated together with some annotations for an easier reading and understanding.

2. RELATED WORKS

Digital images acquisition by means of drones for the survey of buildings or sites of historical relevance and related digital model development by structure from motion techniques gained more and more importance in the last years and different papers on the subject can be found in the literature. For the definition of adequate working methodologies, the studies by Mozaches-Calvache can be mentioned, who describes acquisition method for conventional photogrammetric procedures (Monzaches-Calvache et al. 2012), and more recently by Federman, who describes a procedure for data acquisition useful for Structure from Motion Techniques elaboration (Federman et al. 2017). Altena compares two types of UAV platform and related optical sensors characteristics (Altena and Goedemé 2014), and Gabrlik discuss direct geo referencing of pictures by drone’s sensors (Gabrlik 2015). Bolognesi make a comparison between traditional aerial photogrammetry and photogrammetry by drones and discuss their possible integration (Bolognesi et al. 2014); also Nikolakopoulos (Nikolakopoulos et al. 2016) propose a similar comparison. Lo Brutto describes results obtained by different UAV systems in the survey of an archaeological site and a land-art one (Lo Brutto et al. 2014). Stek discuss the potentiality of survey by drones for discovering archaeological remains under the soil surface (Stek, 2016). Barrile describes the adopted methodology for elaborating a 3D model of the Greek Walls of Reggio Calabria starting from the images acquired by drones (Barrile et al. 2017). Particularly relevant to the research on Bergamo city walls is the research made by Barazzetti, who discuss a working method for extracting Orthographic projections from pictures acquired by drones (Barazzetti et al. 2014).

3. DATA ACQUISITION CAMPAIGN AND RESULTS WORKFLOW

Considering the great extent of the site to be surveyed, one of the main scope of this first phase was to set up and test the procedure in order to define the right workflow that could be repeated in the next years supporting the maintenance program. On the basis of what already discussed above, the photogram-metric survey has been articulated in the following phases: 1. Flight plan definition: taking into account site

characte¬ristics, possible presence of obstacles, such as roads open to the vehicles or temporary working sites, and boundaries of different estate properties.

2. Planning and acquiring a topographic reference network: because previously used topographic network was not sufficient for the actual survey, a new one was set up.

3. Image capturing by drones. 4. Model Building: automatic image processing by means of

3DFlow Zephir software, with the following steps: 4.1. Image orientation: after the completion of automatics camera auto calibration, the first phase of the 3D reconstruction (SFM - ‘structure from motion’) will generate a sparse point cloud.

4.2. Dense point cloud extraction: second phase of the 3D reconstruction. 4.3. Geo-referencing and scaling of the model by sectors of flight. 4.4. Mesh building over the dense cloud. 4.5. Texture projection on the mesh. 4.6. Orthophoto images production, sector by sector

These are the step already tested and completed. Afterwards, some further passages are under execution and verification: 5. Building a tridimensional global model of the walls at a

territorial scale. 6. Mapping at macro-scale of main deterioration and damage

phenomena involving each sector of wall perimeter. 7. Detection of more damaged sectors and deepening of the

investigations in these sectors, with extractions of single portions of the walls in 1:50 scale.

8. Planning a survey program to be repeated by a three-year schedule.

3.1 Topographic reference network

First of all a terrestrial topographic network was established, consisting in total of 75 points, whose spatial geographic Gauss-Boaga and ETRF2000 coordinates were determined using static survey methods and the Real Time Kinetic survey with connection to the positioning service provided by the public network of the Lombardy-Piedmont Region (SPINGNSS) using the differential correction VRS-Nearest-DGPS. This technique allows reaching accuracies of a few centimetres at distances in the order of the kilometre. The noticed data provided a planimetric Square Mean Error (SQM) of less than 3 centimetres and altimetric SQM of 4 centimetres. The vertices were fixed through already existing stationary points, as well as with new metal studs anchored in the wall structure and highlighted in the survey phase by larger paper targets put above them (Figure 3).

3.2 Data acquisition

The first step of the photogrammetric survey consists in the acquisition of photographic images from UAV in collaboration with EcoGeo S.R.L. of Bergamo, and it was carried out subdividing the perimeter of the city walls in 20 freelance parts with enough overlapping to grant robustness to the model to be obtained. This stage evaluated the disposition, the necessary requisites for the access to the various properties, the potentials and the limits of the technique and the instrumentation (the

Figure 3. Topographic reference network around city walls.



47

duration of the batteries and the flight time, the presence of people and vegetation in areas of interest and the conditions for flying). The UAV is equipped with a gimbal with control range between 90° and +30°, allowing both zenithal and inclined shooting, fitting well the needs of surveying nearly vertical surfaces. The camera has a CMOS sensor 1/2,3” and 12 MPixel, with a lens 94° wide (20 mm equivalent in 35 mm format), following DJI specifications: the sensor is 6.17 mm wide, with focal distance of 3.53 mm. Therefore, assuming as G.S.D. (Ground Sampling Distance) a value of 0.01 m, the maximum acquisition distance is equal to 22.85 m. (Figure 4).

The opportunity of representing portions of the structure to a different scale has been evaluated. This will be possible if particular requirements in the survey phase are respected. Assuming a printer resolution of 300 DPI, the photographic acquisitions will have to guarantee a G.S.D. and a maximum acquisition distance lower than those associated with the 1: 200 scale. At the scale 1:100 corresponds a G.S.D. equal to 0.00847 m and a maximum acquisition distance equal to 19.35 m. At the 1:50 scale corresponds a G.S.D. equal to 0.00423333 m and a maximum acquisition distance equal to 9.67 m (Figure 5). At least five points per sector of flight, both on the top and on the feet of the walls, defined the reference at the ground of each part could be easily identified on the photographic views, in order to determinate the scale and the position of the model with better accuracy than that allowed by the UAV GPS. The survey phase was carried out by capturing images in JPG format at the highest quality allowed, acquired with UAV in motion at a constant and limited speed to avoid drag errors. The necessary balance between improvement of results and costs has produced the observation that the management of files in RAW format and multiple acquisitions at different exposures (HDRI technique) would have been unsustainable, because of the time required in that phase of the survey. Multiple horizontal steps have been carried out at close range (less than 22.85 m) with camera axis as normal as possible on the details in the best possible way, besides further steps at greater distance to capture the height of the entire wall curtain in its entirety. The UAV has been managed and piloted ‘in sight’ from the top or from the bottom of the walls, choosing from time to time the most effective and safe position in order to perform the approach manoeuvres of the aircraft to the surfaces and the vegetation (Figure 6). Although the provision was that the survey operations have been followed in chronological order the phases of vegetation removal from walls, the widespread presence of residual vegetation and

trees adjacent to the walls required a particular skill in getting around these obstacles, trying to detect and capture the most possible part of the artefact (Figure 7). Presently the most part of the wall perimeter survey has been already concluded, while we are still waiting for the access to the remaining portion in order to complete the survey.

Figure 4. Digital pictures acquisition with Phantom IV DJI.

Figure 5. Limit distance value for each image acquisition with GSD=0.01m.

Figure 6. UAV survey positions related to a typical vertical section of the walls.

Figure 7. A portion of city walls hidden by nearby vegetation.



48

3.3 Data processing

The model was developed using ©3D Flow Zephyr Aerial software: the model will be validated by a comparison of a laser-scanning survey of some selected portions. As a result, the orthographic projection of the whole perimeter of the walls will be given at 1:200 scale, which is supposed enough for a first general survey of the maintenance conditions of the wall complex perimeter. In this way, the principal issues for the conservation of the site are detected, selecting the parts that will undergo to a more accurate and detailed investigation to be repeated in time. The software ©3DFlow Zephyr Aerial has allowed to perform various workflows that lead from photographic images to the three-dimensional textured model of the surface of the walls. The overall three-dimensional model is the result of the composition of single minor traits. This turned out to be necessary by evaluating the time and hardware resources that are required during processing. In this way, each single part of the model was manageable from the computational point of view, although considerable processing time has been employed in order to obtain the requested level of detail. The single stretches have an extension generally lower than 250 m and are defined by the position of at least five topographical vertexes with known spatial coordinates arranged therein: this characteristic provides an adequate redundancy, useful for the correct spatial positioning of the model (Figure 8). The EXIF data of the pictures are the starting point for the preliminary orientation of the cameras (structure from motion) based on the automatic recognition of features present in each image; in this way a sparse points cloud was defined. Despite the choice of settings that privileged quality and precision at the expense of processing time, the average error that characterizes the control points of the model in this phase reaches a few tens of cm. However, this apparently superfluous procedure is useful in the next phase; actually, the definition of a cloud of scattered points speeds up the manual identification of the targets in the single images through the ‘hide far and occluded cameras’ function. After assigning the spatial coordinates of the topographic vertices, the model is aligned with the 3D constraints by choosing which points are reliable (that is free from accidental error) and the ‘projective star compensation’ is calculated by assigning the confidence weight of the constraints. This iterative procedure calculates the exact values of the internal and external parameters of the camera and produces a new cloud of scattered points characterized by errors that are much lower, in the order of a few millimetres (Table 1). The workflow continues in a typical way with the definition of the calculation settings, the processing and manual refinement of the dense point cloud, the meshed surface and the textured mesh by manually specifying which images are more appropriate to

consider in its fine definition avoiding backlight illumination effects. Each textured mesh of the models is exported by ©3DFlow Zephyr Aerial in Ply format or Obj/Mtl in combination with the PNG format and it is then imported into the overall model of the entire perimeter of the walls. This model is defined in its exact reference system and it is intended for implementation in an information system whose exact characteristics are currently being defined. For the purposes of drawing up the Wall Maintenance Plan, it is necessary to provide a metric representation of each individual model in its elementary parts. In particular, each bulwark, curtain, platform and gate is subdivided into linear or curvilinear elementary parts and is represented as orthographic images (orthophoto from textured mesh) projected to a vertical plane parallel to the mean alignment of the represented portion and free from perspective and scale errors (figure 9,10). Using a slightly laborious manual procedure with ©3DFlow Zephyr Aerial it is possible to rigorously define the vertical plane (with respect to the model reference system) passing through two distinct points. With reference to the plane that interpolates the

Figure 8. An example of UAV camera displacement for the embrasure of St. Michele bulwark. Control point Error [px] Error [m] N. of images

C 1.6910 0.0088 28 108 0.4794 0.0404 26 109 1.0980 0.0166 26 A 1.6292 0.0065 48 00 1.6294 0.0148 20 B 0.8068 0.0089 27

Table 1: Square Mean Error of control points

after projective star compensation

Figure 9. Portion of the walls model, with St. Michael’s bulwark, near to St. Augustine gate.



49

surface of the model, it is possible to define the dimensions of the output in addition to the pixel size (G.S.D.) on that plane and there is the possibility of inserting a grid of cross-pieces for the scale control.

4. CONCLUSIONS

At the end of the first data collection campaign, some considerations can be formulated about the survey process adopted and the first results obtained this way. Using UAV appear to be an efficient approach for the photogrammetric survey of such an extended site. The high cost, in terms of data elaboration time, is however more convenient compared to other more consolidated survey techniques, such as terrestrial photogrammetry or laser scanning. Anyway, operating by drones in an urban environment gives rise to many difficulties, making impossible a survey with higher detail that could be performed later by means of traditional methods, integrating the results in the survey already completed. More improvements can be achieved in the next survey campaign if boundary conditions will be more favourable (eliminating most of nearby vegetation, temporarily interrupting road traffic) and cameras with higher performances could be used. Another favourable condition for obtaining results that are more accurate could be the possibility of flying in dull weather conditions, but this requires a greater effort in maintaining a surveying team ready to intervene on request. Light conditions in fact have a great impact on the modelling results obtained by photogrammetric techniques. The obtained 3D model will anyway allow building a digital database where more information sets could be associated other than the geometric ones, such as maintenance conditions, the timing of interventions done, private/public properties data and so on.

ACKNOWLEDGEMENTS

The authors would like to acknowledge the Municipality of Bergamo, department of Environment, who supported the project and Orobicambiente Onlus, by means of EcoGeo srl, for providing the UAV image acquisition and collaborating to restitution. The founding of ‘Fondazione Cariplo’ in 2015 grants is gratefully acknowledged.

REFERENCES

[ASLAB] Ateneo di Scienze, Lettere ed Arti di Bergamo, 1990. 1588-1988. Le mura di Bergamo. in Atti dell'Ateneo di Scienze, Lettere ed Arti di Bergamo 49. Bergamo: Edizioni dell'Ateneo.

Altena, B. and Goedemé, T., 2014. Assessing UAV platform types and optical sensor specifications, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., II-5, 17-24. Barazzetti, L., Brumana, R., Oreni, D., Previtali, M., and Roncoroni, F., 2014. True-orthophoto generation from UAV images: Implementation of a combined photogrammetric and computer vision approach, ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., II-5, 57-63 Barrile, V., Gelsomino, V. and Bilotta, G., 2017. UAV and Computer Vision in 3D Modeling of Cultural Heritage in Southern Italy. IOP Conference Series: Materials Science and Engineering 225, 012196. Bolognesi, M., Furini, A., Russo, V., Pellegrinelli, A. and Russo, P., 2014. Accuracy of cultural heritage 3D models by RPAS and terrestrial photogrammetry, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-5, 113-119. Eisenbeiss, H. and Zhang, L., 2006. Comparison of DSMs generated from mini UAV imagery and terrestrial laser scanner in a cultural heritage application, ISPRS Archives, XXXVI-5, 90-96. Federman, A. et all,, 2017. UAV photogrammetric workflows: a best practice guideline, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-2/W5, 237-244. Gabrlik, P., 2015. The Use of Direct Georeferencing in Aerial Photogrammetry with Micro UAV, IFAC-PapersOnLine 48, 380-385. Lo Brutto, M., Garraffa, A., and Meli, P., 2014. UAV platforms for cultural heritage survey: first results, ISPRS Ann. Photo¬gramm. Remote Sens. Spatial Inf. Sci., II-5, 227-234. Mozas-Calvache, A.T., Pérez-García, J.L., Cardenal-Escarcena, F.J., Mata-Castro, E. and Delgado-García, J., 2012. Method for photogrammetric surveying of archaeological sites with light aerial platforms, Journal of Archaeological Science 39-2, 521-530. Nikolakopoulos, K., Soura, K., Koukouvelas, I., Argyropoulos, N., 2016. UAV vs. classical aerial photogram-metry for archaeological studies. Journal of Archaeological Science: Reports 14, 758-773.

Figure 10. The orthographic projections of the Venetian Walls: the wall on the left of St. Augustine gate.



50

uav photogrammetry for cultural heritage ......uav photogrammetry for cultural heritage preservation...

Documents