CRITICAL INFRASTRUCTURE MONITORING USING UAV IMAGERY

Evangelos Maltezosa, Michael Skitsasb, Elisavet Charalambousb, Nikolaos Koutrasb, Dimitris Bliziotisa and Kyriacos Themistocleousc

aGeosystems Hellas, Athens, Greece bADITESS Ltd, Nicosia, Cyprus

cCyprus University of Technology, Department of Civil Engineering and Geomatics, Limassol, Cyprus

International Conference on Remote Sensing and Geoinformation

of Environment, 4-8 April, 2016 - Paphos, Cyprus

2

The use of two rapidly evolving approaches, the Unmanned Aerial Vehicles (UAVs) and Dense Image Matching (DIM) techniques is an attractive solution to extract high quality photogrammetric products like 3D point clouds and orthoimages. The combination of the UAVs and DIM constitute a useful tool for several applications such as: Geometric documentation of a cultural heritage site Cadastre Urban studies 3D modeling, Change detection Emergency response Critical infrastructure monitoring

INTRODUCTION

3

The main advantages of the use of the UAVs are:

High navigation accuracy High-risk flight potentials Low operational cost Ability to integrate several type of sensors (e.g. hyperspectral, push room) Large scale images High overlapping of collected images

INTRODUCTION

The aforementioned characteristics offer applicability to a wide range of applications providing accuracy, economy, rapidity and automation during operation.

4

INTRODUCTIONIn the literature, interesting studies have been implemented that exploit the advantages of the UAVs. Concerning the emergency response applications: Development of a rapid aerial mapping system based on a UAV, whose key features are the effective acquisition of the sensory data, real-time transmission and processing of the data (Choi and Lee, 2011). Integration of geospatial video server with a UAS and fire brigades crisis management system, so that real-time geospatial airborne video and derived products can be made available at all levels during a fire incident (van Persie et al., 2011). Two scenarios were exercised, the first considered a car accident with multiple cars including a truck with chemicals while the second considered a fire in an industrial complex next to a railway. Development of procedures of using remote sensing by UAV to aid in the rapid evaluation and monitoring of environmental impacts caused by accidents involving transport of dangerous cargoes on highways (Longhitano and Quintanilha, 2013).

5

INTRODUCTIONConcerning the infrastructure inspection: Field-based investigation and inspection using mobile devices and base maps (Nakagawa et al., 2015). The base maps were generated by a UAV and a terrestrial laser scanner aiming at assisting investigators in infrastructure asset monitoring with location-based applications. Usage of a low-cost lightweight laser scanner onboard a UAV for an infrastructure project, and more specifically, a bridge (Mader et al., 2015). The 3D laser point cloud was compared with the corresponding DIM point cloud that extracted using the UAV imagery in conjunction with the Structure from Motion (SfM) technique which is utilized to obtain the image orientations. In this study, the benefits of the use of the UAVs in critical infrastructure monitoring applying DIM are highlighted extracting a 3D point cloud and orthomosaic for an Olympic venue in Attica, Greece.

6

Critical infrastructure protection is an important issue of European perspective in trying to identify a holistic framework for the protection against multiple types of risks. In order to ensure a high level protection of the designated European Critical Infrastructures, concrete actions are required to be established within a holistic protection framework. Besides technological and operational measures, the establishment of effective and endured partnerships between involved organizations that employ integrated, collaborative engagement and interaction between public – and private – sector partners act as vital components to these actions.

Modern tools should fuse Earth Observation data with in-situ data streams such as automated monitoring stations, radar, aircrafts, intelligence information and other sources for enhancing protection of critical infrastructures through monitoring and surveillance of critical components, risk mapping and management products.

CRITICAL INFRASTRUCTURES PROTECTION

7

A critical parameter for success is the introduction of the data, both primary and processed satellite images, risk and impact assessment into common format, and subsequently into a geographical (GIS) entity. This would allow for defragmentation and homogenization of data sets from different type of sources and at different geographical scales and elimination of the duplication of information collection.

Establishment of a high resolution 3-D mapping platform and development of relevant CAD and GIS records concerning security aspects of critical infrastructure plants and installations is being discussed. Information may be extracted from synthetic aperture radar (SAR) and electro-optical (EO) imagery, intelligence and surveillance of Critical infrastructure with particular emphasis to remote and isolated assets. Access to information will be protected with full security credentials.

CRITICAL INFRASTRUCTURES PROTECTION

8

STUDY WORKFLOW

Implementation:The ERDAS IMAGINE UAV software package provided by Geosystems Hellas was used on a workstation computer (CPU with dual Xeon E5 and 64Gb Memory) for the extraction of the DIM point cloud and the orthomosaic of the area of interest, in the premises of ADITESS LTD.

9

The application area is an Olympic venue that used during the Olympic Games in Athens (2004), at an area of 400 acres.

DESCRIPTION OF THE AREA OF INTEREST AND IMAGE COLLECTION PROCEDURES

10

• A Multicopter type UAV (owned by ADITESS LTD), the AP-M – S800 was used, equipped with a Panasonic DMC-GH4 camera and a 12 mm High Quality lens.

• The area of interest was covered by 585 overlapped NADIR-looking RGB digital aerial images with a ground sample distance (GSD) of 2 cm. Additionally 215 oblique images were acquired covering specific building blocks.

• The GPS/INS of the UAV data is used for the bundle adjustment. Additionally, Ground Control Points (GCPs) were used to improve the accuracy measured by Leica GPS station with accuracy better than 1 cm. The flight duration was 90 minutes and the height flight approximately 70 m.

DESCRIPTION OF THE AREA OF INTEREST AND IMAGE COLLECTION PROCEDURES

11

For a considerable period, the acquisition of a quality and accurate dense 3D point cloud of an area or of an object was mainly based on using Aerial Laser Scanning (ALS) or Terrestrial Laser Scanning (TLS) systems. Over the past years more and more researchers have developed robust matching cost functions and stereo matching algorithms used in Dense Image Matching (DIM).

DENSE IMAGE MATCHING

The DIM is an image matching process which can identify homologies for the total number of pixels of the reference image. The final product is a 3D point cloud of the area or object of interest.In stereo matching methods, a reference and a matching image are required at the minimum. To fully cover the area or the object of interest and therefore to eliminate occluded areas and mismatches, a Multi-View Stereo (MVS) approach is utilized.

12

Unlike laser systems requirements, the use of DIM technique does not require expensive hardware…

…However, DIM point clouds suffer from occlusions, complex scenes, radiometric differences, shadows, texture-less areas, repetitive pattern of objects, etc. To eliminate occlusions and mismatches as well as to achieve accurate and reliable results:

Overlaps of 80% along track and 60% across track (instead of the typical 60% forward and 20% sideward overlap of airborne image blocks) are used.

A variety of constraints and assumptions such epipolar geometry, left-right consistency, image brightness constancy, surface smoothness etc., are applied.

DENSE IMAGE MATCHING

13

More than 85% of the images, are subject to overlap with at least 9 other images increase of the redundancy compared to traditional photogrammetric blocks with one stereo-pair improvement of the accuracy of the image based 3D point cloud generation.

DENSE IMAGE MATCHING POINT CLOUD In this study, the DIM point cloud of the area of interest is extracted taking into consideration the previous mentioned concerns...

Camera locations and image overlap from UAV-flight

14

DENSE IMAGE MATCHING POINT CLOUD

DIM point cloud using the NADIR-lookingThe computational time for the extraction of a 13 million points

was 2.5 hours approximately, while the achieved accuracies were better than 4 cm.

15

DENSE IMAGE MATCHING POINT CLOUD

DIM point cloud using the oblique images

16

ORTHOMOSAIC

Orthomosaic of the area of interest using the DIM/digital surface model (DSM) with a GSD of 2 cm

17

ORTHOMOSAIC

Sub-region of the overall scene of the produced orthomosaic

18

ORTHOMOSAIC

Sub-region of the overall scene of the produced orthomosaic

19

This study leverages the benefits of the two increasing technological approaches of the UAV systems and DIM techniques for critical infrastructure monitoring and protection.The results are considered to be satisfactory and they demonstrate that accurate and reliable photogrammetric products such 3D point cloud and orthomosaic of the area of interest can be generated using a UAV+DIM approach. Surpassing the expectations of other methodologies that aim to find only “appropriate” points or discrete homologous points between the captured images, the DIM technique gives promising results even in demanding scenes such as critical infrastructures. …In this context, it seems that the DIM has the potential to become an alternative process, worthy of the laser scanning systems, which in conjunction with the UAVs can extract accurate and high quality dense 3D point clouds of an area of interest with lower cost production.

CONCLUSIONS

20

Thanks for your attention !