Image analysis and A.I. for Earth Observation applications

From data to information

RMA/ Brussels 26_03_2019

remotesensing.vito.be

VITO RS in a nutshell

• VITO (Flemish Technological Research Institute)

• ~ 700 people

• Private but non-profit (shares in the hand of the Flemish government)

• Bridge between universities and industry (very applied research)

• 172 M€ turnover in 2017

• Research domains : Energy, Chemistry, Materials, Health and Environment

• Remote Sensing Department

• 85 people mostly working on (automated) image processing

• Archiving and Data Processing Center > 7 PBy

• More than 20 years operational satellite data processing

• More than 10 years operations with drones (environment, agriculture,

water, infrastructure, forestry and security)

remotesensing.vito.be

Satellite/airborn activities

• Data archiving and processing (geometric, radiometric corrections,…)

• Data fusion (SAR, Lidar, RGB, Thermal, Hyperspectral)

• Belgian Ground segment (e.g. Copernicus Sentinels satellites)

• Daily coverage of the whole world at 300/100 m resolution

• 20 years data

• S1 and S10 products

• Service for e.g. private company, farmer groups, FAO, …

• Satellite sensors (also cubsat)

• Specifications

• Design

• Performances assessment (modeling)

• PI of consortium with industrial partners

• Sensor calibration

remotesensing.vito.be

RS products and solutions

Sensors

Vegetation Agriculture Water Environment

& Security

Markets

Value Added Services

& Information

Products

UAV AIRBORN HALE UAV SATELLITE

Infrastructure

STANDARD IMAGE

PROCESSING

remotesensing.vito.be

From data to information

• Stitching (video or fixed frame)

• Image georeferencing => orthophoto

• 3D reconstruction and analysis

• Automated object detection (color, shape, …)

• Automated segmentation and classification

• Time series analysis

• Data fusion (e.g. RGB-LIDAR or SAR)

• Spectral signature (multi and hyperspectral sensors)

• Visualization and data management (Terrascope, Watchitgrow, Cropmap,…)

• Using the cloud of our own data center (7 Pby)

LIDAR

remotesensing.vito.be

Test performed live during the NATO

SRBIJA 2018 exercice (8-11/10/18)

Video (1 min.) = 750 MB

Orthophoto : 72 MB

Thanks to Mister W. Vanhamme

who provided the images

Medium resolution

Size = 10 MB

(factor 75 lower)

Degradated version

0.06 MB

(factor 12.500 lower)

USING DEEP

LEARNING ALGORITHMS

remotesensing.vito.be

Use of Artificial Intelligence (Neural Networks) for

image processing in Remote sensing applications

Convolutional Neural Networks (CNN) => Spatial information

• Diabetic retinopathy (not remote sensing)

• Cooling systems

• Asbestos

• Small canals

• Parcels

Recurrent Neural Networks (RNN) = > Temporal information

• Fraction of Absorbed Photosynthetically Active Radiation

(fAPAR) using fusion of S-1 and S-2 data

remotesensing.vito.be

Detection of Diabetic retinopathy based on

Retinal image analysis

Diabetic retinopathy (DR) is the leading cause of blindness in the working-age (20 – 74) population of

the developed world and is estimated to affect over 93 million people worldwide.

Accuracy 97% (allowing error of 1 class)

Automatic vessel

segmentation

Took 2 weeks to train the ‘high

resolution’ (512 x 512 pixels) models

Allowed us to check thousands of retina

pictures on DR for the Qatar Biobank

remotesensing.vito.be

Detection of industrial cooling systems

remotesensing.vito.be

Water based cooling systems are

potential sources for Legionella

Mandatory to register

Facts :

~ 300 registered,

~ 4.000 estimated (based on sales)

Control/detection happens currently with inspectors on terrain

remotesensing.vito.be

Examples of cooling towers (training)

Positive examples Negative examples

remotesensing.vito.be

Examples of cooling tower detection

remotesensing.vito.be

Examples of cooling tower detection

remotesensing.vito.be

Detection quite successful

Limited data set for training and validation (~ 300)

Large variety of systems

Detected ~3.000 installations

False positive were eliminated manually

Some unit not detected as not provided for the training set

remotesensing.vito.be

Automatic detection of corrugated sheets and

grey slates (potential sources of asbestos)

Idea :

• Use DSM to detect/segment roofs

• Detect Grey Slates (GS) and Corrugated Sheets (CS) using CNN

=> 2 training sets

• Compare results with cadastrial data (construction year)

• Identify places (X,Y) that could contain Asbestos

remotesensing.vito.be

Automatic detection of grey slates (GS)

remotesensing.vito.be

Automatic detection of corrugated sheets (CS)

remotesensing.vito.be

Detection sucessfull

• Very few false positie and/or false negative

• Trail performed for the region Mol and Mechelen

Mol (5 cm GSD): 95 % for CS and 85 % for GS

Mechelen (10 cm GSD): 90 % for CS - 80% for GS

• Now for all flanders

• Detection expected to be even better as training/validation

sets will be larger

https://blog.vito.be/remotesensing/deep-learning-keeps-your-feet-dry

remotesensing.vito.be

Detection of small canals in Flanders

• Small canals : l ~20 cm; H ~40 cm

• No digital atlas available for the small canals

• If some data, mostly position not correct (1 – 2 meters error

on X,Y coordinates)

• Altlas for the larger canals is also “old”

Idea :

=> use LIDAR data to detect automatically all canals

=> fuse with RGB data

remotesensing.vito.be

Using the hydrological atlas

remotesensing.vito.be

Small canals : l ~20 cm; H ~40 cm

DEM CNN output

remotesensing.vito.be

Automatic detection of small canals in Flanders

remotesensing.vito.be

Automatic detection of small canals in Flanders

remotesensing.vito.be

Main challenges

No real ground true available

Control of the results has to be done visually

LIDAR detect altitude: if canal full water no detection

Some false negative were observed for some slopes

Advantages :

Enhanced real positioning (1-2m error with manual GPS)

Make easier the work op operator (manual work) for new atlas

remotesensing.vito.be

Automatic delineation of agricultural parcels in

Belgium using Copernicus Sentinel-2 data

remotesensing.vito.be

CROPSAR : using Recurrent NN (temporal

evolution) for classification of crops

fAPAR can be deducted from Sentinel-2 multispectral data

S-2 revisiting time : 5 days

Problem : when cloudy => no data

As a result: only very limited amount of values for each parcel

Idea : enhance the timely information using Sentinel-1 SAR data

that offer a 6 days revisiting period

remotesensing.vito.be

Combining Sentinel-1 and Sentinel-2 data to beter

estimate the temporal evolution of the fAPAR

Multispectral

images 20 m GSD

SAR data

Sentinel-1

uninterrupted

Sentinel-2

interrupted

Sentinel-2 uninterrupted

Example potato

field monitoring

based on a deep neural network

CropSar: Optical – Radar fusion

remotesensing.vito.be

Better classification achieved

Field-based

fused S1-S2 time

series for all

agricultural

parcels of 2017

growing season

in Flanders

CropSar: Current test dataset:

Pre

dic

ted b

y f

usi

on

Observed

https://blog.vito.be/remotesensing/cropmap

THANK YOU

remotesensing.vito.be

Sentinel-2 image Copernicus Sentinel data (2016)

b l o g . v i t o . b e / r e m o t e s e n s i n g

Boeretang 2002400 Mol - Belgiummailadres@vito.be

r e m o t e s e n s i n g . v i t o . b e

SEETHEBIGGERPICTURE

NICOLAS

LEWYCKYJ

Project Manager

Security Applications

Contact :