airborne high resolution remote sensing for near real time forest … · 2019-10-28 · airborne...

www.joanneum.at/digital

Airborne high resolution remote sensing for near real

time forest fire detection and surveillance

03rd - 05th October 2019

Rome, Italy

DI Alexander Almer

12th EARSeL Forest Fire Special

Interest Group Workshop

www.joanneum.at/digital

03rd - 05th October 2019

Rome, Italy

DI Alexander Almer

12th EARSeL Forest Fire Special

Interest Group Workshop

Airborne high resolution remote sensing for near real

time forest fire detection and surveillance

Topics

Motivation

Disaster management phases and time-critical processes

Platform concept & components

Data acquisition and geometric processing

Forest Fire Fighting Exercise - data examples

and analyse results

Conclusions

3

Motivation

System enhancements focusing on:

Portability and flexibility in carrier support.

Data quality and data fusion methods .

Result quality and information extraction.

Extended scenarios

Main changes

Target carrier platform

Support of a single airborne carrier platform

(Pilatus Porter PC6 Austria Army)

Sensory

30 MPx RGB

640x480 Px TIR camera

Geometry complexity

Simple handling of optical and thermal data

Result quality and information extraction.

Fields of application

Mainly large scale events

− Support of helicopters, ultra light planes and UAS

− Needed size and weight reduction down to

~10kg @ 70x25 cm

− Enhanced data quality / resolution

− Combined usage of one high resolution optical

camera and four thermal cameras

− Solve the enhanced complexity of geometric

matching of multiple synchronized sensors

− New sensor fusion methods for optical and thermal

sensor data

− Supporting small scale and large scale events

− Support high altitude flights

Disaster management phasesTime-critical process

6

• Prevention

• Event

• Coping / Management

• Regeneration

Not time-critical information products

Vegetation, biomass, moisture, permanent water masks, etc.

Risk analyses, hazard maps, weather forecast, forest fire index, etc.

Not time-critical information products

Comprehensive documentation for the support of management

processes in the "recovery phase"

Support for time-critical management tasks

Early detection of disaster situations (monitoring)

Near real-time generation of a COP on demand

Spread modeling (floods, forest fire)

Situation monitoring (water level, fire-watch, etc.)

➔Optimization of operational command and

decision-making processesTim

e c

riti

ca

lp

ha

se

s

AR

GU

S

OverallSystem Concept

Support a Multi-level-management concept

Different carrier platforms

Multi sensor solution

Data generation on demand (airborne & terrestrial sensors)

Broadband communication – data down and up link (flight and PoI information)

NRT data processing (geo-processing & data analysis) multimodal COP

Geo-oriented data management

Information distribution to support operations

Management of networked ressources

Firefighting helicopters and planes

Groundforces and vehicles

Impact analysis (feedback loop) Emergency Management Center

Multi-sensor shell components

150 MPx optical medium format camera

Resolution: 14204 x 10652 px (150 MPx)

Lenses: 35mm ➔ 50 mm 200mm

4 synchronised thermal cameras aligned across track

Resolution: 4x (640x480 px)

Small size GNSS/INS (synchronous position, attitude and navigation)

device

Multi-sensor shell concept

Compact shell

concept

Size: 70x25 cm

Weight: 10 kg

9

Stabilization motors for pitch & roll)

High resolution optical camera4 aligned thermal cameras

GPS / INS

Ethernet switch,

trigger/micro controller

and processing unit

ARGUS-Flex camera integration4 thermal cameras (640 x 480 pixel) to improve thermal resolution

10

RGB coverage(14204 x 10652 px )

Thermal coverage

ARGUS-Flex camera integration

4 thermal cameras (640 x 480 pixel)

to improve thermal resolution

Time-synchronous acquisition of

RGB image and 4 thermal images

11

RGB coverage Thermal coverage

4-layer stack of thermal and RGB image

Data acquisition and processing

Rapid (NRT) Mapping Baseline:

Calibrated camera system

Inner/relative orientation of RGB and thermal cameras

Boresight (exterior orientation) of RGB and thermal cameras

Time-synchronous data acquisition

Precision Mapping:

Enhanced data fusion to generate 4-layer image product:

Co-registration of RGB and thermal image based on image matching

Enhancement of geometric quality of thermal image (adequate to pan-sharpening)

Aerotriangulation, using tie-points and ground control points

12

Data processing less than 5‘‘

Coverage ~70-250km² / h(depending on the carrier platform)

Forest Fire ExerciseAllentsteig

13

Full fire: 12:04 max. 225o FH: 650m Full fire: 14:13 max. 60o FH: 900m

Types of fire:

Bush fire

Full fire

Wildfire

Forest Fire Exercise - AllentsteigTest areas

14

B1

B2

B3

Types of fire:

B1: Bush fire

B2: Full fire

B3: Wildfire

Forest Fire Exercise - AllentsteigBush fire

15

Flight 8:57:24 Flight 9:09:16

Forest Fire Exercise - AllentsteigFull fire

16

Flight 11:45:13 Flight 12:03:48

Forest Fire Exercise - AllentsteigWildfire

17

Flight 14:13:05 Flight 14:26:12

Forest Fire Exercise - AllentsteigFire out situation

Flight 16:46Flight 14:13

Forest Fire Exercise - AllentsteigFull fire analysis at 8 times

19

Flug 4 (16:46:51)

ConclusionsThe ARGUS-Flex concept supports different carrier platforms

The system enables to support early detection of fire sources, the generation of a NRT-COP and also the post fire monitoring phase

Depending on the carrier platform ~70-250km² / h can be covered

A first prototype of the new multi-sensor shell solution will be realised in the first half of 2020

Extended developments of methods for automated analysis of thermal & optical data to

classify different fire types to support the management and operation strategies

monitor and analyse the impact of the different firefighting operations

monitoring after the “fire out situation” (detection of glowing embers)

Thank you for your attention!

www.joanneum.at/digital

21

www.joanneum.at/digital

JOANNEUM RESEARCH Forschungsgesellschaft mbH

DIGITAL – Institute for Information and Communication Technologies

Steyrergasse 17, 8010 Graz

DI Alexander Almer

alexander.almer@joanneum.at

Tel.: +43 316 876 1738

Mobil: +43 664 602-876 1738

Thank you for your attention!

www.joanneum.at/digital

22

www.joanneum.at/digital

JOANNEUM RESEARCH Forschungsgesellschaft mbH

DIGITAL – Institute for Information and Communication Technologies

Steyrergasse 17, 8010 Graz

DI Alexander Almer

alexander.almer@joanneum.at

Tel.: +43 316 876 1738

Mobil: +43 664 602-876 1738

Ongoing Developments and Outlook

Research project: ARGUS Flex

Flexible, small and light high resolution multisensor platform

Phase One RGB medium format camera iXM 150

Resolution: 14204 x 10652 pxl (150 MPxl)

Lenses: 35, 80, 150 mm

4 thermal cameras syncronised

Resolution: 4x (640x512 pxl)

Carrier platforms

Planes and ultra light planes

Helicopters

ARGUS-Flex camera integration25

RGB coverage

Thermal coverage

Areal mapping

RGB 60%

Overlap – 3D

analysis

ARGUS-Flex camera integration26

RGB coverage

Thermal coverage

Flexible multi-sensor platform27

29 MP optical camera (RGB)10 cm @1000m AGL / coverage 720m @50mm

0,3 MPx thermal camera (moving)640 x 512 + swiping mirror technology

50 cm @1000m AGL / coverage 720m

Direct orthorectification / geo-sensorsL1/L2 GNSS with IMAR/FSAS IMU

Line of Sight (LoS) Bidirectional IP connection

Time Division Multiplexing (TDD)

Automatic tracking antenna

Cost efficient sensors, good accuracy in real time

Coverage ~100-200km² / h

On-board platform control

Support applications for

Pilot

Flight management and target

allocation

Operator

Sensor command & control

Data storage

Geo data managementData acquisition planning, execution & monitoring

Spatial- / temporal data storage and retrieval

Interactive geo-based visualization & analysis

Role & scenario based data distribution

C4S integration, GIS, Web, analogue maps

Interaction and task management with mobile units

Support the generation of COPs

29

Geo data processing

Near-real-time

ortho-rectification

Mosaicking

Rapid value-adding

Hotspot mapping

Change detection

3D model generation

Precision Mapping

30

Forest FireMulti-Level Management

32

Management topics

MonitoringPC 6 airborne carrier platform

ARGUS MS platform

Airborne operationsFirefighting planes and helicopters

Ground operationsFirefighting vehicles

Ground forces

Mobile Operation CenterManagement

Communication

Operational Command Support

33

ARGUSAirborne Real-Time

Ground Unit Support

ARGUS Fire

Situation picture (COP)

Resource management

Communication with mobile units

Networking of involved emergency forces and vehicles

Logging of communication and event/operation log

ARGUS Fire Mobile

Data exchange and communication module

for ground and air forces

ExamplesThermal Analysis

ExamplesThermal Analysis

Hotspot Position

14°53'06.79'' East

47°26'11.92'' North

1545 m Height

Hotspot

Temperature 259 o C

Ground 12,9 o C

Forest Fire Exercise - AllentsteigFire monitoring

36

Flug 3 (14:17:05)

Analyse PER

Graphik EB

ARGUS-Flex camera integration37

Flug 3 (14:17:05)Flug 4 (16:46:51)

International Expert-Workshop38

EU Program „Exchange of Expert in Civil Protection“

3-day workshop with participants from 12 countries

Real system demonstration

Discussion of strengths and weaknesses

Derivation of specific system requirements (add. requirements)

Different organisation structuresNational responsibilities

Involved organisations

Different strategies to support the management phasesEarly detection of fire sources

Event management support (COP, impact analysis, resources, etc.)

Post fire monitoring

System concept

Multi sensor platform

Real-time communicationto the ground

Data processing and -analysis

Geo-oriented management support applications

Ongoing Developments and Outlook

Research project: NRT-COP

Flexible light UAV based multisensor platform

Commercial RGB camera

Resolution: 42 MPxl

2 thermal cameras synchronised Resolution: 2x (640x512 pxl)

Precise geo-sensors (GPS/IMU)

Carrier platforms

Fixed wing UAVs

Multicopter UAVs

Payload > 10 kg

Duration > 45‘