image processing for tactical uav1 branche défense et sécurité s e 20855266-image processing for...

1

Branche DBranche Dééfensefense et et SSéécuritcuritéé

s

ee

20855266-

Image processing for tactical UAVImage processing for tactical UAVImage processing for tactical UAVM Broekaert, D Duclos

et M Sirieix

RTO SET SYMPOSIUM

Advanced Sensor Payloads for UAV

LISBON, April 2005

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE 01 MAY 2005

2. REPORT TYPE N/A

3. DATES COVERED -

4. TITLE AND SUBTITLE Image Processing for Tactical UAV

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Directeur Technique et Scientifique SAGEM SA Défense et Sécurité LePonant de Paris 27 rue Leblanc 75512 Paris Cedex 15 FRANCE

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited

13. SUPPLEMENTARY NOTES See also ADM202032., The original document contains color images.

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT

UU

18. NUMBEROF PAGES

37

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

2

BrancheBranche DDééfensefense et et SSéécuritcuritéé

s

ee

20855266-

OUTLINEOUTLINE

SAGEM Tactical UAVs

Imagery chain

3 rd generation IR imaging Payload

Image processing

Automatic target recognition

Target localization

Conclusions

3


s

ee

20855266-

SAGEM : EUROPEANSAGEM : EUROPEAN LEADER FOR UAVLEADER FOR UAV

BATTLE & FIELD BATTLE & FIELD -- PROVENPROVENMOSTMOST ORDEREDORDERED SYSTEM IN EUROPESYSTEM IN EUROPE

•• FRENCH ARMY (2+1+4 systems),FRENCH ARMY (2+1+4 systems),•• DUTCH ARMY (4 systems),DUTCH ARMY (4 systems),•• SWEDISH ARMY (3 systems),SWEDISH ARMY (3 systems),•• DANISH ARMY (6 systems),DANISH ARMY (6 systems),•• GREEK ARMY (3 systems)GREEK ARMY (3 systems)

ININ FULL RATEFULL RATE PRODUCTIONPRODUCTION

1993199319951995

1997199719991999

20012001 20022002

From Swedish Lapland Operation ...From Swedish Lapland Operation ...

…… to Saudi Arabia Operationto Saudi Arabia Operation

4


s

ee

20855266-

MAIN SPERWER SYSTEM CHARACTERISTICSMAIN SPERWER SYSTEM CHARACTERISTICS

THE MISSIONS ?Artillery (30m CEP)Intelligence

HIGHLY TACTICAL

•• Catapult launchCatapult launch

•• Parachute recoveryParachute recovery

THE SYSTEM Ground SegmentLauncherAVs

NATO INTEROPERABLE•• C4I integrated (C4I integrated (AdatAdat P3)P3)

•• KuKu--band data link (STANAG 7085)band data link (STANAG 7085)

•• Multiple payload possibleMultiple payload possible

•• Hand Over between ground stationsHand Over between ground stations

•• 2 aircraft simultaneously2 aircraft simultaneously

FLEXIBLE CAPABILITIES

5


s

ee

20855266-

Tactical UAV MissionTactical UAV Mission

Control Transfer

Launch and RecoveryStation

Station 1

Station 2Recovery

400 km

400 km

6


s

ee

20855266-

SPERWER - LONG ENDURANCESPERWER - LONG ENDURANCE

Endurance 12+ hoursEndurance 12+ hours

Stealthy (RCS, IR, acoustic)

Airworthiness

Robust delta design

Anti icing

Typical time to reach 2000m < 10 mn

Stealthy (RCS, IR, acoustic)

Airworthiness

Robust delta design

Anti icing

Typical time to reach 2000m < 10 mn

SPERWER - STANDARDSPERWER - STANDARD

MTOW 350 kg

Span 6.5 m

Speed 148 km/h

Ceiling 6000 m

Mission radius 200 km

SPERWER AIR VEHICLESSPERWER AIR VEHICLESCommon avionics

Common data link

Common avionics

Common data linkCommon ground segment

Common logistic support

Common ground segment

Common logistic support

MTOW 330 kg

Span 4.2 m

Speed 150 km/h

Ceiling 4500 m

Mission radius 200 km

Endurance 6+ hours

7


s

ee

20855266-

IR PAYLOAD MUST BE ADAPTED TO THE MISSION IR PAYLOAD MUST BE ADAPTED TO THE MISSION AND THE AIRCRAFT SPECIFICSAND THE AIRCRAFT SPECIFICS

IRLS 8 IRLS 8 -- 12 12 µµ

8 - 12 µ IRIS 2nd Generation & IR 3 - 5 µ MATIS 3rd Generation IR Camera

OLOSP & ARTEMIS

Stabilised gimbal

8


s

ee

20855266-

Payload system control and Payload system control and monitoringmonitoring Image analysisImage analysis

Payload Control Panel

TV SELECT

FOCUS∞ AV1

TV Control

VIDEO OFF

ACTIVE

IR SELECTE. ZOOM ENABLE INV POLAR

AV2

PLD ON

FOCUSZOOM / FOV

C-WTC-BK

SCEN

GAIN / LEVEL AUTO

Image Acquisition

LOS Control

COLOR

Overlay PCP Control

Tracker

Video Selection

PLD Mode Control

PLD ControlAV Request

STOW LOCKED

OPERATE RATE

FIX FORWARD

TEST LIGHT

Sensor Control

E

IR C ontrol

ORBIT CENTER

CONTRAST

GAIN

+

GAIN -

LEVEL

+

LEVEL -

TV ON IR ON AUTOTEST PLD

AUTOTRACK TOP POS SPEED

TRK ALGO TARGET

Tracker Control

Payload Control Panel

PO station

Mission Exploitation (Image analysis)

MP station

Maps display Payload

Video display

DATA BASE

9


s

ee

20855266-

MATIS 3MATIS 3--5 5 µµm CAMERAS m CAMERAS

BINOCULARS MANPORTABLE /BINOCULARS MANPORTABLE / LONG RANGELONG RANGESTANDARDSTANDARD

10


s

ee

20855266-

SELECTION CRITERIAS for IR PAYLOAD SELECTION CRITERIAS for IR PAYLOAD

Compactness to be fitted into a platform with other equipment’s (visible camera, Laser Range Finders/Laser Designator,..)Consistency with the mission and the environmentImage quality (high definition)Sensitivity (low NETD)Maturity of technology Affordable cost

11


s

ee

20855266-

ATMOSPHERIC TRANSMISSION ATMOSPHERIC TRANSMISSION Influence of the Relative HumidityInfluence of the Relative Humidity

Visibility : 25 kmTransmission 1 km,Altitude 0 mTemperature : 20°CPressure 1013 mbar; 1013 hPa

Visibility : 25 kmTransmission 10 km,Altitude 0 mTemperature : 20°CPressure 1013 mbar; 1013 hPa

12


s

ee

20855266-

DIFFRACTIONDIFFRACTION

Angular Resolution is Limited by Diffraction: α

= 1.22 λ

/ Ø

α

: angular resolution λ: wavelength Ø: pupil diameter

2 possibilities: Increase Ø : Ineffective as already limited by

the available space

Decrease λ : λ : move from 8-12 µm to 3-5 µm thermal imager, X2.5 benefit in term of resolution

13


s

ee

20855266-

Background Background limitedlimited NETD for a 2 D NETD for a 2 D ArrayArray

Nmax : Maximum storagecharge in ROIC capacitance (Nb of electrons)CΔT : Thermal contrast in the bandwidth

maxTmin N

2C

1NETDΔ

= ( )

( )λ∂

λ∂

=Δ ,TLT

,TL

CB

B

B

14


s

ee

20855266-

MATIS image MATIS image processingprocessing

Usual : non uniformity correction:Usual : non uniformity correction:Gain tableGain table

Shutter to show a uniform sceneShutter to show a uniform scene

Improvement : use of optical flow estimate Improvement : use of optical flow estimate from scene and camera movementfrom scene and camera movement

⇒⇒ Better image quality Better image quality

⇒⇒ New image processing capabilitiesNew image processing capabilities

15


s

ee

20855266-

Optical flowOptical flow

In a visible image sequence , the value of the intensity In a visible image sequence , the value of the intensity I(x,yI(x,y) at a point ) at a point x,yx,y of the image plane may fluctuate of the image plane may fluctuate because of two phenomena :because of two phenomena :

A movement of the scene objects or a movement of A movement of the scene objects or a movement of the camera,the camera,

An intensity variation of the scene objectsAn intensity variation of the scene objects

The optical flow detection consists in finding the The optical flow detection consists in finding the velocity field of the video sequence elements in the velocity field of the video sequence elements in the image plane.image plane.

16


s

ee

20855266-

Optical flowOptical flow

pixels movement 2D Fieldpixels movement 2D Field

Basic equation : Basic equation :

0dt t)y,I(x, t)y,dy(x,t)y,I(x, t)y,dx(x,t)y,I(x,=

∂∂

+∂

∂+

∂∂

tyx

17


s

ee

20855266-

Hypothesis : Hypothesis : The scene optical flow is principally linked to camera The scene optical flow is principally linked to camera

movementsmovementsIn the IR, intensity is a combination of incoming flux In the IR, intensity is a combination of incoming flux

(through the lens) and a parasitic flux ((through the lens) and a parasitic flux (eg narcisuseg narcisus). ). To simplify the equation, we can make two To simplify the equation, we can make two

observations concerning the optical flow of the IR observations concerning the optical flow of the IR camera :camera :

There is no fast movements for the parasitic There is no fast movements for the parasitic radiationradiationIntensity variations are linked to thermal Intensity variations are linked to thermal

phenomena, and consequently have time constants phenomena, and consequently have time constants that are broadly superior to the image periodicity.that are broadly superior to the image periodicity.

IR IR Optical Optical fflow : low : algorithm descriptionalgorithm description

18


s

ee

20855266-

IR IR Optical Optical fflow : low : algorithm descriptionalgorithm description

– Image pre processing : to avoid aliasing

– Gradient calculation

– Use of a predictive model to reduce the intensity variation

– Select N more convenient pixels

– We express movement in the focal plane as a function of 3 angles (roll, yaw and pitch), 3 translations and a zoom factor by a projection in the image plane of the movement model

– These expressions are injected in the Optical flow equation

– We solve the N equations by least mean square estimate

19


s

ee

20855266-

Application of optical flow processingApplication of optical flow processing

IR image without any correction

• the non-uniformity correction of the IR sensor• the stabilisation in the terrestrial co- ordinate system, or the filtering of the carrier vibrations• the extraction and the tracking of the mobile objects in the scene• the generation of hyper resolved images combining over sampling and restoration

The suggested processing permits to achieve successively or simultaneously the following functions :

20


s

ee

20855266-

Optical flow Optical flow => Calculate 3 angles camera rotation=> Calculate 3 angles camera rotation

algorithm descriptionalgorithm description

NUC 1 pointNUC 1 point

StabilisationStabilisation OverOver--samplingsampling

Mobile target detectionMobile target detection hyperhyper--resolutionresolution

21


s

ee

20855266-

Stabilisation of IR imageStabilisation of IR image

Stabilised and Referenced movie

Original movie Filtered and stabilised movie

22


s

ee

20855266-

IncreasedIncreased resolutionresolution with motionwith motion

Hyper-resolution

23


s

ee

20855266-

Image Processing ChainImage Processing ChainLow LevelProcessing

Hot spots detection

Edge extract.

Region segmentation

Texture analysis

High Level Processing

Correlation

Voting process

Bayes Class.

Fussy Logic

Fusion (s)

ANN

3D position estimation

Time Processing

Kalman filtering

Tracking / Association

Calibration -Restauration

Contrast Enhancement

Dead pixel correction

Wiener filter

Optical Flow Motion

AnalysisMoving Target Detection

Monocular Stereo

Calibration with OF

Stabilisation

Deblurring

Hyper- resolution

Sensor / Navigation Fusion

24


s

ee

20855266-

Automatic Target Detection, Recognition, Automatic Target Detection, Recognition, Identification and Localization Identification and Localization

Automatic generic target detection and recognition :Automatic generic target detection and recognition :vehicle type target : plane, tank, truck, vehicle type target : plane, tank, truck, ……

infrastructure target : bridge, building, infrastructure target : bridge, building, ……..

MultiMulti--bands, multibands, multi--contexts target identificationcontexts target identificationPrecise complex target localization helped with Precise complex target localization helped with

targettarget’’s area models area model

25


s

ee

20855266-

Generic Target Detection and Generic Target Detection and RecognitionRecognition

Detection

Best Alarm

Nav. Information

IMAGES

AlarmsSegmentation

AlarmsClassification

GenericTargetsModels

Alarms selection

Preliminary designation :Target type

Raw localization

UAV Flight

Target’s precise

localization

26


s

ee

20855266-

Generic Target Detection and RecognitionGeneric Target Detection and Recognition

« Airplane » target « Helicopter» target

27


s

ee

20855266-

« Tank » target « Truck » target


28


s

ee

20855266-


Low LevelFeatures Extraction

Alarms Selection

Selected3D Generic

Model

IMAGES

Projected 3D Model

Navigation Information

Research of Model’s Optimal Parameters

DETECTIONS

Alarms Notation and selection

3D GenericModels

29


s

ee

20855266-

« Bridge » target « Bulding » target


30


s

ee

20855266-

3D Models: geometry (facet) + radiometry: (emissivity, temperature)

Shapes for each position:distance/targetSynthetic Images

(Ray tracing,sensor

model)

Models Data Base

Target Segmentation

Image: distance to edge

Shape recognition

00,5

1

1,52

2,53

3,54

4,5

1 31 61 91 121 151 181 211 241 271 301 331 361 391 421 451 481 511

pré classe classe type

Notes and decisions

OFF-LINE

ON-LINE

Target IdentificationTarget Identification

31


s

ee

20855266-

Target IdentificationTarget IdentificationAMX 10MARDER

Input Image Identification result Input Image Identification result

32


s

ee

20855266-

TARGET LOCALIZATIONTARGET LOCALIZATIONPerformance Performance constantconstant and and independentindependent from GCS/AV from GCS/AV distancedistanceAutomaticAutomatic real timereal time calculation (no human interpretation)calculation (no human interpretation)Target coTarget co--ordinatesordinates available from video and still imagesavailable from video and still imagesRelative coRelative co--ordinates : ordinates : Accuracy 6 mAccuracy 6 m

1/2 angularcone

65°

10°

TargetLocation Error

< 50 m

< 20 m< 20 m

(CEP 50, altitude 1000 m)

65°

AccurateAccurate33--axes INSaxes INS

and attitude referenceand attitude reference

AV AV LocalisationLocalisation

Ground stationGround stationLocalisationLocalisation

NAVNAV

PLDPLDPositionPosition

LOSLOSOrientationOrientation

TargetTargetLocalisationLocalisation

Accurate LOSAccurate LOSposition measurementposition measurement

Payload / InertialPayload / InertialNavigation UnitNavigation Unit

boresightingboresighting

DGPSDGPS

GPS,GPS,Land NavigationLand Navigation

Systems, ...Systems, ...

DigitalDigitalTerrainTerrain

ElevationElevationDataData

33


s

ee

20855266-

Target localization improvement by image processingTarget localization improvement by image processing

In-flight processingTARGET

+ENVIRONMENT

Mission planning

Observation Means

Observation Means

InputimagesDTED

Automatic + manual offline modeling process

Target’s area model

Target’s area modelingTarget’s area modeling

Onboard IR sensor Onboard IR sensor

IR images

Low level image processing

Low level image processing

Extracted features

Automatic real-time modeling process

Localisation data

High level matching process

High level matching process

Precise Target localisation

34


s

ee

20855266-

ConclusionsConclusions

IR image sequence processing can offers new IR image sequence processing can offers new capabalitiescapabalities for target detection.for target detection.Motion is usable to make over sampling and provides Motion is usable to make over sampling and provides better resolution with 2D arrays by aliasing reduction.better resolution with 2D arrays by aliasing reduction.Improved target detection, recognition and Improved target detection, recognition and identificationidentificationPrecise target localisation by mixed processing Precise target localisation by mixed processing between navigation system, image between navigation system, image datasdatas and terrain and terrain modelling modelling

To day, large FPGA permits fast design of hardware To day, large FPGA permits fast design of hardware implementation from new algorithmsimplementation from new algorithms


TC X

XXX

s

ee

02/08/2000 3520855266-

Questions ?Questions ?Questions ?

R

UNCLASSIFIED/UNLIMITED

Image Processing for Tactical UAV

M. Broekaert, D. Duclos and M. Sirieix* * Directeur Technique et Scientifique

SAGEM SA Défense et Sécurité Le Ponant de Paris

27 rue Leblanc 75512 Paris Cedex 15

FRANCE

[email protected]

This paper was received as a PowerPoint presentation without supporting text.

Broekaert, M.; Duclos, D.; Sirieix, M. (2005) Image Processing for Tactical UAV. In Advanced Sensor Payloads for UAV (pp. 11-1 – 11-2). RTO Meeting Proceedings RTO-MP-SET-092, Paper 11. Neuilly-sur-Seine, France: RTO. Available from: http://www.rto.nato.int/abstracts.asp.

TO-MP-SET-092 11 - 1


http://www.rto.nato.int/abstracts.asp

Image Processing for Tactical UAV

11 - 2 RTO-MP-SET-092



image processing for tactical uav1 branche défense et sécurité s e 20855266-image processing for...

Documents