introduction of ship detection technology using sar and ... kazuo ouchi (p… · 09/07/2007 ·...
TRANSCRIPT
Introduction of Ship Detection Technology Using SAR and Trend of Ship Classification
Kazuo Ouchi Former Professor, National Defense Academy, Japan
Agenda
1. Purpose and need for ship detection and classification systems by synthetic aperture radar (SAR)
2. Ship detection by SAR: Algorithms and examples
3. Ship classification: Algorithms, examples, and current trend
4. Summary: Flow of ship monitoring system
First, Look at Optical Image at Night by SUOMI
Suomi (NOAA) 2011~
“Father of satellite meteorology”
Verner E. Suomi (1915 – 1995)
Sept. 2012
Night Time Light over Yellow Sea
Sept. 2012
Exclusive Economic Zone: EEZ
Korean EEZ Korean TZ (Transitional Zone) Chinese and Korean Prov. EEZs Chinese and Korean PMZ (Provisional Waters Zone) Chinese EEZ Chinese TZ (Transitional Zone)
Overlay of Night Time Light and EEZs
Korean EEZ Korean TZ (Transitional Zone) Chinese and Korean Prov. EEZs Chinese and Korean PMZ (Provisional Waters Zone) Chinese EEZ Chinese TZ (Transitional Zone)
Current Vessel Monitoring Systems
○ VMS : Vessel Monitoring System
○ LRIT : Long-Range Identification and Tracking of ships
○ VTS : Vessel Traffic Service
○ AIS : Automatic Identification System
Problem: Ships Without AIS/VMS/LRIT Signals
© JCG
Illegal oil release
Taiwan
Illegal oil release Poaching boat
Problem: Ships Without AIS/VMS/LRIT Signals
© JCG
Illegal oil release
Taiwan
Illegal oil release Poaching boat
Need for Vessel Monitoring Systems from Space with Imaging Sensors
Ship Detection and Identification with Imaging Sensors
Two types of airborne and satellite-borne imaging sensors:
1) Optical sensors
2) Imaging radars
Pleiades-1A: 50 cm resolution
cloud
Optical sensors are useful, but the problem is cloud cover !!!
Cloud Cover of the World during 2007 to 2009
Use Synthetic Aperture Radar (SAR) : All-Weather and Day-and-Night Data Acquisition Capability
Optical Image of the Tokyo Bay SAR Image of the Tokyo Bay
ALOS-Optical ALOS-PALSAR (L-band)
Simultaneously acquired optical and SAR images
Ship Detection by SAR
Principle: Radar Backscatter from Ships and Sea
TerraSAR-X Image of Gibraltar Strait
TSX StripMap HH 2007/07/09 06:29 UTC
Az.
Rg.
Selected Ship Detection Algorithms
1) CFAR (Constant False Alarm Rate) : Most frequently used algorithm. A threshold is set for a required false alarm rate (FAR) using a probably density function (PDF) of the background clutter. Resultant FAR is constant.
2) Adaptive Threshold : The value of target pixel is compared with the statistical values (mean, standard deviation) of background pixel. A threshold is variable and FAR is not constant.
3) Multi-Look / Sub-Look Detection : Uses the inter-look correlation property between moving windows in multi-/ sub-look images. Can detect images with low SNR.
4) Polarimetric Analyses : Polarimetric approach such as scattering power decomposition, eigenvalue analysis, inter-polarization correlation, etc. Not many datat by spaceborne SAR.
5) Many Others : Wavelet Transform, Notch filter, Standard deviation filter, Along-track InSAR, etc.
Selected Institutes and Ship Detection Algorithms
Institute System Name Algorithm
JRC / TNO (ESA / Netherland) SUMO/K CFAR / K-distribution
CCRS / MDA / RSI (Canada) OMW Modified CFAR / K-distribution
Veridian Syst. Div. (USA) --- Adaptive Threshold : IT = < I > + c σI
QinetiQ (UK) MaST Adaptive Threshold : IT = < I > + c σI
Kongsberg / FFI (Norway) MEOS Adaptive Threshold : IT = < I > + c σI
CLS (France) OMW CFAR
NOAA / NESDES (USA) AKDEMO CFAR
EMSA (EU) VDS CFAR
Edisoft / ISEL-Lisbon (Portugal) VDC softwear/SUMO Adaptive Threshold : IT = c (< I > + σI) / Sub-Look
MSS / MELCO (Japan) HuygensWorks Standard Deviation Filter
GMV (Spain) / IHI (Japan) SIMONS/ iOMS Wavelet/ Adaptive Threshold
eOsphere Limited (UK) R&G Median filter/Pauli Decomposition
Open University/IHI (UK/Japan) --- Sub-Look Entropy/Coherence
IT: Threshold intensity/amplitude, σI: Standard deviation, c: Constant
Example: Cell-Averaging CFAR A
mpl
itude
A
mpl
itude
Range or azimuth
Targets
Range or azimuth
Original signal, z
Clutter
Variance 分散値
After CFAR operation
Mean SAR Image
Moving window
Moving Window
Buffer Window (Not used)
Signal/Target Window
Background Window
After thresholding
FAR does not change even if σz changes
False alarm rate is constant !!!
Example: Adaptive Threshold Algorithm
Yes No
Target cell
Reference cells
Compare the value of the target cell and the mean + standard deviation
Reference cells
Example: Adaptive Threshold
Pixel value of target cell ≤ Threshold
No ship
Pixel value of target cell > Threshold
Ship
Target cell
Buffer window
Background window
Threshold = Constant x (Mean + Standard Deviation)
Example: Adaptive Threshold
Pixel value of target cell > Threshold
Ship
After threshold
False alarm rate is not constant !!!
Ship Detection with Polarimetric SAR Data
HH HV VV
Polarimetric SAR data: Current status
More information than single-polarization data Resolution of present polarimetric SAR is not fine compared with single- and dual-polarization SAR Resolution may be increased by digital-beam forming SAR in future
Ship Detection by 4-Component Scattering Power Decomposition
ALOS-PALSAR PLR 2008.10.09
HH VV
HV ★ Single-bounce surface scattering component
★ Double-bounce scattering component
★ Volume scattering component
★ Helix scattering component
Single-bounce Double-bounce Volume Helix
Ship Detection by 4-Component Scattering Power Decomposition
Single-bounce surface scattering component Double-bounce scattering component
Volume scattering power component
HH-VV Co-polarization Phase Difference
2012/09/20/ 08:28:10-12 (UT)
HH amplitude image HH-VV phase difference image
HH-VV Co-polarization Phase Difference
HH-VV phase difference image
Detection of Ship Wakes and Kelvin Waves
Ship wake
Ship
Ship wake V-shaped Kelvin waves
Smooth
Equilibrium rough surface
Water motion
Rough
Ship Wakes and Kelvin Waves: Gibraltar Strait
TSX StripMap HH 2007/07/09 06:29 UTC
Az.
Rg.
Ship Classification by SAR Passenger Ro-Ro Passenger Cargo
Battleships
LNG Tanker
Product Tanker
Battleship roles
Sushi
Fishing Boat
Pirates
Squid Fishing
SAR Images of Different Types of Ships: Tokyo Bay
VV
EMILIA
OCEAN SOUTH
NYK SILVIA
136 x 25 m
166 x 28 m
210 x 32 m
TSX SL 3.2 x 1.17 [m] (azimuth x slant-range)
Range range
azimuth
2008.12.06: TSX SL (Asc.) 3.2 x 1.17 (az x sr)
HH
187 x 23 m
TerraSAR-X over Tokyo Bay Azimuth
2011.05.30
CARIBE MAIDEN (General Cargo)
SAR Images of Different Types of Ships: Tokyo Bay
VV
EMILIA
OCEAN SOUTH
NYK SILVIA
136 x 25 m
166 x 28 m
210 x 32 m
TSX SL 3.2 x 1.17 [m] (azimuth x slant-range)
Range range
azimuth
2008.12.06: TSX SL (Asc.) 3.2 x 1.17 (az x sr)
HH
187 x 23 m
TerraSAR-X over Tokyo Bay Azimuth
2011.05.30
CARIBE MAIDEN (General Cargo)
Progress of ship classification is slow in comparison with detection, mainly because of resolution and complexity of imaging process.
Ship Classification Algorithms
As resolution is becoming finer in recent years, some algorithms have been emerging. The main approaches are as follows.
Feature Extraction
Feature-based Template Matching
Machine Learning
Multi-channel
Possible features include: • Length, width • Moments of inertia • Fractal measures • Wavelet coefficient • Target and shadow contour
• Image itself • Energy in brightest pixels • Polarimetric channel ratio • Attributed Scattering Center • Topographic Features
Pattern Matching SAR image
ISAR image
Feature Vector Method by GMV Spain: SIMONS
Reference parametric vectors by GRECOSAR ** simulator
* G. Margarit and A. Tabasco, “Ship classification in single-pol SAR images based on fuzzy logic,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 8, pp. 3129-3138, 2011. ** G. Margarit, J. J. Mallorqui, J. M. Rius, and C. Lopez-Martinez, “On the usage of GRECOSAR, an orbital polarimetric SAR simulator of complex targets, to vessel classification studies,” IEEE Trans. Geosci. Remote Sens., vol. 44, no. 12, pp. 3517-3525, 2006.
SIMONS (Ship Monitoring Service)*
SIMONS: Classification Results of ENVISAT-ASAR
* G. Margarit and A. Tabasco, “Ship classification in single-pol SAR images based on fuzzy logic,” IEEE Trans. Geosci.Remote Sens., vol. 49, no. 8, pp. 3129-3138, 2011.
ENVISAT-ASAR data: 30 m medium resolution
9 ship categories:
Oil tanker, Coaster, Bulk Carrier, Reefer, Cruise, Container, Carr Ferry, Medium fishing boat, Small fishing boat
Using 311 ships with AIS pols Classification accuracy ~ 70%
Important: Accuracy depends also on the number and types of categories !!!
SIMONS: Experiment in Tokyo Bay
NDA power-driven craft
S-band radar X-band radar
Ground-Truth Data Collection
AIS
Visual observation with a video camera
Yokohama
Tokyo Bay
Ground-truth data collection
NDA
TerraSAR-X Ground-Truth Data
2012/09/20/ 08:28:10-12 (UT) HH
Fumika
Shoei Maru
Tosei
Zenken Maru
Nippon Maru
Blue Ridge Highway
Hokuyo Maru Izumi Maru
Mie Maru
Vsg Pride
Chang Yu
Kyokuyo Maru Chang Yu
Mie Maru Kyokuyo Maru
Blue Ridge Highway Zenken Maru
Ysg Pride Izumi Maru
Hokuyo Maru
Shoei Maru Nippon Maru
JCG 1 JCG 1
No AIS 4
No AIS 1
No AIS 3
NDA
No AIS 7
Visible + No AIS
Visible + AIS
Ground-Truth by AIS + Visual
No AIS 4
Tosei
Fumika
No AIS 6
No AIS 5
No AIS 5
JCG 2
No AIS 2
Felicity Ace Felicity Ace
Yutoku Maru
Yutoku Maru No AIS 1
Name True Category: L/W [m]
Classified Category: L/W [m]
Photographs AIS + JCG
SAR image HH-polarization
Blue Ridge
Highway
Vehicles Carrier: 180/32
Class 3: Vehicles Carrier: 152.83/22.38
Zenken Maru
Waste Disposal Cargo: 98/18
Class 5: Coaster/LPG Tanker/Regional Cargo:
94.73/17.41
VSG-Pride
General Cargo: 91/15
Class 5: Coaster/LPG Tanker/Regional Cargo:
105.01/22.38
Izumi Maru
Vehicles Carrier: 129/20
Class 3: Vehicles Carrier: 116.81/9.95
Hokuyo Maru
Limestone Carrier: 123/20
Class 5: Coaster/LPG Tanker/Regional Cargo:
103.42/7.46
Chang Yu
Vehicles Carrier: 110/21
Class 5: Coaster/LPG Tanker/Regional Cargo:
93.61/22.32
Blue: Correct Red: Incorrect
SIMONS: Classification Results
37
Name True Category: L/W [m]
Classified Category: L/W [m]
Photographs AIS + JCG
SAR image HH-polarization
Mie Maru General Cargo: 75/13
Class 5: Coaster/LPG Tanker/Regional
Cargo: 59.64/7.46
Kyokyuyo Maru
Cargo: 61/9 Class 5: Coaster/LPG Tanker/Regional
Cargo: 64.08/9.95
JCG 1* Patrol Craft: 27/5.6 Class 6: Patrol/Small Ship: 24.22/14.92
JCG 2** Patrol Craft: 23/6 (Stationary)
Class 6: Patrol/Small Ship:: 14.94/9.95
* JCG 1 (Japan Coast Guard 1): Cruising with high speeds at 51 deg. from range ** JCG 2 (Japan Coast Guard 2): Stationary
Blue: Correct Red: Incorrect
SIMONS: Classification Results
Accuracy: ~ 87 % 38
Classified 16 ships + 5 ships in the TerraSAR-X image over the Melilla Bay in Mediterranean
Lockheed Martin Canada: Hierarchical Classification
Length [m]
Prob
abili
ty D
ensi
ty F
unct
ion
Classification accuracy:
Frigate 77% Destroyer 96% Cruiser 82% Battleship 62%
Aircraft Carrier 75%
Same principle as SIMONS of GMV Spain, but hierarchical neural net classifier with high-resolution airborne SAR: Many parametric vector components
Target Recognition of Vehicles: Deep Learning Theory
Wang et. al., “ Application of deep-learning algorithms to MSTAR data,” Proc. IEEE IGARSS 2015, pp. 3743-3745. Chen et. al., “Classification using the deep convolution networks for SAR images, IEEE Trans. Geosci. Remote Sens., vol. 54, pp. 4806-4817, 2016.
Target Recognition of Vehicles: Deep Learning Theory
Wang et. al., “ Application of deep-learning algorithms to MSTAR data,” Proc. IEEE IGARSS 2015, pp. 3743-3745. Chen et. al., “Classification using the deep convolution networks for SAR images, IEEE Trans. Geosci. Remote Sens., vol. 54, pp. 4806-4817, 2016.
Summary: Flow of Ship Monitoring System
1. Produce SAR image 2. Land/island masking 3. Apply ship detection algorithm
Image format is different depending on detection and classification methods
Mask out land, islands and other manmade objects from archive data or SAR image
CFAR: Constant False Alarm Rate ATM: Adaptive threshold Method MLCC: Multi-Look Cross- Correlation PolSAR: Polarimetric SAR
etc.
5. Check and change, if necessary, threshold
4. Apply wake detection algorithm
6. Comparison with AIS/ VMS data
Check if ship wakes are visible for improvement of detection accuracy
If the systems are not fully automatic, an operator checks ambiguity ghost images, false alarm images in rough seas under high sea states, others
Compare with AIS/VMS and/or maritime radar data and find non-AIS ships
Summary: Flow of Ship Monitoring System
7. Apply classification algorithms to non-AIS ships
Classify the ships’ category, and cruising speed, if possible
Summary: Flow of Ship Monitoring System
8. Reporting the results
Near real-time, if possible, is desirable
9. Action
Thank you for your attention !!!