map asia 2003 m. İzzet saĞlam welcomewelcome classification of satellite images by using self...

Map Asia 2003Map Asia 2003

M. İzzet SAĞLAMM. İzzet SAĞLAM

WELCOMEWELCOMEWELCOMEWELCOME

CLASSIFICATION OF SATELLITE IMAGES BY USING SELF ORGANIZING MAP AND LINEAR

SUPPORT VECTOR MACHINE DECISION TREE (SOM-LSVMDT)

M. IZZET SAGLAMITU ADVANCED TECHNOLOGIES IN ENGINEERING

KUALA LUMPUR 2003



CONTENTSCONTENTSCONTENTSCONTENTS

IntroductionThe Classification Problem Problem Description Complexity Description

Linear Support Vector Machine Decision Tree (LSVMDT)The Proposed Algorithm (SOM-LSVMDT)Experimental Results and Conclusions



INTRODUCTIONINTRODUCTIONINTRODUCTIONINTRODUCTION

Origins of remote sensingHigh resolution imaging sensors are very important in modern remote sensing technology. When pattern recognition methods are applied to remote sensing problems, Smallness of the training data Complex statistical distribution of a large number of

classes




The main purpose of developing a special classifierA new support vector learning algorithmThe SOM-LSVMDT consists of Clustering part Binary tree structure with linear support vector

machines in all tree nodes.




The SOM-LSVMDT simplifies the model selection problem The SOM-LSVMDT has in-built properties for dealing with classes which can be considered as rare events. Reasons of occurrence of rare events. Natural reason SOM-LSVMDT’ structure

To solve rare event problem



THE CLASSIFICATION PROBLEMTHE CLASSIFICATION PROBLEM THE CLASSIFICATION PROBLEMTHE CLASSIFICATION PROBLEM

Ten-class remote sensing problem. Highly complex Highly nonlinear problem space

In addition, this work includes eight and thirteen class remote sensing problems of the same area.



11.. Problem DescriptionProblem Description11.. Problem DescriptionProblem Description

The Colorado data set consists of 7 data channels obtained from the following 4 data sources: Landsat MSS data (4 data channels) Elevation data (in 10m contour intervals, 1data

channel) Slope data (0-90 degrees in degree increments, 1

data channel) Aspect data (1-180 degrees in 1 degree increments,

1 data channel)



1. 1. Problem DescriptionProblem Description 1. 1. Problem DescriptionProblem Description

Table 1. Table 1. Number of Samples, and the Ground Cover Classes of 10 Class Colorado DataNumber of Samples, and the Ground Cover Classes of 10 Class Colorado Data

Class Field Testing Training

1 Water 195 408

2 Colorado blue spruce 24 88

3 Mountain/Subalpine meadow 42 45

4 Aspen 65 75

5 Ponderosa pine 139 105

6 Ponderosa pine/Douglas fir 188 126

7 Engelman spruce 70 224

8 Douglas fir/White fir 44 32

9 Douglas fir/ Ponderosa pine/Aspen 25 25

10 Douglas fir/White fir/Aspen 39 60



2.2. Complexity DescriptionComplexity Description2.2. Complexity DescriptionComplexity Description

Some of the classes are extremely under-represented. Class 9 Class 1, 5, 6, 7

Highly nonlinear separation of the classes



LSVMDTLSVMDTLSVMDTLSVMDT

LSVMDT includes binary tree structure with a linear SVM at each tree node (in the next Figure)All data vectors Negative (output of the linear SVM 0) Positive (output of the linear SVM > 0)

Terms Common class Rare class Class ratio



LSVMDTLSVMDTLSVMDTLSVMDT

An example of a binary tree structure of the LSVM-DT.An example of a binary tree structure of the LSVM-DT.



A. Training ProcedureA. Training ProcedureA. Training ProcedureA. Training Procedure

1. Train a linear SVM. After training is done, check a) lie on only one side of the decision hyperplane

i. still lie on only one side of the decision hyperplane ii. else, store the linear SVM at the node currently

pointed to by the pointer, and go to step 2

b) else, store the linear SVM at the node currently pointed to by the pointer, and go to step 2.

2. Separate all the vectors in the data set into two subsets



A. Training ProcedureA. Training ProcedureA. Training ProcedureA. Training Procedure

3. For all vectors lying in the negative side a) if all of these vectors belong to the -1 class b) if not, create a new left child of the current tree

node

4. For all vectors lying in the positive sidea) if all of these vectors belong to the +1 class b) if not, create a new right child of the current tree

node



B. Testing ProcedureB. Testing ProcedureB. Testing ProcedureB. Testing Procedure

1. Input the test vector to the linear SVM pointed to by the pointer

2. Check the output of the SVM a) if the output value is less than or equal to 0 b) if the output value is greater than 0

3. If the pointer points to a leaf node



THE PROPOSEDTHE PROPOSED ALGORITHM ALGORITHMTHE PROPOSEDTHE PROPOSED ALGORITHM ALGORITHM

The SOM-LSVMDT consists of Clustering part (in the following figure) Binary tree structure with a linear SVM at each tree

node (in the following figure)



THE PROPOSEDTHE PROPOSED ALGORITHM ALGORITHMTHE PROPOSEDTHE PROPOSED ALGORITHM ALGORITHM

An example of a binary tree structure of the An example of a binary tree structure of the SOM-SOM-LSVMDT.LSVMDT.



1. Clustering Part1. Clustering Part1. Clustering Part1. Clustering Part

The clustering part of SOM-LSVMDT Initialization

RandomSampleLinear

Training




The training procedure The first stage is the winner node search

The second stage is adaptation

minc ii

x m x m

c( ) ( ) i N( 1)

( ) otherwise

i ii

i

m t t x t m tm t

m t




Adaptation process in the clustering partAdaptation process in the clustering part



2. LSVMDT Part2. LSVMDT Part2. LSVMDT Part2. LSVMDT Part

In the secoond part of SOM-LSVMDT, each cluster generated by SOM is classifiedRare class problemTotal error of the proposed algorithm


M. İzzet SAĞLAMM. İzzet SAĞLAMEXPERIMENTAL RESULTS & EXPERIMENTAL RESULTS & CONCLUSIONSCONCLUSIONS

EXPERIMENTAL RESULTS & EXPERIMENTAL RESULTS & CONCLUSIONSCONCLUSIONS

SOM-LSVMDT can achieve better performance than linear support vector machine decision tree (LSVMDT). The eight, ten, and thirteen Colorado data sets are used to obtain experimental results. The number of samples of each data set is showed in the following table.




Table 2. Number of Samples each Colorado Data Set.Table 2. Number of Samples each Colorado Data Set.

8-Class Colorado 10-Class Colorado 13-Class Colorado

Training

1600 1188 1008

Test

3000 831 1011




The results are shown in the following table. Repeated for three times Colorado data sets are clustered, the statistical distribution of the classes changes The most important point




Table 3. Performance of the SOM-LSVMDT using the Colorado data sets.Table 3. Performance of the SOM-LSVMDT using the Colorado data sets.

LSVMDT SOM-LSVMDT II SOM-LSVMDT III SOM-LSVMDT IV

8 Class Colorado

%7,17 ERROR %3,4 ERROR %0,97 ERROR %1,90 ERROR

10 Class

Colorado %48,62 ERROR %35,02 ERROR %30,69 ERROR %35,98 ERROR

13 Class Colorado

%29,48 ERROR %22,64 ERROR %20,08 ERROR %18,40 ERROR


M. İzzet SAĞLAMM. İzzet SAĞLAMTHANK YOUTHANK YOUTHANK YOUTHANK YOU

THANK YOU

M. Izzet SAGLAM *

ITU INFORMATICS INSTITUEADVANCED TECHNOLOGIES IN ENGINEERING

* ITU SATELLITE COMMUNICATION AND REMOTE SENSING PROGRAM, Phd Student, INFORMATICS INSTITUE Research Assistant,

ISTANBUL

Prof. Dr. Bingul YAZGAN Prof. Dr. Okan ERSOY

[email protected]@yahoo.com

ITU SAGRES: +90 212 2856813

map asia 2003 m. İzzet saĞlam welcomewelcome classification of satellite images by using self...

Documents

map asia

zzet salam lsvmdtlsvmdt

data channel slide

adaptation slide

current tree node slide

linear svm

proposed algorithm slide

leaf node slide