advanced classifiers

Advanced classifiers• Decision trees• Object-based classifiers• Others

DECISION TREES• Non-parametric approach• Data mining tool used in many applications,

not just RS• Classifies data by building rules based on

image values– Rules form trees that are multi-branched

with nodes and “leaves” or endpoints• Also referred to as classification trees

– Regression trees are similar but work for predicting continuous data

– Classification trees only predict thematic data (e.g. landcover class)

DECISION TREES

• Based on entropy and information gain

• Entropy represents homogeneity in the data; the less homogeneous the data, the more entropy you have in the system

• Try to reduce entropy through the clumping of data into groups. Reduced entropy = increase in information gain

DECISION TREES

• A tree is created by splitting the source set into subsets based on an attribute value test. – This process is repeated on each derived

subset in a recursive manner called recursive partitioning. The recursion is completed when the subset at a node has all the same value of the target variable, or when splitting no longer adds value to the predictions.

• This is a ‘top-down’ strategy known as top-down induction of decision trees (TDIDT).

Source: Wikipedia

DECISION TREES

DECISION TREES

• Widely used in the remote sensing community

• Evolved from ID3 and then C4.5• Developed by Russ Quinlan in Australia– Founded Rulequest Research company,

specializing in data mining tools• Now available via open source products like

R and Weka• Other decision trees are available, notably

Random Forests which is available in R

DECISION TREES – C5/See5

Dection tree vs. MLC

Change Detection in Sumatra

Knowledge-based classifiers

11

• Rules-based approach that uses conditional statements to test a series of hypotheses describing a set of informational classes; these conditions are usually supported by ancillary GIS data (image or vector-based)

12

Knowledge-Based Classifier

13

Knowledge-Based Classifier

WORKFLOW IN ERDAS KNOWLEDGE ENGINEER

• Traditional image classification methods only consider “color” (spectral signature) when classifying imagery. Object-oriented classification also considers shape, size, texture, pattern, shadow, and association

Object-Based Image Analysis

14

• Image pixels are treated as groups, or objects, not individually– Statistics and relationships are generated

and assigned for the entire object• Especially useful for high resolution imagery,

which contain a large amount of within-class spectral variability– Consider example of forest area where

trees and shadows are visible. Some pixels will be much darker because of the shadow, but still belong to the forest class.

15

Object-Based Image Analysis

• Segmentation is necessary to create the groups– Based on similar image characteristics,

as well as location. Neighboring pixels are joined.

• Segments can be created at multiple scales• Multiple datasets can be used to create the

segments

Segmentation

16

17Jensen, 2005, Plate 9-4

Segmentation

Level 25

Level 3

Segmentation

19

Image Objects

• Are attributed with spectral and spatial metrics: spectral mean, spectral variance, mean difference to neighbor, length to width ratio, area, shape index (smoothness), etc.

20

Object-based classification

• Image objects can be classified using statistical cluster analysis algorithms.

Object ID Band1_Mean Band2_Mean Band3_Mean Area Compactness …

1 123 23 45 47 0.3

2 201 34 200 88 0.65

3 34 135 136 300 0.2

…

21Jensen, 2005, Plate 9-5

22

Advantages of object-oriented classification

Most accurate method with high-resolution imagery Can be applied to both multi-spectral and single band data More consistent, efficient, and replicable than visual interpretation/digitizing Established methods, though expensive and “black box”

23

Object-oriented classification - example

24

Object-oriented classification - example

25

Object-based Classification Accuracy Assessment

• Traditional methods may not be appropriate when mapping a single feature; often used in combination with a “quantity difference” comparison (overestimation/underestimation of class compared with digitized data)

26

Feature Extraction

• Feature extraction is an object-oriented classification technique traditionally used to extract the key features (or objects) of a given image

27

Feature Extraction – common example

28

Feature Extraction

• Feature extraction software typically responds to training and feedback by the analyst in a process known as learning

29

Train Learner

Examine Results

Identify Correct,

Incorrect, and Missed

Run Process

Accept Results

30

Train Learner

Examine Results

Identify Correct,

Incorrect, and Missed

Run Process

Accept Results

learning

31

Feature Extraction

• Utilizes spatial context information as well as spectral information to determine results

32

Basic steps of feature extraction

1. Define the targeted features by digitizing training examples

2. Use spectral, spatial, and mathematical image information to generate results from the training set

3. Improve results using “learning” to select correct and incorrect examples from returned features

4. Examine final results and post-process to remove “clutter”

33

Other advanced classifiers: machine learning options

1. Support vector machines2. Neural networks

Highly sophisticated – generally used by more advanced image analysts

The End