desheng liu, maggi kelly and peng gong dept. of environmental science, policy & management
DESCRIPTION
Classifying Multi-temporal TM Imagery Using Markov Random Fields and Support Vector Machines. Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management University of California, Berkeley May 18, 2005. Outline. Introduction - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/1.jpg)
Desheng Liu, Maggi Kelly and Peng GongDept. of Environmental Science, Policy &
ManagementUniversity of California, Berkeley
May 18, 2005
Classifying Multi-temporal TM Classifying Multi-temporal TM Imagery Using Markov Random Imagery Using Markov Random
Fields and Support Vector Fields and Support Vector MachinesMachines
![Page 2: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/2.jpg)
OutlineOutlineOutlineOutline
I. Introduction1. Two aspects of Multi-temporal
Imagery 2. Classification Models
II. Methods1. Support Vector Machines2. Markov Random Fields3. Spatio-temporal Classification
III.Results IV.Conclusions
![Page 3: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/3.jpg)
Two Aspects of Multi-temporal Two Aspects of Multi-temporal ImageryImagery
Spatial Dependence– Pixels are not I.I.D.– Spatial
Autocorrelation
Temporal Correlation– Land Use– Phenology– Disturbance
Introduction
XX
YY
TT
![Page 4: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/4.jpg)
Classification ModelsClassification Models
Non-contextual Model
Contextual Models– Spatial
– Temporal
– Spatio-temporal
Introduction
*
( ), ( ( ))( ) { ( ( ) ( ) )}
Spatialc i Cd c N ic i Arg Max P c i i
*
( ), ( ( ))( ) { ( ( ) ( ) )}
Tempc i Cd c N ic i Arg Max P c i i
*
( ), ( ( )) ( ( ))( ) { ( ( ) ( ) , )}
Spatial Tempc i Cd c N i c N ic i Arg Max P c i i
*
( )( ) { ( ( ) ( ))}
c i Cdc i Arg Max P c i i : ; : c class label d spectral value
( ) : ; ( ) : Spatial Temporal
N i spatial neighbors of pixel i N i temporal neighbors of pixel i
![Page 5: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/5.jpg)
Generative Spatio-temporal Generative Spatio-temporal ModelsModels
Estimation of conditional probability– Maximum Likelihood Classifier (MLC)– Support Vector Machines (SVM)
Modeling spatio-temporal context – Markov Random Fields (MRF)
Introduction
, ( ( )) ( ( )) ( ( )) ( ( ))( ( ) ( ) , ) ( ( ) ( )) ( ( ) , ) spatial temp spatial temp
d c N i c N i d c N i c N iP c i i P c i i P c i
![Page 6: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/6.jpg)
SVM: A Graphic View SVM: A Graphic View (1)(1)
Linear Cases: find the optimal linear separating boundary with (a) maximum margin ρ (b) best trade-off between maximum margin ρ and minimum classification errors ξ ρ
Methods
ρ
ξj
ξi
(a) (b)
![Page 7: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/7.jpg)
SVM: A Graphic View SVM: A Graphic View (2)(2)
Non-Linear Cases: find the optimal linear separating boundary in a transformed higher dimensional feature space
Φ(x)
Methods
![Page 8: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/8.jpg)
Methods
SVM: A Mathematic SVM: A Mathematic View (1)View (1)
Training samples: Decision function: Discriminant function
– Linear cases:
– Nonlinear cases:
Probability output:
1
( , ) , 1,1
n
di i i i
ix y x R y
( ( ))y sign f x
( ) ,
i i ii SV
f x y x x b
( ) ( ), ( )
,
i i i
i SV
i i ii SV
f x y x x b
y K x x b
( )
1( 1 )
1 f x
p y xe
Binary Cases:Binary Cases:
![Page 9: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/9.jpg)
Methods
SVM: A Mathematic SVM: A Mathematic View (2)View (2)
Combination of binary SVM– “One-versus-one”– “One-versus-all”
Probability output– Pairwise coupling of binary probability
outputs– Soft-max function
Multi-category Cases:Multi-category Cases:
( )
( )
1
( )
i i i
i i i
f x
K f x
i
ep y i x
e
![Page 10: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/10.jpg)
Markov Random FieldsMarkov Random Fields
Markov Random FieldsMarkov Random Fields (MRF) ---(MRF) ---
Probabilistic image models which define the inter-Probabilistic image models which define the inter-pixel contextual information in terms of the pixel contextual information in terms of the conditional priorconditional prior probability of a pixel given its probability of a pixel given its neighboring pixelsneighboring pixels
Methods
1, if ( ) ( )( ( ), ( )) ; ( ( ) ( )) : ( ) ( )
0, if ( ) ( )
, : / Spatial Temp
c i c jI c i c j P c i c k transition probability of c k c i
c i c j
parameters to control the influnces of spatial temporal neighbors
( ( ))1( ( ) ( ( )) ( ( ))),
spatial temp
U c iP c i c N i c N i eZ
( ) ( )
( ( )) ( ( ), ( )) ( ( ) ( ))
spatial temp
Spatial Tempj N i k N i
U c i I c i c j P c i c k
: ; : Z normalizing constant U energy function
Time 1
Time 2
S1 S2 S3
S4 S5
S6 S7 S8
S1 S2 S3
S4 S5
S6 S7 S8
P
T1 T2 T3
T4 T5 T6
T7 T8 T9
T1 T2 T3
T4 T5 T6
T7 T8 T9
![Page 11: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/11.jpg)
Bayes’ Decision RuleBayes’ Decision Rule:: Maximum a posterior (MAP)Maximum a posterior (MAP)
MAP-MRFMAP-MRF: : the joint formulation of MAP and MRFthe joint formulation of MAP and MRF
MAP-MRF MAP-MRF
Methods
( ( ) ( ) , ( ( )) ( ( ))) ( ( ) ( )) ( ( ) ( ( )) ( ( )))
1
, ,
spatial temp spatial temp
spectral context
P c i d i c N i c N i P d i i P c i c N i c N i
U Ue
Z
c
( ) { ( ( ) ( ) , ( ( )) ( ( )))}( )
, spatial temp
c i Arg Max P c i d i c N i c N ic i C
ln( ( ( ) ( ))), ( ( )) spectral contextwhere U P d i i U U c ic
( ) { }( )
spectral contextc i Arg Min U U
c i CMAP-MRFMAP-MRF
![Page 12: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/12.jpg)
Spatio-temporal Spatio-temporal ClassificationClassification
Methods
( ) { ( ( ) ( ) , ( ( )) ( ( )))}( )
{ ( ( ) ( )) ( ( ) ( ( )) ( ( )))}( )
,
,
spatial temp
spatial temp
c i Arg Max P c i d i c N i c N ic i C
Arg Max P d i i P c i c N i c N ic i C
c
Conditional Probability Conditional Prior
Support Vector Machines Markov Random Fields
MAP-MRF
![Page 13: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/13.jpg)
( )1
( )2
( 1)( ) ( ) ( ) ( 1)
11 1 1 2
( 1)( ) ( ) ( ) ( 1)
22 2 2 1
1
2
1 ( )
2 ( )
ˆ( ( ) ( )) ( ( ) ( ( )), ( ( ))
ˆ( ( ) ( )) ( ( ) ( ( )), ( ( ))
ˆ ( ) { }
ˆ ( ) { }
spatial temp
spatial temp
t
t
tt t t t
tt t t t
c i C
c i C
P d i c i P c i c N i c N i
P d i c i P c i c N i c N i
c i Arg Max
c i Arg Max
Iterative Conditional Mode (ICM)Iterative Conditional Mode (ICM)
iteratively estimate the class label of each pixel given the iteratively estimate the class label of each pixel given the estimates of all its neighbors estimates of all its neighbors
Implementation Implementation AlgorithmAlgorithm
Methods
![Page 14: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/14.jpg)
TM Imagery of June 11, 1997
Results
Data and Study SiteData and Study SiteSan Bernardino National Forest,
CA
![Page 15: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/15.jpg)
Results
Data and Study SiteData and Study SiteSan Bernardino National Forest,
CATM Imagery of June
10, 2002
![Page 16: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/16.jpg)
Classification FlowClassification Flow
Results
TM Image
Convergence?
SVM
Conditional probability
Classification(intermediate)
MAP-MRF
SVM
Conditional probability
Classification(intermediate)
Convergence?
1997 2002
Yes Yes
No No
Initialization Initialization
MAP-MRF
TM Image
Classification(Final)
Classification(Final)
Fire Perimeter
![Page 17: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/17.jpg)
Training Sample Test Sample Class Name 1997 2002 1997 2002
Bare Land 1069 1363 356 454 Conifer 3162 2352 1054 784 Conifer Open 1355 2618 452 873 Hardwood 3443 2231 1148 744 Hardwood Open 1016 992 339 331 Herbaceous 503 417 168 139 Shrub 4153 2920 1384 973 Residential 1352 1119 451 373 Water 512 512 171 171
Results
Training/Test SamplesTraining/Test Samples
![Page 18: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/18.jpg)
Results
Classification Accuracies of TM Classification Accuracies of TM 19971997
50.00%
55.00%
60.00%
65.00%
70.00%
75.00%
80.00%
85.00%
90.00%
95.00%
Non-Context Spatial-Only Temporal-Only Spatio-Temporal
MLC
SVM
![Page 19: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/19.jpg)
Results
Classification Accuracies of TM Classification Accuracies of TM 20022002
50.00%
55.00%
60.00%
65.00%
70.00%
75.00%
80.00%
85.00%
90.00%
95.00%
Non-Context Spatial-Only Temporal-Only Spatio-Temporal
MLC
SVM
![Page 20: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/20.jpg)
Results
Convergence RateConvergence Rate
1 2 3 4 5 6 7 8 9 100
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Iteration
Ch
ang
e R
ate
(%)
MLC 97
MLC 02
SVM 97
SVM 02
![Page 21: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/21.jpg)
Results 19971997Original Image MLC MLC-Spatio-Temp
SVM SVM-Spatio-TempBare Land
Conifer
Conifer Open
Hardwood
Hardwood Open
Herbaceous
Shrub
Residential
Water
![Page 22: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/22.jpg)
Results 20022002Original Image MLC MLC-Spatial-Temp
SVM SVM-Spatial-TempBare Land
Conifer
Conifer Open
Hardwood
Hardwood Open
Herbaceous
Shrub
Residential
Water
![Page 23: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/23.jpg)
ConclusionsConclusions
SVM are much better in the processing of spectral data than MLC for the initialization of the iterative algorithm.
MRF are efficient probabilistic models for the analysis of spatial / temporal contextual information.
The combination of SVM and MRF unifies the strengths of two algorithms and leads to an improved integration of the spectral, spatial and temporal components of multi-temporal remote sensing imagery.
![Page 24: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/24.jpg)
AcknowledgementsAcknowledgements
USDA Forest Service
NASA Earth System Science Graduate Student Fellowship
![Page 25: Desheng Liu, Maggi Kelly and Peng Gong Dept. of Environmental Science, Policy & Management](https://reader036.vdocument.in/reader036/viewer/2022062323/56815911550346895dc63dec/html5/thumbnails/25.jpg)
Thank you!Thank you!