pattern recognition 1: introduction€¦ · arne leijon pattern recognition 1: introduction....
TRANSCRIPT
ClassificationSummary
Pattern Recognition 1: Introduction
Arne Leijon
KTH Sound and Image Processing
Aug 27, 2012
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
A Pattern-Classification System
Feature
ExtractorClassifier
Transducer
Input
signal
Output
decision
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Source Categories
Sub-source #1
Sub-source #2
Sub-source #
€
Ns
Noise
Noise
Noise
Noise
Signal Source
Noise
S
Random State
Switch
Transducer
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Optimal Classification Mechanism
Select index
of largest
Discriminant Function
Discriminant Function
#1
#K
x
Feature Extraction
Classifier
†
g1 x( )
†
gNdx( )
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
First Classifier Example: Man or Woman?
150 160 170 180 190 20050
60
70
80
90
100
x1= Height / cm
x 2=
Weig
ht / kg
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Two Gaussian Feature Probability Density Functions
Women
150160
170180
190200
60
80
1000
1
2
3
4
5
x 10!3
x1= Height / cmx
2= Weight / kg
Men
150160
170180
190200
60
80
1000
1
2
3
4
5
x 10!3
x1= Height / cmx
2= Weight / kg
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Contours of Equal Probability Density
Women
x1= Height / cm
x 2=
Weig
ht / kg
150 160 170 180 190 20050
60
70
80
90
100
Men
x1= Height / cm
x 2=
Weig
ht / kg
150 160 170 180 190 20050
60
70
80
90
100
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Decision Boundary and Decision Regions
x1= Height / cm
x 2=
Weig
ht / kg
150 160 170 180 190 20050
60
70
80
90
100
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Gaussian Probability Density (2-dim)
x1
x2
!5 !4 !3 !2 !1 0 1 2 3 4 5!5
!4
!3
!2
!1
0
1
2
3
4
5
cov [X ] =
✓32 00 12
◆=
=
✓9 00 1
◆
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Gaussian Probability Density (2-dim)
x1
x2
!5 !4 !3 !2 !1 0 1 2 3 4 5!5
!4
!3
!2
!1
0
1
2
3
4
5
cov [X ] =
✓12 00 32
◆=
=
✓1 00 9
◆
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Gaussian Probability Density (2-dim)
x1
x2
!5 !4 !3 !2 !1 0 1 2 3 4 5!5
!4
!3
!2
!1
0
1
2
3
4
5
cov [X ] =P
✓32 00 12
◆P
T =
=
✓5 44 5
◆
where
P =1p2
✓1 11 �1
◆
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
Gaussian Probability Density (K -dim)
f
X
(x) =1
(2⇡)K/2pdetC
e
� 12(x�µ)TC�1(x�µ)
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
General Probability Density: Gaussian Mixture (1-dim)
!2 0 2 4 60
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
Pro
b. D
ensi
ty
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
General Probability Density: Gaussian Mixture (2-dim)
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
IntroExampleProbability Density
General Probability Density: Gaussian Mixture (K -dim)
f
X
(x) =MX
m=1
w
m
1
(2⇡)K/2pdetC
m
e
� 12(x�µ
m
)TC�1m
(x�µm
)
Arne Leijon Pattern Recognition 1: Introduction
ClassificationSummary
Summary: Important Concepts
Feature Vector: vector x containing all input data to the classifier
Observation Space: set of all possible feature-vector values
Decision Function: d(x) maps feature vector x into discretedecision output, coded as integer
Decision Region: set of feature-vector values giving same decision:⌦i
= {x : d(x) = i}Discriminant Function: g
i
(x) maps any feature vector x into realnumber, used for classification
General Optimal Classifier: d(x) = argmaxk
g
k
(x)
Two-category Classifier: can be implemented using a singleDiscriminant Function: d(x) = sgn g(x)
Gaussian Mixture Model: can approximate any distribution
Arne Leijon Pattern Recognition 1: Introduction