1 using bayesian network for combining classifiers leonardo nogueira matos departamento de...
TRANSCRIPT
1
Using Bayesian Network for combining classifiers
Leonardo Nogueira MatosDepartamento de Computação
Universidade Federal de Sergipe
2
Agenda
Why combining classifiers?
Bayesian network principles
Bayesian network as an ensemble of classifiers
Experimental results
Future works and conclusions
3
Why combining classifiers?
Classifiers can colabore with each other
Minimizes computational effort for training
Maximizes global recognition rate
4
Why not to do so?
Because combining individual preditions can be so difficult as divising a robust single classifier
5
Why not to do so?
Because combining individual preditions can be so difficult as divising a robust single classifier
Decision
Classifiers
Combiner
6
Approaches for combining classifiers
L1. Data Level L3. Decision Level L2. Feature Level
Fixed rules Trainable rules
7
Approaches for combining classifiers
L1. Data Level L3. Decision Level L2. Feature Level
Fixed rules Trainable rules
8
Why not to do so?
Because combining individual preditions can be so difficult as divising a robust single classifier
Decision
Classifiers
Combiner
p(w|x)
p(w|x)
p(w|x)
9
Approaches for combining classifiers
L1. Data Level L3. Decision Level L2. Feature Level
Fixed rules Trainable rules
10
Existent scenarios
Pattern space
Pattern21
classifiers
classifiers
11
Our scenery
Pattern space
classifiers
12
A closed look
13
A closed look – discriminant function
14
A closed look – using multiple classifiers
15
A closed look – using multiple classifiers
The challegers:
How can we combine classifier's output?How can we identify regions in pattern space?
16
Agenda
Why combining classifiers?
Bayesian network principles
Bayesian network as an ensemble of classifiers
Experimental results
Future works and conclusions
17
Bayesian network principles
A
B C
Those circles represent binary random variables
18
Bayesian network principles
A
B C
Those circles represent binary random variables
a0a1
b0b1
c0c1
19
Bayesian network principles
A
B C
Those circles represent binary random variables
a0a1
b0b1
c0c1
a0 b0 c0
a1 b1 c1
⋮ ⋮ ⋮aN bN cN
dataset
20
Bayesian network principles
A
B C
Those circles represent binary random variables
a0a1
b0b1
c0c1
a0 b0 c0
a1 b1 c1
⋮ ⋮ ⋮aN bN cNinstance
21
Bayesian network principles
A
B C
Jointly probability inference is a combinatorial problem
P abc = P a P b∣a P c∣ab
2 possibilities
4 possibilities
22
Bayesian network principles
A
B C
Jointly probability inference is a combinatorial problem
P abc = P a P b∣a P c∣ab
P abc = P a P b∣a P c
Independence makes computation alittle more simple
23
Bayesian network principles
A
B C
Arest – indicates statistical dependence between variables
24
Bayesian network principles
A
B C
Arc – represents causality
25
Bayesian network principles
A
B C
A Bayesian network is a DAG (DirectAciclic Graph) where nodes representrandom variables and arcs representcausality relatioship
26
Bayesian network principles
A
B C
There are polinomial time algorithmsto compute inference in BN
27
Bayesian network principles
A
B C
There are polinomial time algorithmsto compute inference in BN
Evidence
28
Bayesian network principles
A
B C
There are polinomial time algorithmsto compute inference in BN
Evidence messages
29
Bayesian network principles
A
B C
There are polinomial time algorithmsto compute inference in BN
Evidence
[P a0∣bP a1∣b]
[P c0∣b P c1∣b ]
30
Agenda
Why combining classifiers?
Bayesian network principles
Bayesian network as an ensemble of classifiers
Experimental results
Future works and conclusions
31
A Fundamental Goal
32
Another insight
From a statistical point-of-view a Bayesian network is also a graphicalmodel to represents a complex and factored probability distribution function
33
Another insight
From a statistical point-of-view a Bayesian network is also a graphicalmodel to represents a complex and factored probability distribution function
34
Another insight
From a statistical point-of-view a Bayesian network is also a graphicalmodel to represents a complex and factored probability distribution function
The challegers:
How can we combine classifier's output?How can we identify regions in pattern space?
35
How can we combine classifier's output?
We use a BN as a graphical model of the pdf P(w|x)
We assume that classifier participate in computing that function
Each classifier must be a statistical classifier
36
How can we identify regions in pattern space?
37
Splitting pattern space
38
Defining a region
39
Patterns in a region
40
Algorithm
41
Bayesian Network Structure
42
Bayesian networks for combining classifiers
43
Agenda
Why combining classifiers?
Bayesian network principles
Bayesian network as an ensemble of classifiers
Experimental results
Future works and conclusions
44
Results with UCI databases
45
Results with NIST database
46
System I classifiers
47
Preliminaries
48
Results with the complete dataset
49
Agenda
Why combining classifiers?
Bayesian network principles
Bayesian network as an ensemble of classifiers
Experimental results
Future works and conclusions
50
Future works
51
Future works
52
Future works
53
Future works
54
Future works
Pattern space
Pattern21
classifiers
classifiers
55
ConclusionsWe have developed a method for combining classifiers using a Bayesian network
A BN act as trainable ensemble of statistical classifiers
The method is not suitable for small size dataset
Experimental results reveal a good performance with a large dataset
As a future work we intend to use a similar approach for splitting the feature vector and combine classifiers specialized on each piece of it.
