advanced diagnostics algorithms in online field device monitoring vagan terziyan (editor) ...
TRANSCRIPT
Advanced Diagnostics Algorithms in Online Field Device Monitoring
Vagan Terziyan (editor)
http://www.cs.jyu.fi/ai/Metso_Diagnostics.ppt
“Industrial Ontologies” Group: http://www.cs.jyu.fi/ai/OntoGroup/index.html
“Industrial Ontologies” Group, Agora Center, University of Jyväskylä, 2003
Contents
Introduction: OntoServ.NetOntoServ.Net – Global “Health-Care” Environment for Industrial Devices;
Bayesian MetanetworksBayesian Metanetworks for Context-Sensitive Industrial Diagnostics;
Temporal Industrial DiagnosticsTemporal Industrial Diagnostics with Uncertainty;
Dynamic IntegrationDynamic Integration of Classification Algorithms for Industrial Diagnostics;
Industrial Diagnostics with Real-Time Neuro-Real-Time Neuro-Fuzzy SystemsFuzzy Systems;
Conclusion.
Vagan Terziyan
Oleksiy Khriyenko
Oleksandr Kononenko
Andriy Zharko
Web Services for Smart DevicesWeb Services for Smart Devices
Smart industrial devices can be also Web Service “users”. Their embedded agents are able to monitor the state of appropriate device, to communicate and exchange data with another agents. There is a good reason to launch special Web Services for such smart industrial devices to provide necessary online condition monitoring, diagnostics, maintenance support, etc.
OntoServ.Net: “Semantic Web Enabled Network of Maintenance Services for Smart Devices”, Industrial Ontologies Group, Tekes Project Proposal, March 2003,
Global Network of Maintenance ServicesGlobal Network of Maintenance Services
OntoServ.Net: “Semantic Web Enabled Network of Maintenance Services for Smart Devices”, Industrial Ontologies Group, Tekes Project Proposal, March 2003,
Embedded Maintenance PlatformsEmbedded Maintenance Platforms
Service Agents
Host Agent
Embedded Platform
Based on the online diagnostics, a service agent, selected for the
specific emergency situation, moves to the embedded platform to help the host agent to
manage it and to carry out the predictive
maintenance activities
Maintenance Service
OntoServ.NetOntoServ.Net Challenges Challenges
New group of Web service users – smart industrial smart industrial devicesdevices.
InternalInternal (embedded) and externalexternal (Web-based) agent enabled service platformsservice platforms.
“Mobile Service ComponentMobile Service Component” concept supposes that any service component can move, be executed and learn at any platform from the Service Network, including service requestor side.
Semantic Peer-to-PeerSemantic Peer-to-Peer concept for service network management assumes ontology-based decentralized service network management.
Agents in Semantic WebAgents in Semantic Web
1. “I feel bad, pressure more than 200,
headache, … Who can advise what to do ? “
4. “Never had such experience. No
idea what to do”
3. “Wait a bit, I will give you some pills”
2. “ I think you should stop drink beer for a while “
Agents in Semantic Web supposed to understand each other because they will share common standard, platform, ontology and language
The Challenge: The Challenge: GGlobal lobal UUnderstanding nderstanding eeNNvironmentvironment ( (GUNGUN))
How to make entities from our physical world to understand
each other when necessary ?..
… Its elementary ! But not easy !! Just to make agents from them !!!
GUN ConceptGUN Concept
Entities will interoperate through OntoShells, which are “supplements” of these
entities up to Semantic Web
enabled agents
1. “I feel bad, temperature 40, pain in stomach, … Who can advise what to do ? “
2. “I have some pills for you”
Semantic Web: Before GUNSemantic Web: Before GUN
Semantic Web Resources
Semantic Web Applications
Semantic Web applications “understand”, (re)use, share, integrate, etc. Semantic Web
resources
GUN Concept:GUN Concept: All GUN resources “understand” each otherAll GUN resources “understand” each other
Real World objects
OntoAdapters
Real World Object ++ OntoAdapter +
+ OntoShell == GUN ResourceGUN Resource
GUNGUN
OntoShells
Real World objects of new generation (OntoAdapter inside)
Read Our Recent ReportsRead Our Recent Reports
Semantic Web: The Future Starts TodaySemantic Web: The Future Starts Today (collection of research papers and presentations of Industrial Ontologies
Group for the Period November 2002-April 2003)
Semantic Web and Peer-to-Peer: Semantic Web and Peer-to-Peer: Integration and Interoperability in IndustryIntegration and Interoperability in Industry
Semantic Web Enabled Web Services: Semantic Web Enabled Web Services: State-of-Art and ChallengesState-of-Art and Challenges
Distributed Mobile Web Services Based on Semantic Web: Distributed Mobile Web Services Based on Semantic Web: Distributed Industrial Product Maintenance SystemDistributed Industrial Product Maintenance System
Available online in: http://www.cs.jyu.fi/ai/OntoGroup/index.html
Industrial Ontologies GroupIndustrial Ontologies Group
V. Terziyan
A. Zharko
O. Kononenko
O. Khriyenko
Vagan Terziyan
Oleksandra Vitko
Example of Simple Bayesian Network
X
Y
P(X)
P(Y)-?
P(Y|X)
n
iiin XParentsXPXXXP
121 ))(|(),...,,(
)|()(),( ijiij xXyYPxXPxXyYP
i
ijij xXyYPxXPyYP )|()()(
)(
)|()()|(
j
ijiji yYP
xXyYPxXPyYxXP
Conditional (in)dependence rule
Joint probability rule
Marginalization rule
Bayesian rule
Contextual and Predictive Attributes
Machine
Environment
Sensors
XX x1 x2 x3 x4 x5 x6 x7
predictive attributes contextual attributes
air pressure
dust
humidity
temperature
emission
Contextual Effect on Conditional Probability
XX x1 x2 x3 x4 x5 x6 x7
predictive attributes contextual attributes
xk xr
Assume conditional dependence between predictive attributes
(causal relation between physical quantities)…
xt
… some contextual attribute may effect
directly the conditional dependence between
predictive attributes but not the attributes itself
Contextual Effect on Conditional Probability
X
Y
P(X)
P(Y)-? P(P(Y|X)|Z)
Z
P(Z) P(Y|X)
pk(Y|X)
P(P(Y|X))
•X ={x1, x2, …, xn} – predictive attribute with
n values;•Z ={z1, z2, …, zq} – contextual attribute with q
values;•P(Y|X) = {p1(Y|X), p2(Y|X), …, p r(Y|X)} –
conditional dependence attribute (random variable) between X and Y with r possible values;•P(P(Y|X)|Z) – conditional dependence between attribute Z and attribute P(Y|X);
})]|)|()|(()([
)()|({)(
1
1 1
q
mmkm
r
k
n
iiijkj
zZXYpXYPPzZP
xXPxXyYpyYP
Contextual Effect on Unconditional Probability
XX x1 x2 x3 x4 x5 x6 x7
predictive attributes contextual attributes
xk
Assume some predictive attribute is a random
variable with appropriate probability distribution
for its values…
xt
… some contextual attribute may effect
directly the probability distribution of the predictive attribute
x1 x2 x3x4
XX
P(X)P(X)
Contextual Effect on Unconditional Probability
X
Y
P(Y)-? P(P(X)|Z)
Z
P(Z)
P(X)
pk(X)
P(P(X))
P(Y|X)
X ={x1, x2, …, xn} – predictive attribute with n
values;
· Z ={z1, z2, …, zq} – contextual attribute with q values
and P(Z) – probability distribution for values of Z;
• P(X) = {p1(X), p2(X), …, pr(X)} – probability
distribution attribute for X (random variable) with r possible values (different possible probability distributions for X) and P(P(X)) is probability distribution for values of attribute P(X);
· P(Y|X) is a conditional probability distribution of Y given X;
· P(P(X)|Z) is a conditional probability distribution
for attribute P(X) given Z
})]|)()(()([
)()|({)(
1
1 1
q
mmkm
r
k
n
iikijj
zZXpXPPzZP
xXpxXyYPyYP
Bayesian Metanetworks for Advanced Diagnostics
3-level Bayesian Metanetwork forManaging Feature Relevance
X
Y
A
BQ
RSX
Y
A
B
Q
RS
2 -lev e l B ay esian M etan e tw o rk fo rm o d e llin g re lev an t fea tu res’ se lec tio n
C o n te x tu a l le ve l
P re d ic tiv e le v e l
Two-level Bayesian Metanetwork formanaging conditional dependencies
X
Y
A
BQ
RS
X
Y
A
B
Q
RS
T w o -lev e l B ay esian M etan e tw o rk fo rm an ag in g co n d itio n a l d ep en d en c ies
C o n te x tu a l le ve l
P re d ic tiv e le v e l
Terziyan V., Vitko O., Probabilistic Metanetworks for Intelligent Data Analysis, Artificial Intelligence, Donetsk Institute of Artificial Intelligence, Vol. 3, 2002, pp. 188-197.
Terziyan V., Vitko O., Bayesian Metanetwork for Modelling User Preferences in Mobile Environment, In: German Conference on Artificial Intelligence (KI-2003), Hamburg, Germany, September 15-18, 2003.
Two-level Bayesian Metanetwork for managing conditional dependencies
Contextual level
Predictive level A
B
X
Y
P(B|A) P(Y|X)
Causal Relation between Conditional Probabilities
xk xr
xm xn
P1(Xn|Xm)
P(XP(Xnn| X| Xmm))
P(P(XP(P(Xnn| X| Xmm))))
P2(Xn|Xm) P3(Xn|Xm)
P1(Xr|Xk)
P(XP(Xrr| X| Xkk))
P(P(XP(P(Xrr| X| Xkk))))
P2(Xr|Xk)
P(P(XP(P(Xrr| X| Xkk)|P(X)|P(Xnn| X| Xmm))))
There might be causal relationship between two pairs of
conditional probabilities
Example of Bayesian Metanetwork
The nodes of the 2nd-level network correspond to the conditional probabilities of the 1st-level network P(B|A) and P(Y|X). The arc in the 2nd-level network corresponds to the conditional probability P(P(Y|X)|P(B|A))
X
Y
P(X)
P(Y)-?
P(P(Y|X)|P(B|A))
A
B
P(A)
pr(B|A)
P(P(B|A)) P(B|A) P(Y|X)
pk(Y|X)
P(P(Y|X))
))]}.|()|(())|()|((|)|()|(([
)()|({)(
ABpABPPXYpABPPXYpXYPP
xXPxXyYpyYP
rr
rk
i kiijkj
Other Cases of Bayesian Metanetwork (1)
P(A) P(X)
X
A
Contextual level
Predictive level
a)
P(P(X)|P(A))
A
pr(A)
P(P(A))
P(A) X
pk(X)
P(P(X))
P(X)
b)
Unconditional probability distributions associated with nodes of the predictive level network depend on probability distributions associated with nodes of the contextual level network
Other Cases of Bayesian Metanetwork (2)
The metanetwork on the contextual level models conditional dependence particularly between unconditional and conditional probabilities of the predictive level
P(A) P(Y|X)
X A
Contextual level
Predictive level
Y
c)
X
Y
P(X)
P(Y)-? P(P(Y|X)|P(A))
A
pr(A)
P(P(A))
P(A) P(Y|X)
pk(Y|X)
P(P(Y|X))
d)
Other Cases of Bayesian Metanetwork (3)
The combination of cases 1 and 2
P(A)
P(Y|X)
X A
Contextual level
Predictive level
Y
P(B)
B
e)
X
Y
P(X)
P(Y)-?
P(P(Y|X)|P(A))
A
pr(A)
P(P(A))
P(A)
P(Y|X)
pk(Y|X)
P(P(Y|X))
B
ps(B) P(P(B))
P(B)
P(P(A)|P(B))
f)
Contextual level
Predictive level
2-level RelevanceRelevance Bayesian Metanetwork (for modelling relevant features’ selection)
Simple Relevance Bayesian MetanetworkWe consider relevance as a probability of importance of the variable to the inference of target attribute in the given context. In such definition relevance inherits all properties of a probability.
X
Y
Probability
P(X)
P(Y)-? P(Y|X)
Relevance
Ψ(X)
X
Y
P(X)
P(Y|X)
Probability to have this model is:
P((X)=”yes”)= X
Y
P0(Y) Probability to have this model is:
P((X)=”no”)= 1-X
.)]1()([)|(1
)( X
XX XPnxXYPnx
YP
Example of 2-level Relevance Bayesian Metanetwork
In a relevance network the relevancies are considered as random variables between which the conditional dependencies can be learned.
X
Y
P(X)
P(Y)-?
P(Y|X) P(Ψ(X)|Ψ(A))
A
P(A) Ψ(A) Ψ(X)
)]}.1()()|()([)|({1
)( XAAXX A
PPXPnxXYPnx
YP
More Complicated Case of Managing Relevance (1)
X
Y
Probability
P(X)
P(Y)-?
P(Y|X,Z)
Relevance
Ψ(X)
Z
Probability
P(Z) Relevance
Ψ(Z)
X
Y
Probability
P(X)
P(Y|X,Z)
Z
Probability
P(Z)
Probability of this case is equal to:
P((X)=”yes”)×P((Z)=”yes”) = = X·Z
11
X
Y
Probability
P(X)
P(Y|X)
Probability of this case is equal to:
P((X)=”yes”)×P((Z)=”no”) = = X·(1-Z)
Y
P(Y|Z)
Z
Probability
P(Z)
Probability of this case is equal to:
P((X)=”no”)×P((Z)=”yes”) = = (1-X)·Z
Y
Probability
P0(Y)
Probability of this case is equal to:
P((X)=”no”)×P((Z)=”no”) = = (1-X)·(1-Z)
22 33 44
More Complicated Case of Managing Relevance (2)
X
Y
Probability
P(X)
P(Y)-?
P(Y|X,Z)
Relevance
Ψ(X)
Z
Probability
P(Z) Relevance
Ψ(Z)
,),|(1
)1()1(
)(),|(1
)1(
)(),|(1
)1(
)()(),|()(
1
1
1
1
nx
i
nz
ikkiZX
nx
i
nz
ikkkiZX
nx
i
nz
ikikiZX
nx
i
nz
ikkikiZX
zZxXYPnznx
zZPzZxXYPnx
xXPzZxXYPnz
zZPxXPzZxXYPYP
General Case of Managing Relevance (1)
X1
Y
Probability
P(X1)
P(Y)-?
P(Y|X1,X2,…,XN)
Relevance
Ψ(X1)
XN
Probability
P(XN) Relevance
Ψ(XN) X2
Probability
P(X2) Relevance
Ψ(X2)
…
Predictive attributes:
X1 with values {x11,x12,…,x1nx1};
X2 with values {x21,x22,…,x2nx2};
…XN with values {xn1,xn2,…,xnnxn};
Target attribute:
Y with values {y1,y2,…,yny}.
Probabilities:
P(X1), P(X2),…, P(XN);P(Y|X1,X2,…,XN).
Relevancies:X1 = P((X1) = “yes”);
X2 = P((X2) = “yes”);
…XN = P((XN) = “yes”);
Goal: to estimate P(Y).
General Case of Managing Relevance (2)
X1
Y
Probability
P(X1)
P(Y)-?
P(Y|X1,X2,…,XN)
Relevance
Ψ(X1)
XN
Probability
P(XN) Relevance
Ψ(XN) X2
Probability
P(X2) Relevance
Ψ(X2)
…
1 2 )"")(()"")((
1
])1()(),...2,1|([...1
)(X X XN noXqq
XqyesXrr
XrN
s
XrPnxrXNXXYPnxs
YP
Example of Relevance Metanetwork
X
Y
A
BQ
RS
a)
X
Y
A
B
Q
RS
b)c)
Relevance level
Predictive level
Combined Bayesian Metanetwork
In a combined Metanetwork two controlling
(contextual) levels will effect the basic level
Contextual level A
Predictive level
Contextual level B
Learning Bayesian Metanetworks from Data
Learning Bayesian Metanetwork structure (conditional, contextual and relevance (in)dependencies at each level);
Learning Bayesian Metanetwork parameters (conditional and unconditional probabilities and relevancies at each level).
Vitko O., Multilevel Probabilistic Networks for Modelling Complex Information Systems under Uncertainty, Ph.D. Thesis, Kharkov National University of Radioelectronics, June 2003. Supervisor: Terziyan V.
When Bayesian Metanetworks ?
1. Bayesian Metanetwork can be considered as very powerful tool in cases where structure (or strengths) of causal relationships between observed parameters of an object essentially depends on context (e.g. external environment parameters);
2. Also it can be considered as a useful model for such an object, which diagnosis depends on different set of observed parameters depending on the context.
Vagan Terziyan
Vladimir Ryabov
Temporal Diagnostics of Field Devices
• The approach to temporal diagnostics uses the algebra of uncertain temporal relations*.
• Uncertain temporal relations are formalized using probabilistic representation.
• Relational networks are composed of uncertain relations between some events (set of symptoms)
• A number of relational networks can be combined into a temporal scenario describing some particular course of events (diagnosis).
• In future, a newly composed relational network can be compared with existing temporal scenarios, and the probabilities of belonging to each particular scenario are derived.
* Ryabov V., Puuronen S., Terziyan V., Representation and Reasoning with Uncertain Temporal Relations, In: A. Kumar and I. Russel (Eds.), Proceedings of the Twelfth International Florida AI Research Society Conference - FLAIRS-99, AAAI Press, California, 1999, pp. 449-453.
Conceptual Schema for Temporal Diagnostics
N
S1 S2 … Sn
Temporal scenarios
1,SND2,SND
nSND ,
Recognition of temporal scenarios
• We estimate the probability of belonging of the particular relational network to known temporal scenarios.
Generating temporal scenarios
• We compose a temporal scenario combining a number of relational networks consisting of the same set of symptoms and possibly different temporal relations between them.
N1
N2
N3
N4N5
S
Terziyan V., Ryabov V., Abstract Diagnostics Based on Uncertain Temporal Scenarios, International Conference on Computational Intelligence for Modelling Control and Automation CIMCA’2003, Vienna, Austria, 12-14 February 2003, 6 pp.
Industrial Temporal Diagnostics (conceptual schema)
Industrial object
Temporal data
Relational network
DB ofscenarios
Estimation Recognition Diagnosis
Learning
Ryabov V., Terziyan V., Industrial Diagnostics Using Algebra of Uncertain Temporal Relations, IASTED International Conference on Artificial Intelligence and Applications, Innsbruck, Austria, 10-13 February 2003, 6 pp.
Event 2
< a1; a2; a3 > - imperfect temporal relation
between temporal points (Event 1 and Event 2):
P(event 1, before, event 2) = a1;
P(event 1, same time, event 2) = a2;
P(event 1, after, event 2) = a3.
Event 1
< a1; a2; a3 >
Imperfect Relation Between Temporal Point Events: Definition
Ryabov V., Handling Imperfect Temporal Relations, Ph.D. Thesis, University of Jyvaskyla, December 2002. Supervisors: Puuronen S., Terziyan V.
Example of Imperfect Relation
Event 2
< 0.5; 0.2; 0.3 > - imperfect temporal relation between temporal points:
P(event 1, before, event 2) = 0.5;
P(event 1, same time, event 2) = 0.2;
P(event 1, after, event 2) = 0.3.
Event 1
< 0.5; 0.2; 0.3 >
1
<= >
R(Event 1,Event 2)
Operations for Reasoning with Temporal Relations
rb,a = bar,~
ra,b
a b
ra,b rb,c
ra,c = ra,b rb,c
a
b
c
r r ra b a b a b, , , 1 2
r 1 a , br 2 a , b
a b
Inversion
Sum
Composition
Temporal Interval Relations
The basic interval relations are the thirteen Allen’s relations:
A before (b) B B after (bi) A
A meets (m) B B met-by (mi) A
A overlaps (o) B B overlapped-by (oi) A
A starts (s) B B started-by (si) A
A during (d) B B contains (di) A
A finishes (f) B B finished-by (fi) A
A equals (eq) B B equals A
A B
AB
AB
BA
AB
AB
BA
Imperfect Relation Between Temporal Intervals: Definition
interval 2
< a1; a2;… ; a13 > - imperfect temporal relation between
temporal intervals (interval 1 and interval 2):
P(interval 1, before, interval 2) = a1;
P(interval , meets, interval 2) = a2;
P(interval 1, overlaps, interval 2) = a3;
…
P(interval 1, equals, interval 2) = a13;
interval 1
< a1; a2 ;… ; a13 >
Industrial Temporal Diagnostics (composing a network of relations)
Sensor 3Sensor 2
Relational network representing the particular caseIndustrial object
Sensor 1
Estimation of temporal relations between
symptoms
Industrial Temporal Diagnostics (generating temporal scenarios)
N1
Scenario S
N3N2
Object A Object B Object C
Generating the temporal scenario
for “Failure X”DB of
scenarios
1. for i=1 to n do
2. for j=i+1 to n do
3. if (R1) or…or (Rk) then
4. begin
5. for g=1 to n do
6. if not (Rg) then Reasoning(, Rg)
7. // if “Reasoning” = False then (Rg)=TUR
8. ( R) = Å ( Rt), where t=1,..k
9. end
10. else go to line 2
Recognition of Temporal Scenario
m
ii
m
iii
w
dwD
1
1SN,
)Bal()Bal(,, , DC,BA,DCBA
RRd RR
12
0,
1
12
1
i
iei BABal(RA,B) =
Industrial object
Temporal data
Relational network
DB ofscenarios
Estimation Recognition Diagnosis
Learning
bm
ofi
disi eq
sd
foi
mi
bi
wbi =1
weq
=0.5
wb =0 wf =0.75
Balance point for RA,B
Balance point for RC,D
Probability value
When Temporal Diagnostics ?1. Temporal diagnostics considers not only a static set of symptoms, but
also the time during which they were monitored. This often allows having a broader view on the situation, and sometimes only considering temporal relations between different symptoms can give us a hint to precise diagnostics;
2. This approach might be useful for example in cases when appropriate causal relationships between events (symptoms) are not yet known and the only available for study are temporal relationships;
3. Combination of Bayesian (based on probabilistic causal knowledge) and Temporal Diagnostics would be quite powerful diagnostic tool.
Terziyan V., Dynamic Integration of Virtual Predictors, In: L.I. Kuncheva, F. Steimann, C. Haefke, M. Aladjem, V. Novak (Eds), Proceedings of the International ICSC Congress on Computational Intelligence: Methods and Applications - CIMA'2001, Bangor, Wales, UK, June 19 - 22, 2001, ICSC Academic Press, Canada/The Netherlands, pp. 463-469.
VaganTerziyan
The Problem
During the past several years, in a variety of application domains, researchers in machine learning, computational learning theory, pattern recognition and statistics have tried to combine
efforts to learn how to create and combine an ensemble of classifiers.
The primary goal of combining several classifiers is to obtain a more accurate prediction than can be obtained from any single classifier alone.
Approaches to Integrate Multiple Classifiers
Integrating Multiple Classifiers
Selection Combination
Global (Static)
Local (Dynamic)
Local (“Virtual” Classifier)
Global (Voting-Type)
Decontextualization
Inductive learning with integration of predictors
rrmrr yxxx ,...,, 21
Sample Instances
tmtt xxx ,...,, 21
yt
Learning Environment
P1 P2 ... Pn
Predictors/Classifiers
Virtual Classifier
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CLDE,FS,TITP,TM,TC,
TC - Team Collector
TM - Training Manager
TP - Team Predictor
TI - Team Integrator
FS - Feature Selector
DE - Distance Evaluator
CL - Classification Processor
Virtual Classifier is a group of seven cooperative agents:
Classification Team: Feature Selector
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CL DE, ,TI TP, TM, TC, FS
FS - Feature Selector
Feature Selector:
finds the minimally sized feature subset that is sufficient for correct classification of the instance
Fea
ture
Sel
ecto
r
Sample InstancesSample Instances
rr yΧrr
' ΧΧΧ ,'rr y
Classification Team: Distance Evaluator
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CL , FS,TI TP, TM, TC, DE
DE - Distance Evaluator
Distance between Two Instances with Heterogeneous Attributes (example)
YyXxi
iii
ii
yxdYXD,,
2),(),(
i
ii
ii
ii
range
yx
yxi
yxd
:else
otherwise ,1
if ,0 - nominal is attributeth if
),(
where:
d (“red”, “yellow”) = 1 d (15°, 25°) = 10°/((+50°)-(-50°)) = 0.1
Distance Evaluator:
measures distance between instances based on their numerical or nominal attribute values
Distance Evaluator
imii xxx ,...,, 21 jmjj xxx ,...,, 21
ijd
Classification Team: Classification Processor
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CL DE, FS,TI TP, TM, TC,
CL - Classification Processor
Classification Processor:
predicts class for a new instance based on its selected features and its location relatively to sample instances
Classification Processor
imii xxx ,...,, 21
iy
Sample Instances
Feature Selector
Distance Evaluator
Team Instructors:Team Collector
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CL DE, FS,TI TP, TM, TC,
TC - Team Collector completes Classification Teams for training
Team Collector
completes classification teams for future training
Team Collector FSi DEj CLk
Feature Selection methods
Distance Evaluation functions
Classification rules
Team Instructors:Training Manager
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CL DE, FS,TI TP, , TC, TM
TM - Training Manager trains allcompleted teams on sample instances
Training Manager
trains all completed teams on sample instances
Training Manager
FSi1 DEj1CLk1
FSi2 DEj2CLk2
FSin DEjnCLkn
rrmrr yxxx ,...,, 21
Sample Instances
rnrrrmrr wwwxxx ,...,,,...,, 2121
Sample Metadata
Classification Teams
Team Instructors:Team Predictor
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CL DE, FS,TI , TM, TC, TP
TP - Team Predictor predicts weights forevery classification team in certain location
Team Predictor
predicts weights for every classification team in certain location
Team Predictor:
e.g. WNN algorithm
rnrrrmrr wwwxxx ,...,,,...,, 2121
Sample Metadata
imii xxx ,...,, 21 inii www ,...,, 21
Predicted weightsof classification teamsLocation
Team Prediction:Locality assumption
Each team has certain subdomains in the space of instance attributes, where it is more reliable than the others;
This assumption is supported by the experiences, that classifiers usually work well not only in certain points of the domain space, but in certain subareas of the domain space [Quinlan, 1993];
If a team does not work well with the instances near a new instance, then it is quite probable that it will not work well with this new instance also.
Team Instructors:Team Integrator
TeamtionClassificasInstructorTeam
Members Team Elective
Members TeamConstant
, CL DE, FS, , TP TM, TC, TI
TI - Team Integrator produces classificationresult for a new instance by integratingappropriate outcomes of learned teams
Team integrator
produces classification result for a new instance by integrating appropriate outcomes of learned teams
Tea
m In
teg
rato
r
FSi1 DEj1CLk1
FSi2 DEj2CLk2
FSin DEjnCLkn
tmtt xxx ,...,, 21
New instance
tntt www ,...,, 21
yt1
yt2
yt1
yt
Weights of classification teamsin the location of a new instance
Classification teams
Static Selection of a Classifier
Static selection means that we try all teams on a sample set and for further classification select one, which achieved the best classification accuracy among others for the whole sample set. Thus we select a team only once and then use it to classify all new domain instances.
Dynamic Selection of a Classifier
Dynamic selection means that the team is being selected for every new instance separately depending on where this instance is located. If it has been predicted that certain team can better classify this new instance than other teams, then this team is used to classify this new instance. In such case we say that the new instance belongs to the “competence area” of that classification team.
Conclusion
Knowledge discovery with an ensemble of classifiers is known to be more accurate than with any classifier alone [e.g. Dietterich, 1997].
If a classifier somehow consists of certain feature selection algorithm, distance evaluation function and classification rule, then why not to consider these parts also as ensembles making a classifier itself more flexible?
We expect that classification teams completed from different feature selection, distance evaluation, and classification methods will be more accurate than any ensemble of known classifiers alone, and we focus our research and implementation on this assumption.
Yevgeniy Bodyanskiy
Volodymyr Kushnaryov
Online Stochastic Faults’ PredictionControl Systems Research Laboratory, AI Department, Kharkov National University of Radioelectronics. Head: Prof. E. Bodyanskiy. Carries out research on development of mathematical and algorithmic support of systems for control, diagnostics, forecasting and emulation:
1. Neural network architectures and real-time algorithms for observation and sensor data processing (smoothing, filtering, prediction) under substantial uncertainty conditions;
2. Neural networks in polyharmonic sequence analysis with unknown non-stationary parameters;
3. Analysis of chaotic time series; adaptive algorithms and neural network architectures for early fault detection and diagnostics of stochastic processes;
4. Adaptive multivariable predictive control algorithms for stochastic systems under various types of constraints;
5. Adaptive neuro-fuzzy control of non-stationary nonlinear systems;
6. Adaptive forecasting of non-stationary nonlinear time series by means of neuro-fuzzy networks;
7. Fast real-time adaptive learning procedures for various types of neural and neuro-fuzzy networks.
Bodyanskiy Y., Vorobyov S, Recurrent Neural Network Detecting Changes in the Properties of Non-Linear Stochastic Sequences, Automation and Remote Control, V. 1, No. 7, 2000, pp. 1113-1124.
Bodyanskiy Y., Vorobyov S., Cichocki A., Adaptive Noise Cancellation for Multi-Sensory Signals, Fluctuation and Noise Letters, V. 1, No. 1, 2001, pp. 12-23.
Bodyanskiy Y., Kolodyazhniy V., Stephan A. An Adaptive Learning Algorithm for a Neuro-Fuzzy Network, In: B. Reusch (ed.), Computational Intelligence. Theory and Applications, Berlin-Heidelberg-New York: Springer, 2001, pp. 68-75.
Existing Tools
Most existing (neuro-) fuzzy systems used for fault diagnosis or classification are based on offline learning with the use of genetic algorithms or modifications of the error back propagation. When the number of features and possible fault situations is large, tuning of the classifying system becomes very time consuming. Moreover, such systems perform very poorly in high dimensions of the input space, so special modifications of the known architectures are required.
Neuro-Fuzzy Fault Diagnostics
Successful application of the neuro-fuzzy synergism to fault diagnosis of complex systems demands development of an online diagnosing system that quickly learns from examples even with a large amount of data, and maintains high processing speed and high classification accuracy when the number of features is large as well.
Challenge: Growing (Learning) Probabilistic Neuro-Fuzzy Network (1)
input layer,n inputs
1-st hidden layer,N neurons
2-nd hidden layer,(m+1) elements
output layer,m divisors
Bodyanskiy Ye., Gorshkov Ye., Kolodyazhniy V., Wernstedt J., Probabilistic Neuro-Fuzzy Network with Non-Conventional Activation Functions, In: Knowledge-Based Intelligent Information & Engineering Systems, Proceedings of Seventh International Conference KES’2003, 3–5 September, Oxford, United Kingdom, LNAI, Springer-Verlag, 2003.
Bodyanskiy Ye., Gorshkov Ye., Kolodyazhniy V. Resource-Allocating Probabilistic Neuro-Fuzzy Network, In: Proceedings of International Conference on Fuzzy Logic and Technology, 10–12 September, Zittau, Germany, 2003.
Challenge: Growing (Learning) Probabilistic Neuro-Fuzzy Network (2)
Implements fuzzy reasoning and classification (fuzzy classification fuzzy classification networknetwork);
Creates automatically neurons based on training set (growing growing networknetwork);
Learns free parameters of the network based on training set (learning networklearning network);
Guarantees high precision of classification based on fast learning (high- performance networkhigh- performance network);
Able to perform with huge volumes of data with limited computational resources (powerful and economical networkpowerful and economical network);
Able to work in real-time (real-time networkreal-time network).
Tested on real data in comparison with classical probabilistic neural network
Unique combination of features
Tests for Neuro-Fuzzy Algorithms
Industrial Ontologies Group (Kharkov’s Branch), Data Mining Research Group and Control Systems Research Laboratory of the Artificial Intelligence Department of Kharkov National University of Radioelectronics have essential theoretical and practical experience in implementing neuro-fuzzy approach and specifically Real-Time Probabilistic Neuro-Fuzzy Systems for Simulation, Modeling, Forecasting, Diagnostics, Clustering, Control .
We are interested in cooperation with Metso in that area and we are ready to present the performance of our algorithms on real data taken from any of Metso’s products to compare our algorithms with existing in Metso algorithms.
Inventions we can offer (1)
Method of intelligent preventive or predictive diagnostics and forecasting of technical condition of industrial equipment, machines, devices, systems, etc. in real time based on analysis of non-stationary stochastic signals (e.g. from sensors of temperature, pressure, current, shifting, frequency, energy consumption, and other parameters with threshold values).
The method is based on advanced data mining techniques, which utilize fuzzy-neuro technologies, and differs from existing tools by flexible self-organizing network structure and by optimization of computational resources while learning.
Inventions we can offer (2)
Method of intelligent real-time preventive or predictive diagnostics and forecasting of technical condition of industrial equipment, machines, devices, systems, etc. based on analysis of signals with non-stationary and non-multiplied periodical components (e.g. from sensors of vibration, noise, frequencies of rotation, current, voltage, etc.).
The method is based on optimization of computational resources while learning because of intelligent reducing of the number of signal components being analyzed.
Inventions we can offer (3)
Method and mechanism of optimal control of dosage and real-time infusion of anti-wear oil additives into industrial machines based on its real-time condition monitoring.
Summary of problems we can solve
Rather global system for condition monitoring and preventive maintenance based on OntoServ.Net (global, agent-based, ontology-based, Semantic Web services-based, semantic P2P search-based) technologies, modern and advanced data-mining methods and tools with knowledge creation, warehousing, and updating during not only device’s lifetime, but also utilizing (for various maintenance needs) knowledge obtained afterwards (various testing and investigations techniques other than information taken from “living” device’s sensors) from broken-down, worn out or aged components of the same type.
Recently Performed Case Studies (1)
Ontology Development for Gas Compressing Equipment Diagnostics Realized by Neural Networks
Available in: http://www.cs.jyu.fi/ai/OntoGroup/docs/July2003.pdf
VolodymyrKushnaryov
SemenSimkin
1212
NN and Ontology using for DiagnosticNN and Ontology using for Diagnostic
SENSOR
SIGNAL
Neural NetworkDiagnostic out
Training
Diagnosing
1515
The creating ontology classes The creating ontology classes instance programinstance program
The subclasses and their slots forming and The subclasses and their slots forming and instances filling by the information is instances filling by the information is carried out automatically with the program carried out automatically with the program on Java. The filling occurs from RDBMS on Java. The filling occurs from RDBMS Oracle, which contains in the Oracle, which contains in the actualizedactualizedbase using in ”base using in ”UkrTransGasUkrTransGas”.”.
OracleJava
Program Ontology
Recently Performed Case Studies (2)
The use of Ontologies for Faults and State Description of Gas-Transfer Units
Available in: http://www.cs.jyu.fi/ai/OntoGroup/docs/July2003.pdf
Agent
SCADA
Agent
SCADA
SCADA SCADA
Diagnosist Diagnosist
GTUGTU GTUGTU
Ontologyfor agent communication
VolodymyrKushnaryov
KonstantinTatarnikov
GTU-
MAINTENANCE
GTU
Control-type
Subsystem
GTU-State
Support-History
Period
Signal-Types
Repair-Reason
PARAMETER
ACTIONS
Shutdown
Launch
REPAIRMid-life Repair
Major Repair
Current Repair Planned Repair
GTU-Node
Compressorstation
SITUATIONS
Oil-temperaturedeviation
Axle-shear
Vibration
Rise-of-temperature
AnalogSignal
ComputeVariable
Trend
Conclusion
Industrial Ontologies Research GroupIndustrial Ontologies Research Group (University of Jyvaskyla), which is piloting the OntoServ.NetOntoServ.Net concept of the Global Semantic Web - Based System for Industrial Maintenance, has also powerful branches in Kharkovbranches in Kharkov (e.g. IOG-Kharkov’s Branch, Control Systems Research Laboratory, Data Mining Research Group, etc.) with experts and experiencesexperts and experiences in various and challenging data mining and knowledge discovery, online diagnostics, forecasting and control, models learning and integration, etc. methods, which can be and reasonable to be successfully utilized within going-on cooperation between MetsoMetso and Industrial Ontologies Group.