v ladimir gorodetsky head of laboratory of intelligent systems space.iias.spb.su/ai

V. Gorodetsky IADM-06, Discussion, Hong Kong, December 18, 2006

VVladimir Gorodetskyladimir GorodetskyHead of Laboratory of Intelligent Systems Head of Laboratory of Intelligent Systems

http://space.iias.spb.su/ai/http://space.iias.spb.su/ai/[email protected]@mail.iias.spb.su

Agent and Data Mining ResearchAgent and Data Mining Researchin Laboratory of Intelligent Systems in Laboratory of Intelligent Systems

(St. Petersburg Institute for Informatics and Automation)(St. Petersburg Institute for Informatics and Automation)


ContentsContents

1. Structure of the research and developments of the Intelligent System Laboratory

2. Multi-Agent System Development Kit (MASDK): A software tool supporting MAS application technology

3. Agent-based distributed data mining and machine learning

4. International collaboration

5. Russian Grant and projects

6. Relevant publications


Laboratory stuff

• 11 researchers including

• Ph.D. -- 3

• Research analysts and programmers – 4

• Ph.D. students -- 4


1. Structure of the Research and Developments of the Intelligent System Laboratory


Types of the Research of IS LaboratoryTypes of the Research of IS Laboratory

Fundamental research: Machine learning, distributed data mining and decision making Resource constraint project planning and scheduling Protocols for distributed data mining and decision making Agent-based simulation

Technology and software tools Technology and software tool for multi-agent application design,

implementation and deployment Agent-based technology for distributed data mining and decision making

system Technology for resource constraint project planning and scheduling Software tool kit for machine learning

Multi-agent applications (software prototyping) Intrusion detection, Design process planning, scheduling and management, Image processing, Airspace deconfliction, Transportation logistics, etc.


Airspace deconfliction (P2P decision making)

Distributed data mining and decision making infrastructure

Research StructureResearch Structure

Multi-agent technology and MASDK software tool

Computer Network security

Information fusion for situation assessment

Transportation logistics

Intrusion detection

Learning of Intrusion detection

Simulation of distributed attacks against computer network

Knowledge-based project planning and scheduling

Image processing

Problem-oriented multi-agent technology

Project planning and scheduling

Agent-based simulation

RoboCup (2004 World winner in

Simulation league)

Data mining & machine learning tool kit

P2P agent-based service-oriented networks (NEW)


2. Multi-Agent System Development Kit: A Software Tool Supporting MAS Application Technology


General Description of MASDK: Multi-Agent System General Description of MASDK: Multi-Agent System Development KitDevelopment Kit

System Core

Applied system

specificationin XML

Host

AgentAgent

Agent

Host

AgentAgent

Agent

Multi Agent System Development Kit

Integrated editor system

Software agent

builder

Communicationplatform

Genericagent

Portal Portal


3. Agent-based Distributed Data Mining and Machine Learning


Agent-based (Mediated) Distributed Learning InfrastructureAgent-based (Mediated) Distributed Learning Infrastructure

Data Source Sensor

Data Source

KE

Data Source Sensor

Data Source KE

Data Source

Sensor

Data Source

KE

Data Source

Sensor

Data Source KE

Communication PlatformMeta-level infrastructure component

User interface

Meta-level KE (manager)

Interaction Protocols

Host 1

Host 2

User interfa

ce

Source-based

Infrastru

cture

component

User interface

Source-based

infrastructure

component

Use

r in

terf

ace

Sour

ce-b

ased

Infr

astr

uctu

re

com

pone

nt

User interface

Source-based

Infrastructure

component

Host 3

Host k

Distributed Learning Infrastructure=source host-based components + meta-Distributed Learning Infrastructure=source host-based components + meta-level component+ interaction protocols + communication platform +user level component+ interaction protocols + communication platform +user

interfaces (not the machine learning algorithms!)interfaces (not the machine learning algorithms!)


Example of Application: Distributed Learning of Example of Application: Distributed Learning of Intrusion Detection (Hierarchical Architecture)Intrusion Detection (Hierarchical Architecture)

NETWORL TRAFFIC

Data Source 1 Data Source 2 Data Source 3 Data Source 4

Preprocessing procedures

Source-based classifiers




Two-level meta-classification

Decision stream 1

Decision stream 2

Decision stream 3

Decision stream 5

Input: composition of asynchronous data streams

Computer security status: {Normal or attack of a class}Output:

Data Source 5


Decision stream 4


International Collaboration (Projects)International Collaboration (Projects)

• US Air Force Research Laboratory - European Office of Aerospace Research and Development--8 year collaboration since 1998, 5 projects successfully completed, 1 - in progress until August 2007, new one is discussed)

• FP4, FP5, FP6: “AgentLink: Coordination Action for Agent-based Computing”,

• FP6 FET Project: “POSITIF” – “Formal specification and verification of computer network security policy”,

• FP5 KDNet NoE: “Data Mining and Knowledge Discovery”,

• FP6 KDUbiq NoE: “Knowledge Discovery for Ubiquitous Computing” (WG2 member)

• Cadence Design System Ltd. (USA, German Research office) – “Multi-agent system for design activity support in microelectronics” (2004-2006)

• INTEL (USA)–”Preprocessing algorithms for intrusion detection” (2004-2005)

• Fraunhofer First Institute, BMBF (Germany) – MIND–”Machine Learning in Intrusion Detection System” (2004-2006)


Grants and Projects: RussiaGrants and Projects: Russia

Grants of Russian Foundation for Basic Research: • Multi-agent technology for distributed learning and decision

making (2004-2006);

Projects from Department of Information Technology and Computer Systems of the Russian Academy of Sciences:• Agent-based stochastic modeling and simulation of adversarial

competition of teams in the Internet environment (2003-2005);• Mathematical models of active audit of computer network

vulnerabilities, intrusion detection and response: Multi-agent approach (2003-2005);

• Multi-agent technology and software tool (2004-2006)


International Conferences etc. Organized by IS International Conferences etc. Organized by IS LaboratoryLaboratory

1-4. Mathematical methods, model and architectures for computer network security (MMM-ACNS): 2001, 2003, 2005 (Proceedings in LNCS of Springer, vol. 2952, 2776, 3685), MMM-ACNS-2007 will be held in September of 2007 (St. Petersburg, Russia).

5. International Workshop of Central and Eastern Europe on Multi-agent Systems (CEEMAS): 1999.

6-7. International Workshop on Autonomous Intelligent Systems: Agents and Data Mining (AIS-ADM): June 2005 (Proceedings in LNAI of Springer, vol.3505), AIS-ADM-2007 will be held in June of 2007 (St. Petersburg, Russia).


Distributed Data Mining and Decision Making – related Distributed Data Mining and Decision Making – related PublicationsPublications

V.Gorodetsky, O.Karsaev and V.Samoilov. On-Line Update of Situation Assessment: Generic Approach. In International Journal of Knowledge-Based & Intelligent Engineering Systems. IOS Press, Netherlands, 2005,

V.Samoylov, V.Gorodetsky. Ontology Issue in Multi–Agent Distributed Learning. In V.Gorodetsky, J.Liu, V. Skormin (Eds.). Autonomous Intelligent Systems: Agents and Data Mining. Lecture Notes in Artificial Intelligence, vol. 3505, 2005, 215-230.

O.Karsaev. Technology of Agent-Based Decision Making System Development. In V.Gorodetsky, J.Liu, V. Skormin (Eds.). Autonomous Intelligent Systems: Agents and Data Mining. Lecture Notes in Artificial Intelligence, vol. 3505, 2005, 107-121.

V.Gorodetsky, O.Karsaev and V.Samoilov. Direct Mining of Rules from Data with Missing Values. Studies in Computational Intelligence, Volume 6, Chapter in book T.Y.Lin, S.Ohsuga, C.J. Liau, X.T.Hu, S.Tsumoto (Eds.). Foundation of Data Mining and Knowledge Discovery, Springer, 2005, 233-264

V.Gorodetsky, O.Karsaev, V.Samoylov, A.Ulanov. Asynchronous Alert Correlation in Multi-Agent Intrusion Detection Systems, Lecture Notes in Computer Science, Vol.3685, Springer, 2005, 366-379


Distributed Data Mining and Decision Making – related Distributed Data Mining and Decision Making – related PublicationsPublications

V.Gorodetsky, O.Karsaev, V.Samoilov, and A.Ulanov. Multi-Agent Framework for Intrusion Detection and Alert Correlation. NATO ARW Workshop "Security of Embedded Systems", Patras, Greece, August 22-26, 2005. In Proceedings of the Workshop, IOS Press, 2005.

V.Gorodetsky, O.Karsaev, and V.Samoilov. On-Line Update of Situation Assessment Based on Asynchronous Data Streams. In M.Negoita, R.Howlett, L.Jain (Eds.) Knowledge-Based Intelligent Information and Engineering Systems, Lecture Notes in Artificial Intelligence, vol. 3213, Springer Verlag, 2004, pp.1136–1142 (Received The Best Paper Award)

V.Gorodetsky, O.Karsaev, V.Samoilov. Multi-agent and Data Mining Technologies for Situation Assessment in Security Related Application. In B.Dunin-Keplicz, A. Jankovski, A.Skowron, M.Szczuka (Eds.) Monitoring, Security, and Rescue Techniques in Multi-agent Systems. Series of books Advances in Soft Computing, Springer, 2004, 411-422.

V.Gorodetsky, O.Karsaev, I.Kotenko, and V.Samoilov. Multi-Agent Information Fusion: Methodology, Architecture and Software Tool for Learning of Object and Situation Assessment. International Conference "Fusion-04", Stockholm, 2004, pp. 346–353


Distributed Data Mining and Decision making – related Distributed Data Mining and Decision making – related PublicationsPublications

V.Gorodetsky, O.Karsaev, and V.Samoilov. Distributed Learning of Information Fusion: A Multi-agent Approach. Proceedings of the International Conference "Fusion 03", Cairns, Australia, July 2003, 318–325.

V.Gorodetsky, O.Karsaeyv, and V.Samoilov. Multi-agent Technology for Distributed Data Mining and Classification. Proceedings of the IEEE Conference Intelligent Agent Technology (IAT03), Halifax, Canada, October 2003, 438–441.

V.Gorodetsky, O.Karsaev, and V.Samoilov. Software Tool for Agent-Based Distributed Data Mining. Proceedings of the IEEE Conference Knowledge Intensive Multi-agent Systems (KIMAS 03), Boston, USA, October 2003, 710–715,

etc.


For more information and related publications please contact

E-mail: [email protected]

http://space.iias.spb.su/ai/gorodetsky

Contact dataContact data


Future Research and Development in Agent and Future Research and Development in Agent and Data Mining AreaData Mining Area

VVladimir Gorodetskyladimir GorodetskyHead of Laboratory of Intelligent Systems Head of Laboratory of Intelligent Systems

http://space.iias.spb.su/ai/http://space.iias.spb.su/ai/[email protected]@mail.iias.spb.su


Focus of the Laboratory Current and Forthcoming Focus of the Laboratory Current and Forthcoming Research ProjectsResearch Projects

1. Algorithms for P2P rule extraction from distributed data sources with overlapping attributes -- DDM area.

2. P2P Agent platform –Agent area (now it is subject of activity of FIPA Nomadic Agent Working Group).

3. Software tool kit supporting agent-based P2P rule extraction from distributed data sources – integrated area

The main idea: From hierarchical agent-based distributed decision making to P2P (serverless) ad-hoc agent-based

service-oriented decision making networks


Example: Hierarchical Architecture of Distributed Example: Hierarchical Architecture of Distributed Decision Making for Intrusion Detection TaskDecision Making for Intrusion Detection Task

NETWORL TRAFFIC

Data Source 1 Data Source 2 Data Source 3 Data Source 4

Preprocessing procedures





Two-level meta-classification

Decision stream 1

Decision stream 2

Decision stream 3

Decision stream 5

Input: composition of asynchronous data streams

Computer security status: {Normal or attack of a class}Output:

Data Source 5


Decision stream 4


Hierarchical Architecture: Multi-Agent IDSHierarchical Architecture: Multi-Agent IDS Intended for Intended for Heterogeneous Alert CorrelationHeterogeneous Alert Correlation

Preprocessing proceduresNETWORK TRAFFIC

Heterogeneous alerts notify about various classes of attacks,

either DoS, or Probe, or U2R

Classifiers : Attack class – data source

1 DoS –connection-based data

2 R2U –time window-based data -1

3 Prob – time window-based data -1

4 R2U – time window-based data -1

5 Prob –connection window data-1

6 Prob – connection-based data

7 R2U – connection-based data

8 DoS – time window-based data -2

9 R2U –time window-based data -2

10 DoS – time window-based data -2


P2P Architecture of Distributed Decision Making for P2P Architecture of Distributed Decision Making for Intrusion Detection Task:Intrusion Detection Task:

Data sources

5

4

UI

6

8

7

23

9

1

10

P2P classifiers

Example : Serverless (P2P) network for intrusion detection (no meta-classifiers). Each agent detecting an alert acts as combiner of decisions provided by other agents (“service providers”) on its request


Ground Object Recognition Ground Object Recognition Based on Infra Red Based on Infra Red Images Produced by Airborne Equipment Images Produced by Airborne Equipment

Agent-classifiers

Recognizedobject

Object recognition components of the agent-based software

Meta-agent

2D Views

Scale Invariant Feature Transform (SIFT)

Object models (set of features)

Wavelet Transform (WT)

Structural Description (SD)

SIFT 1

SIFT 2

WT 1

WT 2

SD 1

SD 2

Infra red data preprocessing and their transformation into feature spaces

Decision combining

The Task: On-line automatic recognition of ground objects based on infra-red images perceived by airborne surveillance system.

Classifier 1

Classifier 2

Objects’ models

Model 1

Model 2

Model 3

Model 16

…Classifier 3

Classifier 16

…


Ground Object Recognition: Structure of Decision Ground Object Recognition: Structure of Decision Making and Decision CombiningMaking and Decision Combining

2–SIFT-basedObject of class 2 -left

2–SIFT-basedObject of class 2 -right


3–SIFT-basedObject of class 2-right

Recognized objects

3–SIFT-basedObject of class 4 -front


2–SIFT-basedObject of class 4 -front

2–SIFT-basedObject of class 4 –l eft

Combined decision of the classifiers trained to detect the object class 4

Meta-classifier combining decision of particular meta-classifiers

Combined decision of the classifiers trained to detect the object class M60

2-SIFT-based Object of class 1

- right

3-SIFT-based Object of class 1

- right



2–SIFT-basedObject of class 3

- front

2–SIFT-basedObject of class 3

- right

3–SIFT-basedObject of class

3 - front

3–SIFT-basedObject of class

3 - right

2–SIFT-based Object of class 3

- back

3–SIFT-based Object of class 3

- back


Agent-based P2P Classification Network Implementing Agent-based P2P Classification Network Implementing Ground Object Recognition SystemGround Object Recognition System

4UI

9

25

7

24

10

8

13

3

17

11

23

5

19

20

1

6

12

14

15

16

18

21

22

Classifiers detecting the

objects of class 1


objects of class 3

4 219 238 10

181915


objects of class 2


objects of class 4

24 123 5

17 1325 620 111 22

147

16

Agent providing user interface


Software Prototype of Agent-based Service- oriented P2P Software Prototype of Agent-based Service- oriented P2P Classification Network for Ground Object RecognitionClassification Network for Ground Object Recognition

The main window of the user interface of the P2P classification network for ground object recognition


Architecture of Agent-based Service-oriented P2P Architecture of Agent-based Service-oriented P2P NetworkNetwork

Network Transport

…

PEER 1

Existing P2P networking middleware

Agent 1-1 Agent 1-2 Agent 1-k

…

P2P agent platform

General requirements to P2P agent platform architecture are formulated in the document of Nomadic Agent Working Group (NAWG) of FIPA. Our expected contribution is a version of its implementation and verification (via software prototyping on the basis of particular classification networks).

PEER 1



…

P2P agent platform


Architecture of a Peer of Agent-based Service-Architecture of a Peer of Agent-based Service-oriented P2P Networkoriented P2P Network

PEER : P2P Agent Platform instance


…

Message Transport System Interface

InterfaceAMS

(dll, Agent)

Transport System (TCP/IP)

(UDP)…

interfaceRouting Book

Interface Yellow Pages

(dll, Agent)

OnReceiveHandler

OnReceiveHandler

Peer Address book

Service book

Agent book

Message history

Search Results

Search Results

OnReceiveHandler

OnReceiveHandler

OnReceiveHandler



Hot ProblemsHot Problems

1. Development of P2P agent platform decoupling peers and applications and supporting open service–oriented architectures, self–optimization of the network structure through on-line learning. Although the last problem is currently the subject of the intensive research in the networking scope, for agent-based architecture it will require specific efforts.

2. Combining of decisions produced by P2P agents within distributed heterogeneous environment. A peculiarity of this task is that in each particular case, the classifications incoming from the peers may be very diverse in the sense that different peers may be involved in service provision. That is why, distributed learning of decision combing that is a challenging task of P2P data mining and ubiquitous computing should be an important component of the technology in question.


For more information and related publications please contact

E-mail: [email protected]

http://space.iias.spb.su/ai/gorodetsky

Contact dataContact data

v ladimir gorodetsky head of laboratory of intelligent systems space.iias.spb.su/ai

Documents