1 serge abiteboul - monitoring 1 monitoring of distributed applications (in p2p) serge abiteboul,...

1

Serge Abiteboul - Monitoring 1

Monitoring of distributed applications (in P2P)

Serge Abiteboul, Pierre Bourhis, Bogdan Marinoiu,

INRIA Saclay and Université Paris 11

2

Serge Abiteboul - Monitoring 2

Organization

The context: the AXML program

Motivation for monitoring

The architecture

Axlog

Example

Conclusion

3

The context: the AXML program

Serge Abiteboul - Monitoring 3

4

Distributed data management with AXML

Active XML: XML document with embedded function calls• Intentional & dynamic information

• In pull or push mode

• Object & Active data

AXML store AXML peer/V2 • persistence, activation of local data

AXML query processor and optimizer Optimax• Distributed query plans

AXML Business artifacts: verification

AXML monitor P2Pmonitor• Distributed monitoring

Serge Abiteboul - Monitoring 4

5

Motivation

6

Serge Abiteboul - Monitoring 6

With the Web, more and more distribution

Often distributed applications are very dynamic• Content change rapidly

• Intense communications

Complex and hard to control systems• Many peers

• Peers are distributed

• Peers are autonomous

• Peers are sometimes unreliable and selfish

• Peers sometimes come and leave

Goal: monitor such systems

& support active features ala active databases

7

Example: Dell supply chain

Serge Abiteboul - Monitoring 7

Shipping Co.

Bank

Customer Webstore

Bank

Plant

Revolver

Shipping Co.

Suppliers

Monitor

Monitor

MonitorMonitor

Monitor

Monitor

Globalsupervisor

Monitoring of distributed information manufacturing system

8

Serge Abiteboul - Monitoring 8

More examples

Business applications• Gather information for billing, bug tracing…

• Procurement

Web Intelligence: e.g., business intelligence and surveillance of competing companies

Security surveillance against intrusion, spamming

P2P applications• Optimization and tuning: e.g., gather statistics, control indexing

• Error management, detection, diagnosis

9

The architecture

10

Architecture

Serge Abiteboul - Monitoring 10

publishers

Alerters

Streams

Streamprocessors

actions

RSS

11

Serge Abiteboul - Monitoring 11

Alerter: monitor a single system

Detect events at the peer level

Depend on the peer that is monitored

Monitor what?• Database updates• Events in RSS feed• Web page changes• WS calls (in/out)

Each event is represented as an XML document

Active stream

Activestream

12

Active streams

Monotone case• Stream of XML documents (possibly Active XML)

Nonmonotone case• Think of the stream as defining a forest

• We can send updates to the previous data, typically deletions

Insert [12,jean,1300], [11,marie,1555]

Insert [3,zoe,1333]

Delete [Id=12]; Insert [5,noé,1111]

In the style of RSS feeds

Streams are published as channel where it is possible to subscribe

They are implement as Web service calls

Serge Abiteboul - Monitoring 12

13

Stream processors

Efficient Filter - can support a large number of selections simultaneously

Binary operators: union, join, etc.

Operator with memory: duplicate elimination

Aggregation operator

Publisher• XML page, RSS, email, Web page

Novelty: axlog (axlog subscription & axlog engine)

Serge Abiteboul - Monitoring 13

14

Serge Abiteboul - Monitoring 14

Algebraic (monitoring) plans

ActiveXML algebra: same algebra as for distributed query processing

Algebra over streams of (A)XML documents• To send just some tree T: T, eos

From ActiveXML: XML + embedded service calls• A service may be any query

The algebraic glue: send, receive, eval

15

Axlog (axml & datalog)

16

Axlog principle = Doc + query

Incoming streams

The outgoing stream is defined by a query Q (tree pattern + join)

Each time an incoming message arrives, it modifies the document

Possibly the query resultAn outgoing message is generated

Incremental view maintenance

Serge Abiteboul - Monitoring 16

Q

Active XML document

17

Axlog - continued

We have implemented an axlog engine

We use datalog to benefit from1. Incremental view maintenance in datalog – Δ technique

2. Query optimization in datalog – Magic set

We have developed specific techniques 1. 3-valued based (true, false, possibleInFuture)

2. Avoid some derivation that Magic would do – preprocessing

3. Precompute in an optimistic manner

4. Garbage collection

Serge Abiteboul - Monitoring 17

18

Example

Serge Abiteboul - Monitoring 18

19

Example

Some process (e.g.,, BPEL workflow mailOrder) may be initiated at a webStore in US or France

The last operation of the process (e.g., shipping) occurs on Shipping – we are interested in shipping from UK

We want to detect when one mailOrder took more that 5 days to process when the customer is a Premium customer

Easy case• Monotone in-stream

• Monotone query

• No time constraint

• No aggregation

Serge Abiteboul - Monitoring 19

20

Local alerter specifications

On webStore US and webStore FR

for $call in(WS-outCall)

where $call/method = “mailOrder“

return <start> <Id> $call/ID <Id>

<time> $call/time </time> </start>

On shipping UK

for $call in(WS-outCall)

where $call/method = “shipping“

return <start> <Id> $call/ID <Id>

<time> $call/time </time> </start>

Serge Abiteboul - Monitoring 20

21

Axlog specification

AXML monitor.axml

<xml> <starts> mailOrder-Alert@webStore.dell.com()

mailOrder-Alert@webStore.dell.fr() </starts>

<ends> shippping-Alert@shipping.dell.uk() </ends>

</xml>

Query from supervisor in India

For $s in doc(“monitor.axml”)/starts/start,

$e in doc(“monitor.axml”)/ends/end

Where $s/Id = $e/Id and $s//Premium

and ($e/time - $s/time) > 1800 and

Return <long ID=$e/id> $s, $e </long>

Serge Abiteboul - Monitoring 21

22

Serge Abiteboul - Monitoring 22

Monitoring task plan

wsAlert@FR

σwsAlert@US

σ

ϴJoin

Reporter

Publisher

∪σ

wsAlert@UK

Emai lto India

23

Serge Abiteboul - Monitoring 23

Monitoring task plan – localisation

wsAlerterσwsAlerter

σ

ϴJoin

Reporter

Publisher

Publish

Site US

Site FR

Site UK

channel

Publish

∪σ

wsAlerter

Email to India

24

Conclusion

Prelim paper in International Workshop on Web Information and Data Management 2007

Demo in ICDE08

On going

• Temporal queries

• Negation in queries

• Deletion

Related work on Mashups with Tel Aviv Univ. & IBM Haifa

• Purely relational

Serge Abiteboul - Monitoring 24

25

Serge Abiteboul - Monitoring 25

Merci

Merci