baldoni pub

7/21/2019 Baldoni Pub

1/74

MINEMA Summer School 1

Universit di Roma La Sapienza

Dipartimento di Informatica e Sistemistica

The Publish/SubscribeCommunication Paradigm and itsApplication to Mobile Systems

Roberto BaldoniDipartimento di Informatica e SistemisticaUniversit degli Studi di Roma La Sapienza


2/74

MINEMA Summer School



Outline

Introduction

Basics of pub/sub communication systems Subscription models

Architectural models

Event Routing

Impact of mobility


3/74

MINEMA Summer School "



Subscribersregister subscriptionscharacterizing information interests

Publisherspublish notifications withinteresting information

Middleware determines whichsubscriptions match which notifications

Middleware delivers notifications tomatching subscribers

Introduction: The Publish/Subscribe

Communication Style

Publisher SubscriberPublish/Subscribe Middleware


4/74

MINEMA Summer School #



Inroduction The Publish/Subscribe

Paradigm Publish/Subscribe pub/sub!" a po#erful abstractionfor building distributed applications $essage%based& anonymous communication

Participants are decoupled

in space" no need to be connected or even 'no# each other in flo#" no need to be synchroni(ed

in time" no need to be up at the same time

)ood solution for highly dynamic& decentrali(edsystems e*g*& #ired environments #ith huge

numbers of publishers and subscribers& $A+E,s&P-P etc!

$any research issues& involving several researchareas e*g*& systems& soft#are eng*& databases etc!


5/74




Introduction

.idely popular both in the industry and in theresearch

Industry solutions for pub/sub are almost centrali(ed

Research has mainy to face the scalability aspect

Applications

Stoc' information delivery

Auction system

Air traffic control

*etc

$any names for pub/sub systems" notification

service& data distribution service


6/74

MINEMA Summer School $



Basic Interaction Model

S0

S-

S1

publisherssubscribers

2IB$3

2A4$E3

2A4$E3

subscription

IB$" %1&56

A4$E" 78&06

IB$" 7-&60

event

S0" IB$S-" A4$E

S1" A4$E

notification serviceIB$" %1&56

IB$" 0-&60

A4$E" 78&06

A4$E" 78&06

notification


7/74

MINEMA Summer School %



istributed !oti"ication Ser#ice

S0

S-

S1

2IB$3

2A4$E3

2A4$E3

IB$" %1&56

A4$E" 78&06

IB$" 7-&60

notification serviceIB$" %1&56

IB$" 0-&60

A4$E" 78&06

A4$E" 78&06

Matching

9etermining the recipients for an event

Routing

9elivering a notification to all the recipients

S1" A4$E

S-" A4$E

S0" IB$

broker


8/74

MINEMA Summer School &



Research ,opics

System architecture

Efficient routing algorithm

Reliable messaging sending the :oS issue!

Security Special application issues

$illions of users

$obile environment

P-P

Sensors net#or's*


9/74

MINEMA Summer School '



Pub/Sub Systems

IBM )ryphon

Stanford SI;, and more

CUBoulder Siena

!rance thers ?$


10/74

MINEMA Summer School 1!



Subscription Models

Topic-Based [Oki et al. 93]:

%events are divided in topics%subscribers subscribe for a single

topic

Content-Based [Carzaniga et al2001]:%subscriptions are generic Cueries

S:


11/74




Pub/Sub $ariants: Topic%based

Event space is divided in topics&

corresponding to logical channels

Participants subscribe for a topic and publish

on a topic

Receivers for an event are 'no#n a priori

4hannel G )roup

,herefore often eploit net#or'%level multicast )roup communication


12/74




Content Based pub%sub: e#ent schema Subscriptions and events defined over an

n%dimensional event space %E*g* StockName = ACME andchange < -3)

Subscription" conFunction of constraints

eventsubscription

Content&ased

su&scriptionscan

include range

constraints

a0

a-


13/74

MINEMA Summer School 1"



Pub/Sub $ariants: Content%based

Receivers are calculated for each event being

published

Eploiting multicast is not straighfor#ard

=o# are subscribers assigned to groupsH

2A4$E3& change %1

2A4$E3

A4$E&changeG%6

A4$E& change G %-


14/74

MINEMA Summer School 1#



Topic #s& Content

,opic%based pub/sub

Recipients are 'no#n a%priori

$any efficient implementations eist" e*g* tib


15/74




Architectural Model o" a pub/sub

Netor* Multicasting

+ro*er layer oerlay

Structure .erlay


16/74

MINEMA Summer School 1$



Architectural Model o" a pub/sub

+et#or' $ulticasting se of multicast net#or'ing facilities also at data lin'

level!

Bro'er layer overlay

Based on transport level connections bet#een nodes

=ierarchical 9ecision tree from publisher to

subscribers!

ndirected Acyclic graph spanning all bro'ers Structured >verlay

9=, abstracting from physical nodes!


17/74

MINEMA Summer School 1%



'$'!T #s S(BSC)IPTIO! )O(TI!*

Etreme solutions Sol 0 event flooding!

flooding of events in the notification event bo

each subscription stored only in one place #ithin

the notification event bo $atching operations eCual to the number of

bro'ers

Sol - subscription flooding!

each subscription stored at any place #ithin thenotification event bo

each event matched directly at the bro'er #herethe event enters the notification event bo


18/74

MINEMA Summer School 1&



/onnectiity

0ro*ers u0lisherssu0scri0ers

Handling subscriptions

Identification of intermediate routing solutions


19/74

MINEMA Summer School 1'



/onnectiity

0ro*ers u0lishers

Su0scrition 2artitioning (s)

s

su0scri0ers

Handling subscriptions: subscription partitioning i.e., where to stores



20/74

MINEMA Summer School !



/onnectiity

0ro*ers u0lishers


s

su0scri0ers

Su0scrition 3outing

Handling subscriptions: subscription routing i.e., how to brings

there



21/74




Su0scrition 3outing


/onnectiity

su0scri0ers 0ro*ers u0lishers

e

Handling events



22/74

MINEMA Summer School



Su0scrition 3outing


/onnectiity


e

Eent 2artitioning (e)

Handling events: Event Partitioning i.e., where to match e



23/74




Su0scrition 3outing


/onnectiity


e


Handling events: Event Routing i.e., How to bring e there

Eent 3outing



24/74




Matching (e)

Handling events: Event Matching i.e., define the set of recipients of e

Su0scrition 3outing


/onnectiity

Eent 3outing


e


Eent 3outing


U i i di R S i


25/74




Su0scrition 3outing


/onnectiity

Eent 3outing


Notification routing

e

Handling events: otification Routing i.e., bring e to the destinations


Matching (e)

Eent 3outing


U i it di R L S i


26/74

MINEMA Summer School $



Su0scrition 3outing

Su0scrition 2artitioning

Eent 3outing

Notification routing


Matching


U i it di R L S i


27/74




Basic Techni+ues "or intermediate

routing solutions ;iltering%based Routing J4ar(aniga et al*

-880K

ndirected Acyclic graph spanning all the bro'ers

Rende(%Lous Routing

J.ang et al* -88-K based on dynamic partitioning

of the event space among a set of bro'ers

J4astro et al -88-K based on 9=,



28/74

MINEMA Summer School &



,iltering based routing

Identifying as soon as possible events that arenot interesting for any subscriber and arrest their

diffusion

4reating 2diffusion paths3 that lead tosubscribers for the event

4onstruction of a diffusion path reCuires routing

info to be maintained at bro'ers

Routing information consists of a set of filtersaggregate of subscriptions! that are reachable

through that bro'er



29/74

MINEMA Summer School '



,iltering based routing: Three main phases

Subscription for#arding" subscriptions filters!are propagated to every bro'er leaving state

along the path

$atching notification follo# bac'#ards! the

path set by subscriptions

Reducing subscription propagation

Avoiding subscription propagation #hen a filter

including the subscription has been alreadyfor#arded



30/74

MINEMA Summer School "!



-

.

0

1

2

3

4

5

SI'!A,iltering%Based )outingSubscription ,orwarding

a

s1: price > 600

s1:as1:a

s1:1s1:1

s1:.s1:.

s1:-s1:-

s1:.s1:.

s1:3s1:3

s1:-s1:-

s1:1s1:1

s1:4s1:4



31/74

MINEMA Summer School "1



-

.

0

2

3

4

5

SI'!A,iltering%Based )outingSubscription Merging

s1:1s1:1

s1:.s1:.

s1:3s1:3

s1:-s1:-

s1:1s1:1

s2: price > 700

b

s1:-s1:-

s1:.s1:.

s1:4s1:4

s1:1

s.:4

s1:1s.:4

s1:.

s.:0

s1:.s.:0

s1:4

s.:b

s1:4

s.:b

a

s1co#ers s2

1

s1co#ers s2

s1:as1:as1:a

s.:.

s1:a

s.:.slitter

slitter



32/74




-

.

0

1

2

3

4

5

SI'!A,iltering%Based )outing!oti"ication eli#ery

b

s1:1

s.:4

s1:1s.:4

s1:.s1:.

s1:3s1:3

s1:-s1:-

s1:1s1:1s1:-s1:-

s1:.

s.:0

s1:.s.:0

a s1:as.:.

s1:a

s.:.

s1:4

s.:b

s1:4

s.:b

n1matches s1n1matches s2

n1:

price = 8



33/74

MINEMA Summer School ""



,iltering based routing

,radeoff bet#een subscription redundancyand subscription flooding

,he more the bro'ers a#are of anysubscription the earlier notifications that donot match any subscription are filtered out

,radeoff bet#een epressiveness of

subscription language and scalability Epressiveness compleity brings to flood filters


34/74

MINEMA Summer School "#



)ende6 $ous )outing

Each event of the event space is o#ned by one or morereliability reasons! bro'ers

,hree functions"

S+!" -+ rende(%vous nodes of !

E+e!" -+ rende(%vous nodes of e! Ife S+! thenS+! E+e! intersection rule!

$ain characteristics"

4ontrolled subscriptions distribution

4ontrolled matching Better load balance



35/74

MINEMA Summer School "5



Rendezvous-based routing

Each node is responsible for a partition of the event space Storing subscriptions& matching events

M

e

Probem" difficult to define mapping functions #hen the set of

nodes changes over time

M

M

M

M

eM



36/74

MINEMA Summer School "$

p


)ende6%$ous routing based on 7T

A matching publisher D subscriber must come up #iththe same hash 'eys based on the content

4istri0ute ash 6a0le

0uc*ets

istri0ute u0lish7su0scri0e system



37/74

MINEMA Summer School "%

p



su0scri0er

home noe


0uc*ets

su0scrition




38/74

MINEMA Summer School "&

p



u0lishersu0scri0er

home noe


0uc*ets

u0lication

su0scrition




39/74

MINEMA Summer School "'



u0lishersu0scri0er


0uc*ets

su0scrition u0lication

home noe




40/74

MINEMA Summer School #!



u0lishersu0scri0er


0uc*ets

su0scrition

u0lication

home noe




41/74

MINEMA Summer School #1


)ende6%$ous routing 8Baldoni et al .994

D!"

'nmapping

#$%pub&sub

Subs a'mapping

'ppication

sub! pub! notify!

send! Foin!delivery! leave!

unsub!

Event space is

mapped into

the universe of

'eys fied!

>verlay

maintains

consistency of

+ mapping

Stateess mapping"%9oes not depend on

eecution history

subscriptions& node Foins

and leaves!



42/74



)ende6 $ous )outing

Problem" bottlenec'at hash buc'et nodes

Structuring subscribers as a multicast diffusion tree

Restrictions on the subscription language"

$apping bet#een multidimensional multi%typed subscriptions tothe uni or bi%dimensional space of a structured overlay is not

straightfor#ard e*g*& string manipulation!

Strong points" =andle inherently dynamic changes in the overlay


Di ti t di I f ti Si t i ti


43/74

MINEMA Summer School #"


Summary o" routing

;iltering based routing"

>rder of hundreds of bro'ers

$anaged environment Rende(%vous routing based on 9=,

>rder of thousands of nodes

nmanaged environment




44/74

MINEMA Summer School ##


A "ew pub/sub systems




45/74

MINEMA Summer School #5


Pub/Sub Sysems: Tib/)$ 8O;i et al

9- ,opic Based

,#o level hierarchical architecture of bro'ers

deamons! on ,4P/IP Event routing is reali(ed through one diffusion

tree per subFect

Each bro'er 'no#s the entire net#or'

topology and current subscription

configuration




46/74

MINEMA Summer School #$


Pub/Sub systems: *ryphon 8IBM 99

4ontent based

=ierarchical tree from publishers to

subscribers ;iltering%based routing

$apping content%based to net#or' level

multicast




47/74

MINEMA Summer School #%


Pub/Sub Systems: SC)IB' 8Castro et al&

9. ,opic Based

Based on 9=, Pastry!

Rende(%vous event routing A random identifier is assigned to each topic

,he pastry node #ith the identifier closest to

the one of the topic becomes responsible forthat topic




48/74

MINEMA Summer School #&


M'*7OOT

4ontent Based

Based on Structured >verlay 4A+

$apping the subscription language and theevent space to 4A+ space

Subscription and event Routing eploit 4A+

routing algorithms




49/74

MINEMA Summer School #'


Mobile 'n#ironment




50/74

MINEMA Summer School 5!


Potential Ad#antages o" Pub/Sub "or

Mobile


51/74



Impact o" mobility: In"ormation%Centric

Mobile


52/74



Scenarios o" mobility

>ne%hop mobile net#or'

4entrali(ed vs distributed

dispatcher

NE9I -880& =uang -880

$ulti%hop mobile net#or'

$A+E,!

+o #ired infrastructure ;reCuent changes in

topology -88- no#!




53/74

MINEMA Summer School 5"

pa t e to d o at ca e S ste st ca

One%hop wireless networ;

Problems in distributedpub/sub architecturesJ=uang -880K" failures&message losses anddisconnections

Important issues in areplicated pub%subJ=uang -880K"

>rderedness 4onsistency

4ompleteness




54/74

MINEMA Summer School 5#

p

Mobile ad%hoc networ;: issues

4ostantly changing topology nicast is epensive #hile broadcast is cheap

4ommunication is less 2reliable3 than in #iredsystems due to disconnections driven by

mobility or volountary!

Effects on ho# to design a middle#are for

pub/sub e*g*& to improve performance of theevent dispatching system reducing thecompleity of epressiveness!




55/74


p

,re+uent =uestions

9o #e really need comple subscription

languages for pub/sun over $A+E,sH

.hat is the right $A+E, modelHH Reconfiguration and a long period of stability or

;reCuent reconfigurations and short period of

stability

Partition or not partitionHHH

Allo#ing partitions means a lot of memory on boardOO




56/74

MINEMA Summer School 5$

Architectural Model "or Mobile ad%

hoc networ;s

MA/

3outing 2u07su0

MA/

2S-3outing

Alication

Alication

MA/

Alication2u07su0

8uang et al !!,

+aloni et al !!5,

+ahemi et al !!59

8Mottola et al !!5,

+acon et al !!59

8Anceaume et al !!,

2icco et al !!"9




57/74

MINEMA Summer School 5%

Topological )econ"iguration

MA/

3outing

Alication

/+ 2u07su0

82icco et al !!"9

!ssumption: the unerlying tree is *et connectean loo-free 0y some algorithm

SIENA is the target /+-2u07Su0 System

"arget: rearrange route traerse 0y eents

in resonse to changes in the toology of the

netor* of isatchers (0ro*ers)

Searation of concerns 0eteen connectiity layer

an eent isatching layer




58/74

MINEMA Summer School 5&


MA/

3outing

Alication

/+ 2u07su0

82icco et al !!"9

A+ lin* 0rea*age

/4 lin* formation

.riginal toology of the

eent isatching layer




59/74

MINEMA Summer School 5'


MA/

3outing

Alication

/+ 2u07su0

82icco et al !!"9

#trawman approach:

A an + trigger unsu0scritions

in their su0trees immeiately after

lin* 0rea*age

/ an 4 trigger su0scritions

immeiately hen lin* aears

Proposed approach:after lin* 0rea*age, A an +

elay the triggerring of

unsu0scritions in their su0trees/ an 4 trigger su0scritions

immeiately hen lin* aears

Rationale:

Su0scritions ersist longer on the tree an uon reconnection

su0scritions roagate only till the first slitter (e;ecuting the filter

merging)




60/74

MINEMA Summer School $!


Retrofitting reliability events lost duringreconfiguration! through gossip%based algorithms

.hat about partitioning or isolationHH

JAnceaume et al* -88-K assumes the net#or' doesnot partition for a time long enough to maintain the9A)

+e# approaches" Semi%deterministic routing 4ostaet al -886!

,rading subscription diffusion for better reaction totopology changes

Similar to RP in $A+E,

MA/

3outing

Alication

/+ 2u07su0

82icco et al !!"9




61/74

MINEMA Summer School $1

Integration Approach

MA/

2S-3outing

Alication

8Mottola et al !!59

!ssumption: the unerlying tree is *et connecte

an loo-free 0y MA.4 their istance from bstrictly increase an their

istance to bstrictly ecrease>

2. bian bi+1are a?acent each other

4o not rely on a ositioning serice (a 0eacon

message is sufficient to estimate the istance

0eteen to a?acent noes)

6he aroach is fully istri0ute (each rocess

ecie autonomously if relaying the message)




65/74

MINEMA Summer School $5

Broadcast%based approach

/+ 2u07su0

MA/

Alication

8+aloni et al !!59

+asic 0eaconing Algorithm:Each 0ro*er erioically sens a 0eacon containing

its filterA 0ro*er receiing 0eacons from a?acent 0ro*ers

form a hint ta0le

6he hint ta0le contains for each ro the last time

the 0eacon has 0een receie from a gien 0ro*er>

6his ermits a 0ro*er to etermine ho many

0eacons hae 0een misse from another 0ro*er> 6hisalue is set to:

if the to 0ro*ers neer come in contact! if they are currently aiacent




66/74

MINEMA Summer School $$

Broadcast%based approach+asic @escreasing istance Algorithm: each eent carries a estination list (the @estimate list of 0ro*ers

intereste to that eent)

3eceiing an eent e, a 0ro*er bgoes in the @ready to forward

state for e if:

(i) 0 is aare of some 0ro*er not mentione in the estination listan matchnig the eent or

(ii) for some 0ro*er bcontaine in the estination list hose

hint is greater than the one store at b

If @ready to forward state for e, bscheules the actualforaring of eith a elay roortional to the loest hint

alica0le from the reiou stes

4uring the elay if breceies efrom some other neigh0ors it

ros the foraring




67/74

MINEMA Summer School $%

Broadcast%based approach

/+ 2u07su0

MA/

Alication

8+aloni et al !!59




68/74

MINEMA Summer School $&

Comparison

,opological reconfiguration +ot suitable for freCuent topology changes too

much overhead for updating the filter information!

not suitable for partition or isolation too much

overhead! this means no disconnected #or'Al#ay need cooperations among bro'ers

Broadcast%based Suitable for freCuent topology changes

4an adapt to partitions and isolations Epoit the broadcast nature of the environment

+odes do not need to cooperate




69/74

MINEMA Summer School $'

)esearch irections "or pub/sub on MA!'Ts

Alications

3eference imlementations

3eal eloyments

/ature funamental ro0lemsose 0y ynamic 0ehaior

Please stop doing papers

on algorithms & sim

+utBmost imortant

Cos efinition an enforcement

2roerties of sarse netor*s




70/74

MINEMA Summer School %!

ynamic istributed Systems

4ommon problems among implementingpub/sub on mobile ad%hoc net#or's and

reali(ing pub/sub on a structured overlay or

on a dense sensor net#or'

Similar techniCues have been used

9ynamic behavior becomes crucial in modern

distributed systems

It #ould be interested to studying general

solutions to such a common problem




71/74

MINEMA Summer School %1

Than;s

Roberto Beraldi& )ianpaolo 4ugola& Antonino Lirgillito& RomanLitemberg& $ariangela 4ontenti& 4arlo $archetti& Sara ,ucci

Piergiovanni&


72/74


A "ew re"erencesPaolo 4osta& $atteo $igliavacca& )ian Pietro Picco& )ianpaolo 4ugola" Epidemic Algorithms for Reliable 4ontent%Based Publish%Subscribe" An Evaluation* I494S -88@" 66-%60

Paolo 4osta& $atteo $igliavacca& )ian Pietro Picco& )ianpaolo 4ugola" Introducing reliability in content%basedpublish%subscribe through epidemic algorithms* 9EBS -881

)ian Pietro Picco& )ianpaolo 4ugola& Amy hio& Nune -886

P* 4osta * )*P* Picco& Semi Probabilistic content based%pub%subscribe ,o appear in Proceedings of the -6thInternational 4onference on 9istributed 4omputing Systems I494S U86!& 4olumbus& >hio& Nune -886




73/74

MINEMA Summer School %"

Other re"erences




74/74

Other re"erences

baldoni pub

Documents