Download - HSIENA: a hybrid publish/subscribe system
![Page 1: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/1.jpg)
Magdeburg sept 25 2012 DESEC4LCCI
HSIENA a hybrid publish/subscribe system
Fabio PetroniLeonardo Querzoni
Department of Computer, Control, and Management Engineering Antonio Ruberti
Workshop on Dependable and Secure Computing for Large-scale Complex Critical Infrastructures
Magdeburg, 25 september 2012
![Page 2: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/2.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 2
Publish/subscribe paradigm
The publish/subscribe interaction scheme [1] provides a form of communication, alternative to the standard clients/server model, where participants are decoupled with respect to:
TIMEThe participants do not
need to be active at the same time,
publishers might publish events while some subscribers are
disconnected, andsubscribers might get notified about some
events while the original publisher is
disconnected
SPACEThe interacting
parties do not need to know
each other. An event notification service (ENS) is responsibleto gather publishers issued events and to diffuse them toward
the subscribers
SYNCHRONIZATION
An event is asynchronously
propagated to all subscribers that
registered interest on it, and publishers
are neverblocked while
producing events
1
![Page 3: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/3.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 3
CONTENT BASED
Subscription: conjunction of predicates.
TOPIC BASED
Subscription:set of topics
Static schemaLimited expressiveness
Topic-based vs Content-based
Broker Publisher / Subscriber
football news
Event Notification Service
News= footballTeam= LazioYear > 2010
SIENARouting Scheme
[2,3]
2
![Page 4: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/4.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 4
SIENA
3
PHYSICAL OVERLAY
BROADCAST LAYER
CONTENT-BASEDLAYER
“all-pairs path
symmetry”
![Page 5: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/5.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 5
The problems of SIENA
SIENA suffers by the lack of adequate support to system reconfiguration
In particular the addition or the removal of a new node to the system requires a full halt, followed by a manual reconfiguration of the broadcast and the content-based layers
This can lead to large management overhead and reduced performance in dynamic or large-scale environments
GOAL: make SIENA layers self-organizing, so that the whole system needs a reduced management by human administrators
4
![Page 6: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/6.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 6
Related work
XSIENA [4] proposes a soft state approach
The idea is to use timed subscriptions, and re-issue them periodically, in order to automatically manage and restore the state of crashed subscribers and publishers
Cugola et al. [5] proposed a solution for the single broadcast tree case, limiting the reconfiguration to a well defined path
Moreover, they have clearly defined the reconfiguration problem
5
![Page 7: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/7.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 7
Reconfiguration problem 6
PHYSICAL OVERLAY
BROADCAST LAYER
CONTENT-BASEDLAYER
![Page 8: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/8.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 8
Reconfiguration problem 6
PHYSICAL OVERLAY
BROADCAST LAYER
CONTENT-BASEDLAYER
reconfiguration of the overlay networkto maintain connectivity among participants
1
BROADCAST LAYER
![Page 9: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/9.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 9
Reconfiguration problem 6
PHYSICAL OVERLAY
BROADCAST LAYER
CONTENT-BASEDLAYER
reconfiguration of the overlay networkto maintain connectivity among participants
1
BROADCAST LAYER
reconfiguration of the subscription information
to bring the routing tables up-to-date with the changes
2
CONTENT-BASEDLAYER
![Page 10: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/10.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 10
Reconfiguration problem 6
PHYSICAL OVERLAY
BROADCAST LAYER
CONTENT-BASEDLAYER
reconfiguration of the overlay networkto maintain connectivity among participants
1
BROADCAST LAYER
reconfiguration of the subscription information
to bring the routing tables up-to-date with the changes
2
CONTENT-BASEDLAYER
minimization of event loss during reconfiguration
3
![Page 11: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/11.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 11
IDEA
How do we coordinate the brokers in the system, since they are geographically spread, independent and possibly linked by networks with unpredictable latency (like internet)?
Solutions based on a central coordinator may be inefficient for several reasons:
fault tolerance (single point of failure)
organizational (different administrative domains)
scalability
PROPOSAL: use a storage system reliable and easily accessible by all processes in the system, supplied by a cloud provider
The cloud provider takes care of maintaining the storage available and consistent
We only have to manage concurrent accesses of brokers to this storage
Montresor and Abeni (2011)[6] proposed a similar idea in the context of gossip algorithms
7
![Page 12: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/12.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 12
HSIENA 8
Cloud Service
Storage
epoch number
Pred D
PHYSICAL OVERLAY
![Page 13: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/13.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 13
HSIENA – insertion of a new broker 1/2 9
Cloud Service
Storage
PHYSICAL OVERLAY
BROADCAST LAYER
Pred
D
![Page 14: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/14.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 14
HSIENA – insertion of a new broker 2/2 10
Cloud Service
Storage
PHYSICAL OVERLAY
BROADCAST LAYER
EPOCH UPDATE
Pred
D
![Page 15: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/15.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 15
HSIENA – reconfiguration of the overlay network 11
Pred
PHYSICAL OVERLAY
BROADCAST LAYER
SELECT
![Page 16: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/16.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 16
HSIENA – reconfiguration of the subscription info 12
PHYSICAL OVERLAY
BROADCAST LAYER
CONTENT-BASEDLAYER
1
2
3PULL
UPDATE
![Page 17: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/17.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 17
HSIENA – Concurrency 1/7
< i, [INS/REM], y, V >
“The broker y is performing a [INS/REM] operation that will bring the system to epoch i. V is a set of broker ids used only for INS operations (neighbors of y)”
Assumption: the storage service provides a test-and-set primitive
13
Opsen queue of
ongoing operations
![Page 18: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/18.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 18
HSIENA – Concurrency 2/7
14
3
21 A
3
21
B
i
< i+1, INS, A, {1,2} > < i+1, INS, B, {2,3} >
![Page 19: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/19.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 19
HSIENA – Concurrency 3/7
14
Ops
3
21 A
3
21
B
ien
< i+1, INS, A, {1,2} > < i+1, INS, B, {2,3} >
T&S T&S
![Page 20: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/20.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 20
HSIENA – Concurrency 4/7
14
Ops
3
21 A
3
21
B
ien
AApending
< i+2, INS, B, {2,3} >START INSERTION PROCEDURE
![Page 21: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/21.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 21
HSIENA – Concurrency 5/7
14
Ops
3
21 A
3
21
B
ien
A
START INSERTION PROCEDURE
T&S
PERFORM A's INSERTIONSTART INSERTION PROCEDURE
![Page 22: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/22.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 22
HSIENA – Concurrency 6/7
15
Cloud Service
Storage
i i+1 i+2
ABlist of couples
of matrices
![Page 23: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/23.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 23
HSIENA – Concurrency 7/7
If B succedes, A omits the epoch update.
As last step, the entry in Ops is deleted
16
3
21 A
3
21
B
i
A
i+2i+1
T&S T&S
![Page 24: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/24.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 24
Conclusions and future works
HSIENA is a hybrid system that complements the SIENA publish/subscribe system by adding the ability to self-reconfigure after brokers additions and removals.
HSIENA has a novel design that mixes the classic SIENA’s distributed architecture with a highly available cloud-based storage service that brokers use as a shared memory space they can rely-on to adapt at runtime the ENS application-level network without service disruption.
We are implementing a prototype of HSIENA to test its behaviour under various realistic loads.
Our purpose is to asses both its ability to support insertion and deletions while providing service continuity and to study the tradeoff existing between the level of service HSIENA can guarantee and the cost incurred for maintaining state information stored on a cloud service.
17
![Page 25: HSIENA: a hybrid publish/subscribe system](https://reader034.vdocument.in/reader034/viewer/2022042723/5875b5e31a28ab8b618b770d/html5/thumbnails/25.jpg)
Magdeburg sept 25 2012 DESEC4LCCI 25
References
[1] R. Baldoni, L. Querzoni, S. Tarkoma, A. Virgillito: “Distributed Event Routing in Publish/Subscribe Communication Systems”. Springer (2009)
[2] A. Carzaniga, D. Rosenblum, A. Wolf: “Design and evaluation of a wide area notification service”. TOCS (2001)
[3] A. Carzaniga, M.J. Rutherford, A. Wolf: “A routing scheme for content-based networking”. INFOCOM (2004)
[4] Z. Jerzak, C. Fetzer: “Soft state in publish/subscribe”. DEBS (2009)
[5] G. Cugola, D. Frey, A.L. Murphy, G.P. Picco: “Minimizing the Reconfiguration Overhead in Content-Based Publish-Subscribe”. SAC (2004)
[6] A. Montresor, L. Abeni: “Cloudy weather for P2P, with a chance of gossip”. P2P (2011)
18