distributed eventing in osgi

Distributed Eventing in OSGi Carsten Ziegeler | Adobe Research Switzerland

1

About cziegeler@apache.org @cziegeler

§  RnD Team at Adobe Research Switzerland §  OSGi Core Platform and Enterprise Expert Groups §  Member of the ASF

§  Most active in Apache Sling, Felix, ACE §  Conference Speaker §  Technical Reviewer §  Article/Book Author

2

Topics

§  OSGi Event Admin §  RFP 158 – Distributed Eventing §  Apache Sling Eventing Part I §  Apache Sling Topology §  RFC 183 – Cloud Ecosystems §  Apache Sling Eventing Part II

3

4

OSGi Event Admin

OSGi Event Admin Publish Subscribe Model

5

OSGi Event Admin Component

A publish

deliver

Component X

Component Y

OSGi Event Admin Publish Subscribe Model

§  OSGi event is a data object with §  Topic (hierarchical namespace) §  Properties (key-value-pairs)

§  Resource Event §  Topic:

org/apache/sling/api/resource/Resource/ADDED §  Properties: path, resource type etc.

6

OSGi Event Admin Publish Subscribe Model

§  Publisher creates event object §  Sends event through EventAdmin service §  Either synchronous or asynchronous delivery

7

OSGi Event Admin Publish Subscribe Model

§  Publisher creates event object §  Sends event through EventAdmin service §  Either synchronous or asynchronous delivery

8

@Componentpublic class EventSender { @Reference private EventAdmin eventAdmin; public void run() { while ( true ) { final Map<String, Object> properties = new HashMap<String, Object>(); properties.put("timestamp", System.currentTimeMillis()); final Event event = new Event("i/am/still/alive", properties); eventAdmin.sendEvent(event); // eventAdmin.postEvent(event); Thread.sleep(2000); } }}

OSGi Event Admin Publish Subscribe Model

§  Subscriber is an OSGi service (EventHandler) §  Whiteboard pattern §  Service registration properties §  Interested topic(s)

§  org/apache/sling/api/resource/Resource/*

§  Additional filters (optional) §  (path=/libs/*)

9

OSGi Event Admin Publish Subscribe Model

§  Subscriber is an OSGi service (EventHandler) §  Whiteboard pattern §  Service registration properties

10

@Component@Service(value=EventHandler.class)@Properties({ @Property(name=EventConstants.EVENT_TOPIC, value="i/am/still/alive")})public class EventReceiver implements EventHandler { public void handleEvent(final Event event) { System.out.println("Still alive at " + event.getProperty("timestamp")); }}

OSGi Event Admin Publish Subscribe Model

11

§  Immediate delivery to available subscribers §  No guarantee of delivery §  No distributed delivery

OSGi Event Admin Publish Subscribe Model

12

§  Immediate delivery to available subscribers §  No guarantee of delivery §  No distributed delivery

Apache Sling Discovery

Distributed Eventing Job Handling

OSGi RFP 158

13

RFP 158 – Distributed Eventing

RFP 158 – Distributed Eventing

§  Asynchronous remote eventing §  One-to-one, one-to-many §  Messages with replies

§  Advertising and selection of Quality of Service §  Sender identification §  Language agnostic

14

15

Apache Sling Distributed Event Admin

Distributed OSGi Events

§  Apache Sling Eventing leverages OSGi Event Admin §  Listener for all OSGi events with

§  event property "event.distribute=true” §  Events are distributed

§  Event property "event.application” contains the origin

16

Distributed OSGi Events

§  Apache Sling Eventing leverages OSGi Event Admin

17

@Componentpublic class EventSender { @Reference private EventAdmin eventAdmin; public void send() { final Map<String, Object> properties = new HashMap<String, Object>(); properties.put("event.distribute", Boolean.TRUE); properties.put("timestamp", System.currentTimeMillis()); final Event event = new Event("i/am/still/alive", properties); eventAdmin.sendEvent(event); }}

Distributed OSGi Events

§  Apache Sling Eventing leverages OSGi Event Admin

18

@Component@Service(value=EventHandler.class)@Properties({ @Property(name=EventConstants.EVENT_TOPIC, value="i/am/still/alive")})public class EventReceiver implements EventHandler { public void handleEvent(final Event event) { System.out.println("Still alive " + event.getProperty("event.application") + " : " + event.getProperty("timestamp")); }}

Distributed OSGi Events

§  Apache Sling Eventing leverages OSGi Event Admin §  Sender controls distribution §  Apache Sling used for distribution

§  (through JCR content repository, Apache Jackrabbit) §  API is independent from implementation

19

20

Apache Sling - Discovery Topologies

Installation Scenarios

21

Single Instance

Instance 1

Instance 2

Instance 3

Topologies I Apache Sling Discovery

22

§  Instance: Unique Id (Sling ID)

ID : A ID : X ID : 42 ID : 1

Single Instance

Instance 1

Instance 2

Instance 3

Topologies I Apache Sling Discovery

23

§  Instance: Unique Id (Sling ID) §  Cluster: Unique Id and leader

Cluster 99

Cluster 35

ID : A ID : X ID : 42 ID : 1

Single Instance

Instance 1

Instance 2

Instance 3

Leader Leader

Topologies I Apache Sling Discovery

24

§  Topology: Set of clusters

Cluster 99

Cluster 35

ID : A ID : X ID : 42 ID : 1

Single Instance

Instance 1

Instance 2

Instance 3

Leader Leader

Topology Topology

Cluster 99

Cluster 35

Topologies I Apache Sling Discovery

25

§  Topology: Set of clusters

ID : A ID : X ID : 42 ID : 1

Single Instance

Instance 1

Instance 2

Instance 3

Leader Leader

Topology

Topologies II Apache Sling Discovery

§  Instance §  Sling ID §  Optional:

Name and description §  Belongs to a cluster

§  Might be the cluster leader

§  Additional distributed properties §  Extensible through own services (PropertyProvider) §  E.g. data center, region or enabled job topics

26

Cluster 99

ID : 42

Instance 3

Topology

Leader

Topologies II Apache Sling Discovery

§  Cluster §  Elects (stable) leader §  Stable instance ordering

27

Cluster 99

ID : 42

Instance 3

Topology

Leader

Topologies II Apache Sling Discovery

§  TopologyEventListener §  Whiteboard pattern §  Receives events on topology changes

§  Topology is changing §  Topology changed §  Properties changed

28

Cluster 99

ID : 42

Instance 3

Topology

Leader

RFC 183 – Cloud Ecosystems

§  “Distributed” FrameworkNodeStatus services §  Information about each cloud node

§  Defined instance properties §  Extensible

29

RFC 183 – Cloud Ecosystems

§  “Distributed” FrameworkNodeStatus services §  Information about each cloud node

§  Defined instance properties §  Extensible

§  Similar to Apache Sling’s topology §  Well defined services vs. listener and data objects §  One can implement the other J

30

31

From Events to Jobs

Job Handling I Apache Sling Job Handling

§  Job : Guaranteed processing, exactly once §  Exactly one job consumer

§  Started by client code §  Job topic §  Payload is a serializable map

§  API is similar to OSGi Event Admin

32

Starting / Processing a Job I Apache Sling Job Handling

33

public interface JobManager { Job addJob(String topic, Map<String, Object> properties); …}

Starting a job

Starting / Processing a Job I Apache Sling Job Handling

34

public interface JobConsumer { String PROPERTY_TOPICS = "job.topics"; enum JobResult { OK, FAILED, CANCEL, ASYNC } JobResult process(Job job);}

Processing a job

Job Handling I Apache Sling Job Handling

§  Sling Job Manager handles and distributes jobs §  Delivers job to a job consumer… §  …and waits for response §  Retry and failover

§  Notification listeners (fail, retry, success)

35

Starting / Processing a Job II Apache Sling Job Handling

36

@Component@Service(value={JobConsumer.class})@Property(name=JobConsumer.PROPERTY_TOPICS, value="org/apache/sling/jobs/backup")public class BackupJobConsumer implements JobConsumer { @Override public JobResult process(final Job job) { // do backup return JobResult.OK; }}

Job Handling Apache Sling Job Handling

§  Distributed job processing within a topology §  Instances might have different capabilities §  Offloading of heavy jobs

§  Various queue configurations §  Failover handling §  Topology changes §  Progress tracking §  History of jobs

37

Topics

§  OSGi Event Admin §  RFP 158 – Distributed Eventing §  Apache Sling Eventing Part I §  Apache Sling Topology §  RFC 183 – Cloud Ecosystems §  Apache Sling Eventing Part II

38