distributed eventing in osgi
DESCRIPTION
Presentation from OSGi Community Event / EclipseCon Europe 2013 One of the major topics the OSGi alliance is working on is a proposal for distributed eventing especially in the cloud. This session starts with an overview of the current state in the alliance and then shows already available solutions from the Apache Sling open source project. This includes distributing events through event admin and controlled processing of events by exactly one processor in distributed installations. The current implementations will be set in context to the ongoing activations in the alliance.TRANSCRIPT
Distributed Eventing in OSGi Carsten Ziegeler | Adobe Research Switzerland
1
About [email protected] @cziegeler
§ RnD Team at Adobe Research Switzerland § OSGi Core Platform and Enterprise Expert Groups § Member of the ASF
§ Most active in Apache Sling, Felix, ACE § Conference Speaker § Technical Reviewer § Article/Book Author
2
Topics
§ OSGi Event Admin § RFP 158 – Distributed Eventing § Apache Sling Eventing Part I § Apache Sling Topology § RFC 183 – Cloud Ecosystems § Apache Sling Eventing Part II
3
4
OSGi Event Admin
OSGi Event Admin Publish Subscribe Model
5
OSGi Event Admin Component
A publish
deliver
Component X
Component Y
OSGi Event Admin Publish Subscribe Model
§ OSGi event is a data object with § Topic (hierarchical namespace) § Properties (key-value-pairs)
§ Resource Event § Topic:
org/apache/sling/api/resource/Resource/ADDED § Properties: path, resource type etc.
6
OSGi Event Admin Publish Subscribe Model
§ Publisher creates event object § Sends event through EventAdmin service § Either synchronous or asynchronous delivery
7
OSGi Event Admin Publish Subscribe Model
§ Publisher creates event object § Sends event through EventAdmin service § Either synchronous or asynchronous delivery
8
@Componentpublic class EventSender { @Reference private EventAdmin eventAdmin; public void run() { while ( true ) { final Map<String, Object> properties = new HashMap<String, Object>(); properties.put("timestamp", System.currentTimeMillis()); final Event event = new Event("i/am/still/alive", properties); eventAdmin.sendEvent(event); // eventAdmin.postEvent(event); Thread.sleep(2000); } }}
OSGi Event Admin Publish Subscribe Model
§ Subscriber is an OSGi service (EventHandler) § Whiteboard pattern § Service registration properties § Interested topic(s)
§ org/apache/sling/api/resource/Resource/*
§ Additional filters (optional) § (path=/libs/*)
9
OSGi Event Admin Publish Subscribe Model
§ Subscriber is an OSGi service (EventHandler) § Whiteboard pattern § Service registration properties
10
@Component@Service(value=EventHandler.class)@Properties({ @Property(name=EventConstants.EVENT_TOPIC, value="i/am/still/alive")})public class EventReceiver implements EventHandler { public void handleEvent(final Event event) { System.out.println("Still alive at " + event.getProperty("timestamp")); }}
OSGi Event Admin Publish Subscribe Model
11
§ Immediate delivery to available subscribers § No guarantee of delivery § No distributed delivery
OSGi Event Admin Publish Subscribe Model
12
§ Immediate delivery to available subscribers § No guarantee of delivery § No distributed delivery
Apache Sling Discovery
Distributed Eventing Job Handling
OSGi RFP 158
13
RFP 158 – Distributed Eventing
RFP 158 – Distributed Eventing
§ Asynchronous remote eventing § One-to-one, one-to-many § Messages with replies
§ Advertising and selection of Quality of Service § Sender identification § Language agnostic
14
15
Apache Sling Distributed Event Admin
Distributed OSGi Events
§ Apache Sling Eventing leverages OSGi Event Admin § Listener for all OSGi events with
§ event property "event.distribute=true” § Events are distributed
§ Event property "event.application” contains the origin
16
Distributed OSGi Events
§ Apache Sling Eventing leverages OSGi Event Admin
17
@Componentpublic class EventSender { @Reference private EventAdmin eventAdmin; public void send() { final Map<String, Object> properties = new HashMap<String, Object>(); properties.put("event.distribute", Boolean.TRUE); properties.put("timestamp", System.currentTimeMillis()); final Event event = new Event("i/am/still/alive", properties); eventAdmin.sendEvent(event); }}
Distributed OSGi Events
§ Apache Sling Eventing leverages OSGi Event Admin
18
@Component@Service(value=EventHandler.class)@Properties({ @Property(name=EventConstants.EVENT_TOPIC, value="i/am/still/alive")})public class EventReceiver implements EventHandler { public void handleEvent(final Event event) { System.out.println("Still alive " + event.getProperty("event.application") + " : " + event.getProperty("timestamp")); }}
Distributed OSGi Events
§ Apache Sling Eventing leverages OSGi Event Admin § Sender controls distribution § Apache Sling used for distribution
§ (through JCR content repository, Apache Jackrabbit) § API is independent from implementation
19
20
Apache Sling - Discovery Topologies
Installation Scenarios
21
Single Instance
Instance 1
Instance 2
Instance 3
Topologies I Apache Sling Discovery
22
§ Instance: Unique Id (Sling ID)
ID : A ID : X ID : 42 ID : 1
Single Instance
Instance 1
Instance 2
Instance 3
Topologies I Apache Sling Discovery
23
§ Instance: Unique Id (Sling ID) § Cluster: Unique Id and leader
Cluster 99
Cluster 35
ID : A ID : X ID : 42 ID : 1
Single Instance
Instance 1
Instance 2
Instance 3
Leader Leader
Topologies I Apache Sling Discovery
24
§ Topology: Set of clusters
Cluster 99
Cluster 35
ID : A ID : X ID : 42 ID : 1
Single Instance
Instance 1
Instance 2
Instance 3
Leader Leader
Topology Topology
Cluster 99
Cluster 35
Topologies I Apache Sling Discovery
25
§ Topology: Set of clusters
ID : A ID : X ID : 42 ID : 1
Single Instance
Instance 1
Instance 2
Instance 3
Leader Leader
Topology
Topologies II Apache Sling Discovery
§ Instance § Sling ID § Optional:
Name and description § Belongs to a cluster
§ Might be the cluster leader
§ Additional distributed properties § Extensible through own services (PropertyProvider) § E.g. data center, region or enabled job topics
26
Cluster 99
ID : 42
Instance 3
Topology
Leader
Topologies II Apache Sling Discovery
§ Cluster § Elects (stable) leader § Stable instance ordering
27
Cluster 99
ID : 42
Instance 3
Topology
Leader
Topologies II Apache Sling Discovery
§ TopologyEventListener § Whiteboard pattern § Receives events on topology changes
§ Topology is changing § Topology changed § Properties changed
28
Cluster 99
ID : 42
Instance 3
Topology
Leader
RFC 183 – Cloud Ecosystems
§ “Distributed” FrameworkNodeStatus services § Information about each cloud node
§ Defined instance properties § Extensible
29
RFC 183 – Cloud Ecosystems
§ “Distributed” FrameworkNodeStatus services § Information about each cloud node
§ Defined instance properties § Extensible
§ Similar to Apache Sling’s topology § Well defined services vs. listener and data objects § One can implement the other J
30
31
From Events to Jobs
Job Handling I Apache Sling Job Handling
§ Job : Guaranteed processing, exactly once § Exactly one job consumer
§ Started by client code § Job topic § Payload is a serializable map
§ API is similar to OSGi Event Admin
32
Starting / Processing a Job I Apache Sling Job Handling
33
public interface JobManager { Job addJob(String topic, Map<String, Object> properties); …}
Starting a job
Starting / Processing a Job I Apache Sling Job Handling
34
public interface JobConsumer { String PROPERTY_TOPICS = "job.topics"; enum JobResult { OK, FAILED, CANCEL, ASYNC } JobResult process(Job job);}
Processing a job
Job Handling I Apache Sling Job Handling
§ Sling Job Manager handles and distributes jobs § Delivers job to a job consumer… § …and waits for response § Retry and failover
§ Notification listeners (fail, retry, success)
35
Starting / Processing a Job II Apache Sling Job Handling
36
@Component@Service(value={JobConsumer.class})@Property(name=JobConsumer.PROPERTY_TOPICS, value="org/apache/sling/jobs/backup")public class BackupJobConsumer implements JobConsumer { @Override public JobResult process(final Job job) { // do backup return JobResult.OK; }}
Job Handling Apache Sling Job Handling
§ Distributed job processing within a topology § Instances might have different capabilities § Offloading of heavy jobs
§ Various queue configurations § Failover handling § Topology changes § Progress tracking § History of jobs
37
Topics
§ OSGi Event Admin § RFP 158 – Distributed Eventing § Apache Sling Eventing Part I § Apache Sling Topology § RFC 183 – Cloud Ecosystems § Apache Sling Eventing Part II
38