stream reasoning/cep

Ereignisse / Streams• Sensor Daten• IoT

• Business Events• DDD/CQRS/ES

• Abgeleitete Events• Log Files

Apache Spark • Batch processing framework• Designed as competition to Hadoop• Offline, data oriented processingÞHigh latency, big throughput

• Defines Cluster/distributed computing topology

Spark Streaming• Extension of Spark• Micro-Batch processing • small batch sizes ~ 1 sec, allows pseudo real-time

• Abstraction of streams with different operators • window functions, transformations, caching/persistence,…

Apache Storm• Complex Event Processing engine/framework• Designed as endless running processing framework• One event at a time => low latency, “real-time”

• Simple basic concept• Spouts produce only data• Blots consumes data, performs calculations and produce (new) data• Connected via a topology

• Low level abstraction • One has to handle “where” data is used – very imperative syntax

Apache Storm - Trident• Extension to Storm• Introduces Micro-Batching to increase throughput • Higher abstraction layer than “vanilla” Storm• More declarative than Storm topology• Concepts for persistence of data / state