challenges, approaches, and solutions in stream reasoning

Emanuele Della Valle - visit http://streamreasoning.org

Challenges, Approaches, and Solutions in

Stream Reasoninghttp://streamreasoning.org

Emanuele Della Valle DEI - Politecnico di Milano

[email protected]://emanueledellavalle.org

http://streamreasoning.org/


mailto:[email protected]

http://emanueledellavalle.org/

Emanuele Della Valle - visit http://streamreasoning.org Emanuele Della Valle - visit http://streamreasoning.org

Agenda• Motivation• Concept • Achievements• Applications• Conclusions

Stavanger, 2012-5-9 2




MotivationIt‘s a streaming World! [IEEE-IS2009] 1/3

• Oil operations

• Traffic

• Financial markets

• Social networks

• Generate data streams!






• … and want to analyse data streams in real time

• In a well in progress to drown, how long time do I have givenits historical behavior?

• Is public transportation where the people are?

• Can we detect any intra-daycorrelation clusters among stock exchanges?

• Who is driving the discussion about the top 10 emerging topics ?






• e.g., Real Time Rome (mobile network data streams)

afte

rnoo

nev

enin

gNormal day Exceptional day

[sou

rce:

http

://se

nsea

ble.

mit.

edu/

real

timer

ome/

]




http://senseable.mit.edu/realtimerome/



• e.g., Pulse of the Nation (social media streams)

happier

12:00

23:00

[sou

rce:

http

://w

ww

.ccs

.neu

.edu

/hom

e/am

islo

ve/tw

itter

moo

d/ ]




http://www.ccs.neu.edu/home/amislove/twittermood/


MotivationWhat are data streams anyway?

• Formally: – Data streams are unbounded sequences of time-

varying data elements

• Less formally: – an (almost) “continuous” flow of information – with the recent information being more relevant as it

describes the current state of a dynamic system

time





MotivationThe continuous nature of streams

• The nature of streams requires a paradigmatic change*

– from persistent data • to be stored and queried on demand • a.k.a. one time semantics

– to transient data • to be consumed on the fly by continuous queries• a.k.a. continuous semantics

* This paradigmatic change first arose in DB community [Henzinger98]Stavanger, 2012-5-9 8




Motivation – The continuous nature of streamsContinuous Semantics

• Continuous queries registered over streams that, in most of the cases, are observed trough windows

window

input streams streams of answerRegistered Continuous

Query





Motivation – The continuous nature of streamsTools exists [Cugola2011]

• Types– Data Stream Management Systems– Complex Event Processors

• Research Prototypes– Amazon/Cougar (Cornell) – sensors– Aurora (Brown/MIT) – sensor monitoring, dataflow– Gigascope: AT&T Labs – Network Monitoring– Hancock (AT&T) – Telecom streams– Niagara (OGI/Wisconsin) – Internet DBs & XML– OpenCQ (Georgia) – triggers, view maintenance– Stream (Stanford) – general-purpose DSMS– Stream Mill (UCLA) - power & extensibility– Tapestry (Xerox) – publish/subscribe filtering– Telegraph (Berkeley) – adaptive engine for sensors– Tribeca (Bellcore) – network monitoring

• High-tech startups– Streambase, Coral8, Apama, Truviso

• Major DBMS vendors are all adding stream extensions as well– IBM InfoSphere Stream – Microsoft streaminsight– Oracle CEP





Motivation New Requirements New Challenges

Typical Requirements• Processing Streams• Large datasets• Reactivity• Fine-grained information

access• Modeling complex

application domains

• Continuous semantics• Scalable processing• Real-time systems• Powerful query

languages• Rich ontology

languages




MotivationAre DSMS/CEP ready to address them?



application domains

DSMS/CEP• Continuous semantics• Scalable processing• Real-time systems• Powerful query


languages




Motivation Is Semantic Web ready to address them?

• The Semantic Web, the Web of Data is doing fine– RDF, RDF Schema, SPARQL, OWL, RIF– well understood theory, – rapid increase in scalability

• BUT it pretends that the world is staticor at best a low change rateboth in change-volume and change-frequency– ontology versioning– belief revision– time stamps on named graphs

• It sticks to the traditional one-time semantics





MotivationNew Requirements New Challenges



application domains

Semantic Web• Continuous semantics• Scalable processing• Real-time systems• Powerful query


languages




Motivation New Requirements call for Stream Reasoning



application domains

• Continuous semantics• Scalable processing• Real-time systems• Powerful query


languages

Stream Reasoning




ConceptStream Reasoning Definition [IEEE-IS2010]

• Making sense – in real time – of multiple, heterogeneous, gigantic and inevitably

noisy data streams – in order to support the decision process of

extremely large numbers of concurrent user

• Note: making sense of streams necessarily requires processing them against rich background knowledge, an unsolved problem in database





Concept Research Challenges

• Relation with DSMSs and CEPs– Just as RDF relates to data-base systems?

• Data types and query languages for semantic streams– Just RDF and SPARQL but with continuous semantics?

• Reasoning on Streams– Theory– Efficiency– Scalability

• Dealing with incomplete & noisy data– Even more than on the current Web of Data

• Distributed and parallel processing– Streams are parallel in nature, …

• Engineering Stream Reasoning Applications– Development Environment– Integration with other technologies– Benchmarks





Achievements• RDF Streams

– Notion defined• C-SPARQL

– Syntax and semantics defined as a SPARQL extension– Engine designed and implemented

• Experiments with C-SPARQL under simple RDF entailment regimes– window based selection of C-SPARQL outperforms the standard

FILTER based selection– algebraic optimizations of C-SPARQL queries are possible– Complex event can be detected using a network of C-SPARQL queries

at high throughputs• Experiment with C-SPARQL under RDFS++ entailment regimes

– efficient incremental updates of deductive closures investigated – our approach outperform state-of-the-art when updates comes as

stream• Streaming Linked Data Framework prototyped





AchievementsOutline

• RDF Streams– Notion defined

• C-SPARQL– Syntax and semantics defined as a SPARQL extension– Engine designed and implemented










Memo: Sensor Network Ontology

Stavanger, 2012-5-9• 20



Memo: Sensor Network Ontology[ ] streaming part [ ] static part




Achievements RDF Stream [WWW2009,EDBT2010,IJSC2010]

• RDF Stream Data Type– Ordered sequence of pairs, where each pair is made of

an RDF triple and its timestamp

Timestamps are not required to be unique, they must be non-decreasing

• E.g.,(< :s1 ssn:generatedObservation :o1 >, 2010-02-12T13:34:41) (< :o1 a weather:SnowfallObservation >, 2010-02-12T13:34:41) (< :s1 om-owl:generatedObservation :o2 >, 2010-02-12T13:36:28)(< :o2 a weather:WindSpeedObservation >, 2010-02-12T13:36:28)(< :o2 ssn:result :a1 >, 2010-02-12T13:36:28)(< :a1 ssn:floatValue "35.4”^^xsd:float >, 2010-02-12T13:36:28)


