how good is your sparql endpoint? a qos-aware sparql endpoint monitoring and data source selection...

How good is your SPARQL endpoint?

A QoS-aware SPARQL endpoint monitoring and data source

selection mechanism for federated SPARQL queries

Ali Intizar and Alessandra Mileo

28/10/2014 2

Linked Open Data and SPARQL Endpoints

•  Linked Data •  LOD cloud

28/10/2014 2


•  Linked Data •  LOD cloud

•  SPARQL Endpoints

28/10/2014 2


•  Linked Data •  LOD cloud •  SPARQL Endpoints •  Both pubic and private

28/10/2014 2


•  Linked Data •  LOD cloud •  SPARQL Endpoints •  Both pubic and private •  Allow easy access to linked data using SPARQL queries •  Provide a querying interface

28/10/2014 2



•  Open Data Management Tools

28/10/2014 2



•  Open Data Management Tools •  Datahub

28/10/2014



•  Open Data Management Tools •  Datahub •  LOD Stats

2

28/10/2014



•  Open Data Management Tools •  Datahub •  LOD Stats

•  SPARQL Endpoint Description •  Vocabulary for Interlinking Datasets •  Service Description

2

28/10/2014 3

Ranking of the SPARQL Endpoints

•  Multiple SPARQL endpoints can represent the same dataset

28/10/2014 3


•  Multiple SPARQL endpoints can represent the same dataset •  Which one is the best for me?

28/10/2014 3


•  Multiple SPARQL endpoints can represent the same dataset •  Ranking of the SPARQL endpoints

28/10/2014 4


•  Multiple SPARQL endpoints can represent the same dataset •  Ranking of the SPARQL endpoints •  Based on QoI/QoS Parameters

28/10/2014 5

QoS Parameters for SPARQL Endpoints

For QoS based ranking of the SPARQL endpoints •  Identification of the various QoS parameters associated with

the SPARQL endpoints

•  Semantic respresentation of the identified QoS parameters

•  Extension of the existing SPARQL endpoints description vocabularies (VoID/SD) to associate QoS parameters

•  Evaluation techniques for the QoS metrics

•  Continuous monitoring of the SPARQL endpoints to generate QoS profiles

28/10/2014


•  Performance

•  Response Time •  Execution Time •  Throughput •  Error Rate

6

28/10/2014 6


•  Performance


•  Data Quality

•  Accuracy •  Data Consistency •  Completeness •  Freshness

28/10/2014 7


•  Interoperabiilty

•  SPARQL Version •  Additional Features •  Restricted Features

28/10/2014 7




•  Availability

•  UpTime •  DownTime •  MeanUpTime •  MTTR

28/10/2014 8


•  Licensing •  PDDL •  ODC-By •  ODC-ODbL •  CC0 0.1 Universal

28/10/2014 8



•  ResultSet •  Size Limit •  Result Format

28/10/2014 8




•  Dataset Description •  VoID •  SD

28/10/2014 9








28/10/2014 10


•  Semantic Description of SPARQL Endpoint (VoID/SD) •  QoS Profile of SPARQL Endpoints

QoSProfile

Property SubClass

QoSProfileEndpoint QoSProfileDefault

QoSProfileUser

SPARQL Endpoint

has

28/10/2014 10


•  Semantic Description of SPARQL Endpoint •  QoS Profile of SPARQL Endpoints

QoSProfile

Property SubClass


QoSProfileUser

SPARQL Endpoint

has

28/10/2014 10


•  Semantic Description of SPARQL Endpoint •  QoS Profile of SPARQL Endpoints •  QoS Profile 1.  Endpoint 2.  Default 3.  User

QoSProfile

Property SubClass


QoSProfileUser

SPARQL Endpoint

has

28/10/2014 11


hasValue

QoSProfile

QoSParameter

Name

QoSCategory

Value QoSMetric

QoSWeight

Tendency

NonNumericMetricNumericMetricExactNumeric

IntervalNumeric

BooleanMetric

LinguisticMetric GradingMetric

No

HighMidLow

QoSUnit

Numeric Value

TextValue

Yes

hasvalue

hasvalue

hasvalue

hasvalue

hasvalue

endstart

hasName hasTendency

hasCategory

contains

hasWeight

hasMetric

isMeasuredIn

sameAs

28/10/2014 12








28/10/2014 13

Evaluation of the QoS Parameters

•  Performance

•  Response Time

Q1 . SELECT ?p where { <s> ?p <o> } Q2 . SELECT ?o where { s1 p1 ?o

s2 p2 ?o }

28/10/2014 14


•  Performance

•  Response Time •  Execution Time

Q1 . SELECT ?p where { <s> ?p <o> } Q2 . SELECT ?o where { s1 p1 ?o

s2 p2 ?o } Q3 . SELECT * where { ? s ?p ?o } LIMIT 1000

28/10/2014 15


•  Performance

•  Response Time •  Execution Time •  Throughput

Repeated execution of Q1.

28/10/2014 16


•  Performance


By putting the counter on the error returns by the SPARQL endpoint during the execution of the queries

28/10/2014 17




•  SPARQL 1.1 test data set

28/10/2014 18


•  Availability

•  UpTime •  DownTime •  MeanUpTime •  MTTR

•  We rely on the service provider for the provision of initial UpTime.

•  Periodic execution of query Q1 to monitor availability •  Started the counter of DownTime whenever Q1 failed •  MeanUpTime calculated as percentage of the time

SPARQL endpoint was available since its initial UpTime. •  Mean Time To Recover (MTTR) is calculated as average

time taken by SPARQL endpoint to recover after failure.

28/10/2014 19



Q6 .

PREFIX dcterms : <http://purl.org/dc/terms/> SELECT ?license WHERE { ?ds a void:Dataset . ?ds dcterms:license ?license . }

28/10/2014 20


•  Dataset Description •  Vocabulary for Interlinking Datasets •  Service Description

Q4 . PREFIX void : <http://rdfs.org/ns/void#> SELECT ?ds WHERE { ?ds a void:Dataset . ?ds void:SPARQLEndpoint

<SPRQLEnpointURI> }

28/10/2014 21



Q5 . PREFIX sd: <http://www.w3.org/ns/sparql-service-description#>

SELECT ?format WHERE { ?s a sd:service . ?s sd:endpoint <endpointURI> . ?s sd:resultFormat ?format . }

28/10/2014 22


•  Data Quality

•  Accuracy •  Data Consistency •  Completeness •  Freshness

Data quality is an overlap between quality of information(QoI) and quality of service(QoS)

28/10/2014 23








28/10/2014 24

Monitoring QoS metrics of SPARQL endpoints

•  Montioring •  Runtime •  Periodic

28/10/2014 24

Monitoring QoS metrics of SPARQL endpoints

•  Montioring •  Runtime •  Periodic

•  QoS Profile Generation

28/10/2014 25

Federated SPARQL Queries

•  SPARQL 1.1 extension provides SERVICE keyword •  Allows remotely execution of the SPARQL queries on several

endpoints

Federated SPARQL Query Engine

Source Selection

Indexing/Caching

Query ExecutionOptimiser

SPARQLEndpoint …SPARQL

EndpointSPARQLEndpoint

SPARQLEndpoint

SPARQL

28/10/2014 26


•  Problem of data source selection •  Automated discovery and execution of the SPARQL endpoints

for any federated query.


Source Selection

Indexing/Caching

Query ExecutionOptimiser



SPARQLEndpoint

SPARQL

28/10/2014 27




•  Candidate Data Sources: “Given a user's query Q and set of n data sources

DS = { dsi | i =1..n} , we define set of candidate data sources as

DSc = { dscj | j = 1..m } that can potentially contribute to answer query Q, where

DSc ⊆ DS and 1 ≤ m ≤ n . “

28/10/2014 28




•  QoS Aware Data Sources: “Given a set of candidate data sources DSc, we define set of QoS aware data sources as

DSqos = { dsqosk | k = 1..l } as set of optimal data sources that can potentially contribute to the answer

of the Query Q and are compliant with the QoS requirements mentioned in the query, where DSqos ⊆ DSc and 1 ≤ l ≤ m ≤ n . “

28/10/2014 29

QoS-Aware Federated SPARQL Queries


QoS Aware Query Parser

Source Selection

Indexing/Caching

QoS Aware Source Selection

QoS Repository

QoS Evaluation

Query Execution

Optimiser



SPARQLEndpoint

User Query& QoS SPARQL

DSc

DSqos

QoSProfile

Monitoring

28/10/2014 30

SPARQL Extension with QoS

•  QoS requirements can be described as part of the SPARQL query

•  We introduce a new QOSREQ keyword in the SPARQL query language

•  QOSREQ operator is applied to the triple pattern or BGP immediarly proceeding the operator

•  Comma separated values of multiple QoS parameters within QOSREQ operator

•  Comparison operators to compare the user defined QoS requirements with QoS profile of the SPARQL endpoint

28/10/2014 30

SPARQL Extension with QoS

•  QoS requirements can be described as part of the SPARQL query

SELECT ?drug ?keggUrl ?chebiImage WHERE { ?drug rdf:type drugbank:drugs .

QOSREQ[ qs:ResponseTime < 10 , qs:SizeLimit > 10000] ?drug drugbank:keggCompoundId ?keggDrug . ?keggDrug bio2rdf:u r l ?keggUrl . { ?drug drugbank:genericName ?drugBankName . ?chebiDrug purl:title ?drugBankName . }

QOSREQ[ qs:DatasetDescription = 'VoID' , qs:MeanUpTime > 80 ]

?chebiDrug chebi:image ?chebiImage . }

28/10/2014 31

QoS-Aware Federated SPARQL Queries


QoS Aware Query Parser

Source Selection

Indexing/Caching

QoS Aware Source Selection

QoS Repository

QoS Evaluation

Query Execution

Optimiser



SPARQLEndpoint

User Query& QoS SPARQL

DSc

DSqos

QoSProfile

Monitoring

28/10/2014 32

Experimental Evaluation •  FedBench Benchmark •  A benchmark suite for federated SPARQL queries

evaluation •  Provides various data sets from Life Sciences, Linked

Data and Cross Domains •  25 queries to evaluate the performance

•  Testbed •  Datasets are deployed as SPARQL endpoints •  Multiple Copies of the data sets to create higher number of

candidate data sources •  Human intervention to create fluctuation •  Montioring of the SPARQL endpoints for more than 2 months •  QoS Profiles generation and updates in QoS metrics values based

on continuous monitoring

28/10/2014 33

Experimental Evaluation

0"

5"

10"

15"

20"CD

1"CD

2"CD

3"CD

4"CD

5"CD

6"CD

7"LS1"

LS2"

LS3"

LS4"

LS5"

LS6"

LS7"

LD1"

LD2"

LD3"

LD4"

LD5"

LD6"

LD7"

LD8"

LD9"

LD10"

LD11"

No."of"Selected"Da

ta"Sou

rces"

FedX?QBenchmark" Splendind?QBenchmark"

QoSAware?QDefault" QoSAware?Quser"

28/10/2014 34

Experimental Evaluation

0"

5"

10"

15"CD

1"CD

2"CD

3"CD

4"CD

5"CD

6"CD

7"LS1"

LS2"

LS3"

LS4"

LS5"

LS6"

LS7"

LD1"

LD2"

LD3"

LD4"

LD5"

LD6"

LD7"

LD8"

LD9"

LD10"

LD11"

No."of"Selected"Da

ta"Sou

rces"

Q_50" Q_75" Q_100"

28/10/2014 35

Conclusion

•  Identification and semantic representation of the QoS parameters of the SPARQL endpoints

•  QoS metrics evaluation mechanism

•  A monitoring Service for QoS Evaluation

•  SPARQL extension for users QoS requirements within query language

•  QoS-Aware Federated SPARQL query evaluation

28/10/2014 35

Future Work

•  QoS monitoring over public SPARQL endpoints & integration with SPARQLES

•  Sophisticated mechanisms for Quality of Information evaluation

•  Taking QoS requirements as well preferences into account (Hard and Soft Constraints)

•  QoS aggregated values

how good is your sparql endpoint? a qos-aware sparql endpoint monitoring and data source selection...

Data & Analytics