searching the web of data (tutorial)

!"#$%&'()*+&"*,"-*./*0#+#!"#$%&'()'#)*+,-)./012)3$45$6

1"$#$2*2"*3"4.56!5*7"$8"4"9

!""#$%%&'()*+*,-%-./%

:#+;#*<.=">#4-.$)*?('@"$='+9

!""#$%%000+12"324!*5'+.'%

AB$*CB+.$'#4*C.2#9

! 7589:"(9;)0/;//<01;1/

! +$==99)>%9'?;)00;1/<0.;//

! @"94#&$A4;)BAC)#&D9

! 7(&:94;)E&(()F9)'G'&('F(9)$A(&A9

C&"*D4#(

! 5(+$.2B%+'.(

! !+$B%+B$"2*0#+#*.(*+&"*,"-

! EB"$9'()*F'(8"2*AG"(*0#+#

! EB"$9'()*+&"*,"-*./*0#+#

! EB"$9*?(2"$=+#(2'()

! AB+4..8*#(2*6.(%4B='.(

5(+$.2B%+'.(

! 5(GB+H)H9C6$%:42)I$54

! 0"='$"2*AB+GB+HI$5"D9A#)-'A?&AJ

5(+$.2B%+'.(

! 5(GB+H)H9C6$%:42)#89)E$%(:K4)I'#'

! C$B"*0"='$"2*AB+GB+HB::%944)L49%K4)M99:4

?="$I=*J""2=

M99:AK#)F9)N('&A)&A=$%D'#&$A)A99:4

?="$I=*J""2=

M99:AK#)F9)J9A"&A9)O&A=$%D'#&$AP)A99:4

AB$*K.%B=H*5(/.$L#+'.(*J""2=

0#+#M0$'@"(*>(=N"$=

0#+#M0$'@"(*>(=N"$=#22$"=='()*.+&"$*5(+"(+=

C&"*,"-*#=*#*0#+#-#="H*O(+'+'"=

C&"*,"-*#=*#*0#+#-#="H>++$'-B+"=PQ"4#+'.(=&'G=

Q$F)!&#(9R$5'#&$A7#'%#&AJ)I'#97'('%C9#5S

M9649'%585'N'F&(&#&94

C&"*D4#(

! 5(+$.2B%+'.(

! !+$B%+B$"2*0#+#*.(*+&"*,"-

! EB"$9'()*F'(8"2*AG"(*0#+#

! EB"$9'()*+&"*,"-*./*0#+#

! EB"$9*?(2"$=+#(2'()

! AB+4..8*#(2*6.(%4B='.(

Searching the Web of Data

Outline I

1 Structured data on the WebSemantic markupSemantic Web and Linked Open DataData management

2 Querying Linked Open DataBrowser-based link traversalKeyword search for Linked DataStructured queries

Katja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 2 / 73

Structured data on the Web

Semantic Markup (metadata, tags) embedded in HTMLMicroformats, hCard, hCalendar, RDFa

Knowledge basesLarge collections of RDF data

Linked (Open) DataReferences between collections of RDF data

Semantic markup

If the search engine gets some help in better “understanding” the contentof a web page, rich snippets highlighting and displaying certain informationcan be created.

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=99170

Semantic markup

Microformats

Efforts started in 2003

Fixed formats for specific type of informationhCard: people, companies, organizations, and placeshCalendar: calendaring and eventshReview: reviews of products, companies, events. . .

Cannot represent arbitrary data

Indexed by Google and Yahoo since 2009

Semantic markup

hCard example<div>

<img src="www.example.com/bobsmith.jpg" /><strong>Bob Smith</strong>Senior editor at ACME Reviews200 Main StDesertville, AZ 12345

</div>

<div class="vcard"><img class="photo" src="www.example.com/bobsmith.jpg" /><strong class="fn">Bob Smith</strong><span class="title">Senior editor</span> at<span class="org">ACME Reviews</span>

<span class="adr"><span class="street-address">200 Main St</span><span class="locality">Desertville</span>,<span class="region">AZ</span><span class="postal-code">12345</span>

</span></div> http://support.google.com/webmasters/bin/answer.py?hl=en&answer=146897

Semantic markup

Proposed in 2004, W3C recommendation

Can be used together with any vocabulary (no restriction on schema)

Can assign URIs as global primary keys to entities

Semantic markup

RDFa example

<div>My name is Bob Smith but people call me Smithy. Here is my home page:<a href="http://www.example.com">www.example.com</a>.I live in Albuquerque, NM and work as an engineer at ACME Corp.

</div>

<div xmlns:v="http://rdf.data-vocabulary.org/#" typeof="v:Person">My name is <span property="v:name">Bob Smith</span>,but people call me <span property="v:nickname">Smithy</span>.Here is my homepage:<a href="http://www.example.com" rel="v:url">www.example.com</a>.I live in Albuquerque, NM and work as an

<span property="v:title">engineer</span>at <span property="v:affiliation">ACME Corp</span>.

</div>

Semantic markup

Facebook’s Open Graph Protocol

Introduction of “like” buttons in 2010Allows site owners to determine how entities are displayed in FacebookRelies on RDFa for encoding data in HTML pages

http://developers.facebook.com/docs/opengraphprotocol/

Semantic markup

Microdata

Proposed in 2009 as part of HTML5

Alternative technique for embedding structured data

Tries to be simpler than RDFa

Semantic markup

Microdata

<div>My name is Bob Smith but people call me Smithy. Here is my home page:<a href="http://www.example.com">www.example.com</a>I live in Albuquerque, NM and work as an engineer at ACME Corp.

</div>

<div itemscope itemtype="http://data-vocabulary.org/Person">My name is <span itemprop="name">Bob Smith</span>but people call me <span itemprop="nickname">Smithy</span>.Here is my homepage:<a href="http://www.example.com" itemprop="url">www.example.com</a>I live in Albuquerque, NM and work as an

<span itemprop="title">engineer</span>at <span itemprop="affiliation">ACME Corp</span>.

</div>

Semantic markup

schema.org

“Standardized” vocabulary 2011, supported by Bing, Google, Yahooand YandexAsk site owners to embed data to enrich search results200+ types: event, organization, person, place, product, review,. . .Encoding: basically microdata (RDFa)Main usage: highlighting and enriching data snippets in search results

Semantic markup

Embedded data in HTML

RDFa and Microdata usage grows, microformats are still present

A rather small set of vocabularies is used

The content and the vocabularies are very focused towards the majorconsumers

Semantic Web and Linked Open Data

Google Knowledge Graph

Currently1, more than 500 million entities, such as celebrities, cities,movies,. . .

Consists of commercial third-party data and Web data

Enriching search results with summaries

Is increasingly being used by Google to answer queries

1Google Official Blog http://googleblog.blogspot.co.uk/2012/05/introducing-knowledge-graph-things-not.html

Semantic Web

��

� ��

http://www.slideshare.net/lod2project/the-semantic-data-web-sren-auer-university-of-leipzig

Semantic Web

Web 1.0 (standard Web)Web 2.0 (data generated by users)Web 3.0 (Semantic Web)Machine-readable dataURIs for documents and concepts (entities)“Web of data”

Web server & Web 1.0:

http://www.slideshare.net/cloudofdata/toward-the-data-cloud

Standards

Sharing structured data across the Web relies on standards

Standard graph-based data modelRDF

Different syntaxes and formatsRDF/XML, RDFa

Powerful, logic-based schema languages and reasoningOWL

Query languages and protocolsHTTP, SPARQL

Linked Open Data: design issues

Design issues (rules)

1 Use URIs as names for things

2 Use HTTP URIs so that people can look up those names

3 When someone looks up a URI, provide useful information, using thestandards (RDF, SPARQL)

4 Include links to URIs in other datasetsGoal: linking URIs in different data sets describing the same realworld entity

Growth: Semantic Web and Linked Open Data

May 2007

March 2008

July 2009

September 2011

Data management

Knowledge bases

Large collections of semantic data

YAGO [9], Freebase [10], DBpedia [2],. . .

Mostly result of information extraction

Data format in general RDF

Often participate as sources in the Linked Open Data cloud

Data management

Each resource (entity) is identified by a globally unique URI

Data is stored in the form of facts

Triple: (subject, property, object)

Subject: URI

Predicate/Property: URI

Object: URI or literal (strings, integers, booleans, etc.)

http://dbpedia.org/resource/Aalborg

http://dbpedia.org/ontology/country

http://dbpedia.org/resource/Denmark

Data management

Each resource (entity) is identified by a globally unique URI

Data is stored in the form of facts

Triple: (subject, property, object)

Subject: URI

Predicate/Property: URI

Object: URI or literal (strings, integers, booleans, etc.)

Using prefixesdbpedia:Aalborg

dbpedia-owl:country

dbpedia:Denmark

(dbpedia:Aalborg, dbpedia-owl:country, dbpedia:Denmark)Katja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 25 / 73

Data management

Triples connect to graphs

dbpedia-owl:country

dbpedia:Aalborg

dbpedia-owl:country

dbpedia:Denmark

dbpedia-owl:isPartOf

dbpedia:North_Denmark_Region

123432

dbpedia:populationTotal

Data management

Triples connect to graphs. . . and possibly other sources

dbpedia-owl:country

dbpedia:Aalborg

dbpedia-owl:country

dbpedia:Denmark

123432

yago:Denmark

geonames:2624886

Data management

Schema

The schema (vocabulary, ontology) can be expressed in OWL

Definition of classes, properties, restrictions,. . .

Allows for validation and reasoning

Schema information is also represented as RDF triples

Data management

Managing large amounts of RDF data

Relational RDF data management

A single relational tableThree columns (subject, property, object)

Property tablesn-ary table columns for the same subject

Binary tablesOne two-column table for each property

Data management

A single relational table

Example triples

(dbpedia:Aalborg, dbpedia-owl:country, dbpedia:Denmark)(dbpedia:Aalborg, dbpedia-owl:isPartOf dbpedia:North Denmark Region)(dbpedia:North Denmark Region, dbpedia-owl:country, dbpedia:Denmark)(dbpedia:Aalborg, dbpedia-owl:populationTotal, 123432)

subject property object

dbpedia:Aalborg dbpedia-owl:country dbpedia:Denmarkdbpedia:Aalborg dbpedia-owl:isPartOf dbpedia:North Denmark Region

dbpedia:North Denmark Region dbpedia-owl:country dbpedia:Denmarkdbpedia:Aalborg dbpedia-owl:populationTotal 123432

Works with standard relational DBMS and SQL

Problems: self joins, query optimization

Data management

A single relational table

Example triples

subject property object

dbpedia:Aalborg dbpedia-owl:country dbpedia:Denmarkdbpedia:Aalborg dbpedia-owl:isPartOf dbpedia:North Denmark Region

dbpedia:North Denmark Region dbpedia-owl:country dbpedia:Denmarkdbpedia:Aalborg dbpedia-owl:populationTotal 123432

Works with standard relational DBMS and SQL

Problems: self joins, query optimization

Data management

Property tablesExample triples

citysubject country isPartOf populationTotal

dbpedia:Aalborg dbpedia:Denmark dbpedia:North Denmark Region 123432

regionsubject country

dbpedia:North Denmark Region dbpedia:Denmark

Grouping information about entities with similar properties

n-ary tables for the same subject

Difficult to create a proper layout

Null values

Problems with multi-valued attributes

Data management

Property tablesExample triples

dbpedia:Aalborg dbpedia:Denmark dbpedia:North Denmark Region 123432dbpedia:Kassel dbpedia:Germany 195530

Null values

Problems with multi-valued attributesKatja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 30 / 73

Data management

Property tables

dbpedia:Aalborg dbpedia:Denmark dbpedia:North Denmark Region 123432dbpedia:Kassel dbpedia:Germany 195530

Null values

Problems with multi-valued attributes

Data management

Binary tablesExample triples

dbpedia-owl:countrysubject object

dbpedia:Aalborg dbpedia:Denmarkdbpedia:North Denmark Region dbpedia:Denmark

dbpedia-owl:isPartOfsubject object

dbpedia:Aalborg dbpedia:North Denmark Region

dbpedia-owl:populationTotalsubject object

dbpedia:Aalborg 123432

Create a seperate table for each property

Can become inefficient for queries involving many common properties

Becomes inefficient if there are too many different propertiesKatja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 31 / 73

Data management

Binary tables

dbpedia-owl:countrysubject object

dbpedia:Aalborg dbpedia:Denmarkdbpedia:North Denmark Region dbpedia:Denmark

dbpedia-owl:isPartOfsubject object

dbpedia:Aalborg dbpedia:North Denmark Region

dbpedia-owl:populationTotalsubject object

dbpedia:Aalborg 123432

Create a seperate table for each property

Can become inefficient for queries involving many common properties

Becomes inefficient if there are too many different properties

Data management

Native triple stores

RDF-3X [7]

Dictionary encoding to reduce storage space

Extensive use of B+-tree indexes(SPO, OPS, PSO, SOP, OSP, POS)Aggregated indexes: S, P, O, SP, SO, PO, PS, OP, OS

Triples are materialized in the indexes

Histograms provide the query optimizer with further statistcis

Data management

Column stores

SW-Store [1]

Binary tables in combination with a column-oriented DBMS (C-store)

Sorted tables

Supports multi-valued attribues (listed in a successive row)

Increased costs for updates and tuple reconstruction

Data management

More alternatives

Store RDF data in a matrix with bit-vector compression

Store RDF as XML and use XML technology

Graph databases

Querying Linked Open Data

Browser-based link traversal

Who is Carlo Pedersoli?

Browser-based link traversal is the most “natural” way of looking upinformation using Linked Data

Might be very tedious and frustrating

Takes much time

But you will discover much information that you never intended tosearch for

Keyword search for Linked Data

Given a set of keywords, find all relevant information.

Falcons

Focus: entities

Crawling

Follow links contained in RDF documents

Construct virtual documents for entitiesLiteralsHuman-readable names and descriptions (rdfs:label, rdfs:comment)

Create indexesTerms in virtual documentsEntity classes

Falcons

Query processing

Keywords

Filtering based on classes/types

Output: entities with snippets

Falcons

http://ws.nju.edu.cn/falcons/objectsearch/index.jsp

Falcons

Sindice

Original focus: documents and sources

Indexes

URI (Unified Resource Identifier)

IFP (Inverse Functional Properties)

Literal

Sindice

http://sindice.comKatja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 45 / 73

Sindice

http://sindice.comKatja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 45 / 73

Indexing

Indexes on virtual documents

Creating virtual documents based on the data (entity,source,triple,. . . )

Create inverted indexes on URIs, properties, classes, literals,. . . orcombinations

Structured queries

What are the movies that both Carlo Pedersoli (Bud Spencer) andMario Girotti (Terence Hill) acted in?Or what are the movies that only one of them (without the other)acted in?

Structured queries

Structured query language

SPARQL

Query processing strategies

Materialized query processing

Lookup-based query processing

Federated query processing

Structured queries

SPARQL

Example query

PREFIX dbpedia: <http://dbpedia.org/property/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>SELECT ?city, ?pop WHERE {?city dbpedia-owl:country dbpedia:Denmark .?city dbpedia:populationTotal ?pop .FILTER (?pop > 100000)

SPARQL

Similar to SQL

Variables start with “?”

Queries consist of triple patterns, e.g.:?city dbpedia-owl:country dbpedia:Denmark

Joins between triple patterns are expressed by common variables

Filters express additional constraintsKatja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 50 / 73

Structured queries

SPARQL

Example query

SPARQL

Similar to SQL

Structured queries

SPARQL

Example query

SPARQL

Similar to SQL

Structured queries

SPARQL

dbpedia-owl:country

dbpedia:Denmark

?pop ?city

dbpedia-owl:country

dbpedia:Aalborg

dbpedia-owl:country

dbpedia:Denmark

123432

yago:Denmark

geonames:2624886

Structured queries

SPARQL

dbpedia-owl:country

dbpedia:Denmark

?pop ?city

dbpedia-owl:country

dbpedia:Aalborg

dbpedia-owl:country

dbpedia:Denmark

123432

yago:Denmark

geonames:2624886

Structured queries

Characteristics

Centralized storage

Crawl and download the data

Evaluate queries locally

As an alternative to evaluating queries on huge data sets on a singlemachine, we can make use of distributed architectures and parallelprocessing, e.g., MapReduce, NoSQL, P2P, Grid,. . . .

Problem

Hash partitioning is not optimal for complex queries

Possible solution

Clustered RDF management using graph partitioning [6]

Structured queries

Characteristics

Centralized storage

Problem

Possible solution

Structured queries

Characteristics

Centralized storage

Problem

Possible solution

Structured queries

Clustered RDF management

Data graph

Structured queries

Graph partitioning

Structured queries

Assigning triples to partitions – 1-hop guarantee

Structured queries

2-hop guarantee

Structured queries

Query execution is more efficient in RDF stores than in Hadoop [6]

Pushing as much of the processing as possible into RDF stores

Minimizing the number of Hadoop jobs

The larger the hop guarantee, the more work is done in RDF stores

Query processing

Choose center of the query graph

Calculate distance from the center to the furthest edge

If distance <= n: query can be handled by nodes independentlywithout communication

If distance > n: communication is needed, split up into smallersubqueries, Hadoop

Structured queries

Query execution is more efficient in RDF stores than in Hadoop [6]

Pushing as much of the processing as possible into RDF stores

Minimizing the number of Hadoop jobs

The larger the hop guarantee, the more work is done in RDF stores

Query processing

Choose center of the query graph

Calculate distance from the center to the furthest edge

If distance <= n: query can be handled by nodes independentlywithout communication

If distance > n: communication is needed, split up into smallersubqueries, Hadoop

Structured queries

Find football players playing for clubs in a region where they were born

SELECT ?player ?club ?region WHERE {?player rdf:type ex:footballer .?player ex:playsFor ?club .?player ex:bornIn ?region .?club ex:region ?region .?region ex:population ?pop .

ex:footballer?pop

?player

?region

rdf:type

ex:bornInex:population

ex:playsFor

ex:region

Structured queries

SELECT ?player ?club ?region WHERE {?player rdf:type ex:footballer .?player ex:playsFor ?club .?player ex:bornIn ?region .?club ex:region ?region .?region ex:population ?pop .

ex:footballer?pop

?player

?region

rdf:type

ex:playsFor

ex:region

Structured queries

ex:footballer?pop

?player

?region

rdf:type

ex:playsFor

ex:region

Structured queries

ex:footballer?pop

?player

?region

rdf:type

ex:playsFor

ex:region

Structured queries

If the query is too “big”, the query is decomposed into multiple smallersubqueries, and the results are combined using MapReduce.

ex:citesex:wr

ittenBy

ex:hasTitle

ex:writtenBy

ex:hasName

isOwned

?name2?author2

?title2

?name1?author1

?art2?art1

Workload-Aware Replication [5]

Replicate additional queries at the boundaries

Avoid MapReduce by using a designated coordinator node

Structured queries

If the query is too “big”, the query is decomposed into multiple smallersubqueries, and the results are combined using MapReduce.

ex:citesex:wr

ittenBy

ex:hasTitle

ex:writtenBy

ex:hasName

isOwned

?name2?author2

?title2

?name1?author1

?art2?art1

Workload-Aware Replication [5]

Replicate additional queries at the boundaries

Avoid MapReduce by using a designated coordinator node

Structured queries

Lookup-based query processingNo statistics or indexesEvaluate parts of the query locallyDereference URIs in intermediate solutions, download the dataUse downloaded data to compute other parts of the query,dereference. . .

Federated query processingBased on technologies originally developed for distributed databasesystems, P2P systems, and data integrationData is storedEvaluate parts of the query on remote sources

Structured queries

Lookup-based query processingNo statistics or indexesEvaluate parts of the query locallyDereference URIs in intermediate solutions, download the dataUse downloaded data to compute other parts of the query,dereference. . .

Federated query processingBased on technologies originally developed for distributed databasesystems, P2P systems, and data integrationData is storedEvaluate parts of the query on remote sources

Structured queries

Federated SPARQL query processing [8]

SPARQL Request Query Result

Parsing Source Selection Query Execution(Bound Joins)

Global Optimizations(Groupings + Join Order)

SPARQLEndpoint 1 . . .

Subquery Generation:Evaluation at

Relevant Endpoints

LocalAggregation ofPartial ResultsCache

Per Triple Pattern

SPARQL ASK queriesSPARQL

Endpoint 2SPARQL

Endpoint N

Assumption

The sources are capable (and willing) to evaluate SPARQL queries(SPARQL endpoints).

Structured queries

Source selection

Identify sources that might contribute to the query result

Approaches

SPARQL ASK requests and cachingStatistics and indexes

Keyword indexesPredicate URIs, types of instancesURI indexesFrequent pathsService-level descriptionsVoiD statisticsHistograms. . .

Structured queries

Source selection

Approaches

Structured queries

Naive federated query processing

Scenario

SPARQL endpoints

No statistics/indexes

Example query

SELECT ?Country ?Capital?CountryPop ?CapitalPop WHERE {

?Country ex:capital ?Capital .?Country ex:population ?CountryPop .?Capital ex:population ?CapitalPop .

Structured queries

Scenario

SPARQL endpoints

Example query

Structured queries

Scenario

SPARQL endpoints

Example query

Structured queries

Example query

Send message with triple patternto all 4 sources → 4 requests

Receive 200 mappings for?Country and ?Capitale.g., ?Country=ex:Germany,

?Capital=ex:Berlin

Structured queries

Example query

Use results to evaluate 2ndtriple pattern (nested loop)

200× 4 requestse.g., SELECT ?CountryPop WHERE

{ex:Germany ex:population

?CountryPop .}

150 mappings

Structured queries

Example query

Use results to evaluate 3rd triplepattern (nested loop)

150× 4 requestse.g., SELECT ?CapitalPop WHERE

{ex:Berlin ex:population

?CapitalPop .}

Structured queries

Example query

In total:4 + 200× 4 + 150× 4 = 1404requests

Many (unnecessary) requests sent to the sources!

Structured queries

Source selection

Approaches

Structured queries

Source selection with SPARQL ASK

Does not require special cooperation from the sources

Example query

Each source receives a message for each triple patternResponse: true/false

ASK {?Country ex:capital ?Capital .

ASK {?Capital ex:population ?CapitalPop .

ASK {?Country ex:population ?CountryPop .

Structured queries

Example query

Structured queries

Example query

Structured queries

Source selection

Approaches

Structured queries

Source selection with VoID statistics

Example DBpedia2

Prefixes@prefix owl: http://www.w3.org/2002/07/owl#.@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .@prefix dbpedia: <http://dbpedia.org/resource/> .@prefix dbpo: <http://dbpedia.org/ontology/> .

General information

Basic statistics

Predicate statistics

Class statistics

2http://code.google.com/p/fbench/source/browse/trunk/EvalBenchmark/suites/SPLENDID/void/

dbpedia3.5.1_subset-void.n3?r=119

Structured queries

Example DBpedia2

Prefixes

General informationvoid:sparqlEndpoint <http://dbpedia.org/sparql> ;

Basic statistics

Class statistics

Structured queries

Example DBpedia2

Prefixes

General information

Basic statisticsvoid:triples "43620475" xsd:integer ;void:entities "2222456" xsd:integer ;void:properties "1063" xsd:integer ;void:distinctSubjects "9495865" xsd:integer ;void:distinctObjects "13636604" xsd:integer ;

Class statistics

Structured queries

Example DBpedia2

PrefixesGeneral informationBasic statisticsPredicate statisticsvoid:propertyPartition [void:property dbpo:aSide ;void:triples "1600" xsd:integer ;void:distinctSubjects "1552" xsd:integer ;void:distinctObjects "1554" xsd:integer] , [void:property dbpo:abbreviation ;void:triples "1144" xsd:integer ;void:distinctSubjects "1141" xsd:integer ;void:distinctObjects "1096" xsd:integer] , [...];

Class statistics2http://code.google.com/p/fbench/source/browse/trunk/EvalBenchmark/suites/SPLENDID/void/

Structured queries

Example DBpedia2

Prefixes

General information

Basic statistics

Class statisticsvoid:classPartition [void:class dbpo:Activity ;void:entities "1234" xsd:integer] , [void:class dbpo:Actor ;void:entities "37898" xsd:integer] , [...]

Structured queries

Source selection

Approaches

Structured queries

Histogram-based indexing

Index complete triples

Index (s,p,o) in combinations capturing correlation

Based on multidimensional histograms

Identify relevant sources for triple patterns

Identify relevant sources for joins

Structured queries

Histograms

Transform triple statements into numerical space (hash functions)(ex:BudSpencer ex:actedIn ex:m1234) → (323, 232, 124)

Insert into the matching bucket

Lookup: transform triple pattern into numerical space(ex:BudSpencer ex:actedIn ?m)

Structured queries

Histograms

Structured queries

Histograms

so 1515 1610 00 00

Structured queries

Histograms

so 1515 1610 00 00 s

o 1515 1610 00 00Katja Hose Searching the Web of Data March 24, 2013 – ECIR 2013 69 / 73

Structured queries

Histograms

Alternative to histograms (QTree or clustering) [11]:

so A1A2 A

Structured queries

Join cardinality estimation and source selection

Determine buckets for 1st triple pattern

Determine buckets for 2nd triple pattern

Determine buckets that overlap in the join dimension (e.g., subject)

Estimate cardinality based on the degree of overlap

1st BGP

2nd BGP

subject

Summary: querying Linked Open Data

Querying Linked Open Data (LOD)

Browser-based link traversalMost natural way of looking up Linked Data

Keyword search for Linked Open DataSeveral search engines available coming in different flavors

SPARQL query processingMaterialized query processingLookup-based query processingFederated query processing

Relies on powerful and available sourcesStatistics require additional cooperation

References I

[1] Daniel J. Abadi, Adam Marcus, Samuel Madden, and Kate Hollenbach. SW-Store:a vertically partitioned DBMS for Semantic Web data management. VLDB J.,18(2):385–406, 2009.

[2] Soren Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak,and Zachary G. Ives. DBpedia: A nucleus for a Web of open data. In ISWC, 2007.

[3] Christian Bizer. Topology of the Web of Data, 2012. Keynote LWDM 2012.http://www.wiwiss.fu-berlin.de/en/institute/pwo/bizer/research/publications/Bizer-TopologyWoD-LWDM2012-BEWEB2012.pdf?1361005355.

[4] Gianluca Demartini, Peter Mika, Thanh Tran, and Arjen P. de Vries. From ExpertFinding to Entity Search on the Web, 2012. Tutorial ECIR 2012.http://diuf.unifr.ch/main/xi/EntitySearchTutorial.

[5] Katja Hose and Ralf Schenkel. WARP: Workload-Aware Replication andPartitioning for RDF. In DESWEB’13, 2013.

[6] Jiewen Huang, Daniel J. Abadi, and Kun Ren. Scalable SPARQL Querying of LargeRDF Graphs. PVLDB, 4(11):1123–1134, 2011.

References II

[7] Thomas Neumann and Gerhard Weikum. RDF-3X: a RISC-style engine for RDF.PVLDB, 1(1):647–659, 2008.

[8] Andreas Schwarte, Peter Haase, Katja Hose, Ralf Schenkel, and Michael Schmidt.Fedx: Optimization techniques for federated query processing on linked data. InISWC 2011, pages 601–616, 2011.

[9] Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: a core ofsemantic knowledge. In WWW, 2007.

[10] Metaweb Technologies. The freebase project. http://freebase.com.

[11] Jurgen Umbrich, Katja Hose, Marcel Karnstedt, Andreas Harth, and Axel Polleres.Comparing data summaries for processing live queries over linked data. World WideWeb, 14:495–544, 2011.

C&"*D4#(

! 5(+$.2B%+'.(

! !+$B%+B$"2*0#+#*.(*+&"*,"-

! EB"$9'()*F'(8"2*AG"(*0#+#

! EB"$9'()*+&"*,"-*./*0#+#

! EB"$9*?(2"$=+#(2'()

! AB+4..8*#(2*6.(%4B='.(

>++$'-B+"M7#="2*O(+'+9*!"#$%&

1$#G&M7#="2*O(+'+9*!"#$%&

1$#G&M7#="2*O(+'+9*!"#$%&TBUV.)>%$649%;)O!"#$"%$&'()*#+,%-+.+%$.+)/+0%01.#*2%

/"+%3),/%$+*/1.4%#*%5#3#$&*%6)33+4%!"

"#$$%&'!('!%)*!+,-../*!000!,-..!1(2'!3(4#!56%&7

1$#G&*EB"$9*D$.%"=='()

! 7D'%#)&A:9W&AJ)$=)#%&N(94

! Q$&A)V%:9%)VN#&D&X'#&$A;)'(()N9$N(9)G4S)'(()M$F9()N%&X9)6&AA9%4

-IY1Z)[M9"D'AA)9#)'(S).//\]OY&A:)M$F9()^%&X9)6&AA9%4)68$)4"%G&G9:)F$#8)6$%(:)6'%4)'A:)#89&%)$6A)58&(:P

1$#G&*EB"$9*Q"4#R#+'.(

! D$.-4"LH)7#%"5#"%9:)I'#')$=#9A)&A5$DN(9#9

! !.4B+'.(H)+'%9="()@"9%C)-9('W'#&$A9SJS)F$%A,A)_)5&#&X9AV=2)(&G94,A6$AB6'%:)_)A$D&A'#9:Y$%2)48$%#R&4#9:Y$%

! D.=='-4"*>GG$.#%&H)R'AJ"'J9)3$:9(4[*(F'44"$A&)9#)'(S)*7E+)./00]

0"2BG4'%#+'.(

>'($J)[,Q+B,)R`I)E$%?48$N)./00];9W#%'5#)N%$:"5#)'##%&F"#94)#$)&DN%$G9)N%$:"5#):9:"N(&5'#&$A

0"2BG4'%#+'.(

! F5J0>*S7T&L*"+*#4U*65:3*VWXVY

! B"#$D'#&5'((C)5%9'#9)A96)(&A?468&(9)'55$"A#&AJ)=$%:9N9A:9A5&94

! 3'N-9:"59<F'49:'(J$%&#8D)=$%)45'('F&(&#C

0"2BG4'%#+'.(

:9)39($)a)E9&?"D)[./0/]S)LA#'AJ(&AJ)#89)+%$44<R&AJ"'()R&A?)7#%"5#"%9)$=)E&?&N9:&'S)B+R)./0/

>':)9A#&#C)(&A?4)(9':)#$)&A5$A4&4#9A#9b"&G'(9A59)5('4494

_)L49):&4#&A5#A9449G&:9A59)#$)49N'%'#9:&4#&A5#)9A#&#&94

0"2BG4'%#+'.(*OR#LG4"HF"R@.U.$)

:9)39($)a)E9&?"D)[,7E+).//\]

R'AJ"'J9475%&N#4+$"A#%&94+8'%'5#9%4E$%:4E$%:)79A494*#5S

`"F)&A)I&J&#'()R&F%'%&94)'A:)R&A?9:)R&AJ"&4#&5)I'#')5($":4

0"2BG4'%#+'.(

I'(#$A2)3&?')a)>('A5$)[+,H3)./00];

-'A?&AJ)A99:4)#$)#'?9)&A#$)'55$"A#)5$%9=9%9A59c:9:"N(&5'#&$A)%94"(#4

J"R+H*Q#(8'()

5JOZ*O(+'+9*C$#%8

! 1.#4H)J&G9A)')(&4#)b"9%C2)=&A:)'A:)%'A?)%9(9G'A#)9A#&#&94)=%$D)'A)Z3R)5$((95#&$A)[E&?&N9:&']

! OR#LG4"H)O!"#$%&'()*$"(+#,&-)./&#&)0)*'()%'1).,+/)!"#$-P

! 7"NN(9D9A#'%C)&AN"#)&A=$%D'#&$A

! '])+'#9J$%C;)2$"(+#,&-[*A#&#C)-'A?&AJ)!'4?]

! F])*W'DN(9)9A#&#&94;)3#'(*&4)5&#6'(14)7%',([R&4#)+$DN(9#&$A)!'4?]

CQO6*O(+'+9*C$#%8

! -'A?&AJ)9A#&#&94)=$"A:)&A)')G4#'(+"R+*5$((95#&$A)[+("9E9F]

! .//d)#'4?;)=&A:)%9('#9:)9A#&#&94

!"#$%&!'"#$%!(")*)+,"-$(%./(*"01&2&!'(")*)+,"-$(%!(")*)+,345%67#(8(9:;<(":::=<&=<:>>;>!'(")*)+,345%!)-?0(),(")*)+%/?0-"*@-)*/"!')-?0(),(")*)+%!"-??-)*A(%B*?7*"(C1)D-)16#??(")7+1#C(1./(*"01&2&1E7-"(CF!'"-??-)*A(%

D$./'4"M7#="2*Q#(8'()

0SL49)A'D9:)9A#&#C):9#95#&$A)#$$(4)#$)=&A:)A'D9:)"(+'+'"=*'(*2.%BL"(+*%.44"%+'.(

.SY$%)9'58)9A#&#C2)"49)#89)$55"%%9A59)5$A#9W#4)#$)5%9'#9)')@'$+B#4*2.%BL"(+)[49%G&AJ)'4)')N%$=&(9]S

1S!89A)"49)$")B4#$*5Q)D9#8$:4)#$)%'A?)#8949)N%$=&(9):$5"D9A#4S

))))))))))))))))))))))[>'($J)9#)'(S)!V,7)./00]

C"R+*Q"=B4+M7#="2*Q#(8'()

0SY&%4#)=&A:)%9(9G'A#):$5"D9A#4

.S*W#%'5#)9A#&#&94)=%$D)#8949):$5"D9A#4

1SL49)G'%&$"4)4$%#4)$=)%'A?&AJ)#958A&b"94)[9SJS)=%9b"9A5C2)('AJ"'J9)D$:9(&AJ2)E&?&N9:&')'4)F'5?J%$"A:)?A$6(9:J9]

C#R.(.L'%*K'4+"$'()

! Y&(#9%)9A#&#&94)F'49:)$A)#89&%)$A#$($J&5'()#CN9)[9SJS)%&#-$(2)$#8'(,9'+,$(2)$%)D$%9)4N95&=&5;)6"-&"62)6$:,&]

! *W'DN(9;)#%C)=&A:&AJ)9W'DN(9)9A#&#&94)=$%)#89)#'%J9#)#CN9)'A:)#89A)5$DN'%&AJ)5'A:&:'#94)6&#8)#89D[e958#$D$G')./0/]

3OJC>H*3B4+'4'()B#4*C#R.(.L9*S/$.L*,.$2J"+*[*V\W*,'8'G"2'#=Y

2"*3"4.*]*,"'8BLU*65:3*VWXWU&++GHPP+'(9U%%PBN(#G'

Q#(8'()*!B-)$#G&=*

))))))))>9=$%9; Y&A:)9A#&#&94))))))))M$6; Y&A:)68$(9)4"FJ%'N842)9SJS)D'#58&AJ))))))))))))))))) ?9C6$%:4

*(F'44"$A&)a)>('A5$S)+,H3)./00;R'AJ"'J9)3$:9(<F'49:)-'A?&AJ)=$%)O5$D9:C)'5':9DC)'6'%:P

Q#(8'()*!B-)$#G&=*

! MBUB)[H'4A95&)9#)'(S).//\])

7#'#&4#&5'()('AJ"'J9)D$:9()=$%)%'A?&AJ)9A#&#C<%9('#&$A48&N)4"FJ%'N84

!89C)'(()8'G9):$5#$%'():9J%994)=%$D)U9%D'A)"A&G9%4&#&94)

[8$A$%'%C):$5#$%'#9)=$%)#89)I'('&)R'D']S

Q#(8'()*!B-)$#G&=

!B-)$#G&=*=&.B42*-"*2"(="*#(2*'(/.$L#+'@"

C9G'%#4*>GG$.#%&H*=&A:)7#9&A9%)!%994

OR#LG4"HBAJ9(')39%?9(B%A$(:)7586'%X9A9JJ9%

7!B-S)H'4A95&)9#)'(S),+I*).//d

EB"=+'.(=^

C&"*D4#(

! 5(+$.2B%+'.(

! !+$B%+B$"2*0#+#*.(*+&"*,"-

! EB"$9'()*F'(8"2*AG"(*0#+#

! EB"$9'()*+&"*,"-*./*0#+#

! EB"$9*?(2"$=+#(2'()

! AB+4..8*#(2*6.(%4B='.(

J#L"2*O(+'+9*Q"%.)('+'.(

3.$"*+&#(*L.$"*+&#(*_W`*./*,"-*="#$%&*"()'("*aB"$'"=*

#'L*#+*"(+'+'"=SD.B(2*"+*#4U*,,,*VWXWY

"U)U*G".G4"b*G4#%"=b*

$"=+#B$#(+=b*G$.2B%+=b%.LG#('"=

J#L"2*O(+'+9*Q"%.)('+'.(

U$$J(9)f9&#J9&4#)./0.)!$N)!%9A:&AJ)79'%5894

[5%9:&#;)-$F9%#)M9"D'C9%]

J#L"2*O(+'+9*Q"%.)('+'.(

Q"="#$%&*6&#44"()"H<.N*+.*L"#=B$"*B="$*=#+'=/#%+'.(^*A/+"(*#GG#$"(+49*(.*/""2-#%8*#4+&.B)&*B="$*'=*&#GG'"$U

J#L"2*O(+'+9*0'=#L-')B#+'.(

F'=+*EB"$9*0"+"%+'.(

XV`*./*#44*aB"$'"=*S3'8#*"+*#4UY

F'=+*EB"$9*0"+"%+'.(

U$$J(9)'4)$=)./01</1

F'=+*EB"$9*0"+"%+'.(

! K'("M)$#'("2b*N'+&*#++$'-B+"=HO,A#9((95#"'()N%$N9%#C)('6C9%4)&A)^'($)B(#$P

! Q#(8'()PAG'('.(*3'('()HOU$$:):9A#&4#4)&A)>%$$?(CAPOF94#)%'D9A)&A)V'?('A:P

! C9G"M!G"%'/'%*?5=^

F'=+*EB"$9*>(=N"$=

! !$A$A2)I9D'%#&A&)a)+":%g<3'"%$"W)[7,U,-)./0.];

! Y&%4#)=&A:)%9(9G'A#)9A#&#&94)&A)4A&NN9#4)=$"A:)&A)%9J"('%)&AG9%#9:)&A:9W

! !89A)%9<%'A?)'A:)9W#9A:)(&4#)FC):&45$G9%&AJ)4&D&('%)9A#&#&94)"4&AJ)7#%"5#"%9:)I'#')%9N$4&#$%C

>++$'-B+"*EB"$'"=

! +"%%9A#(C)'%$"A:)hi)$=)E9F)@"9%&949SJS)OX&N)5$:9)6'#9%G&((9)D'&A9P)[^9#9%)3&?']

>++$'-B+"*EB"$'"=

! OD9J)%C'A)%$D'A59PD'#5894):&==9%9A#)'##%&F"#94)[=&9(:4]5'4#; D9J)%C'AJ9A%9; %$D'A59

! M'jG9)4$("#&$A;)+$DN"#9)>3.h)=$%)9'58)=&9(:2)5$DF&A9

! ^%$F(9D;)9'58)>3.h)G'("9)%9D'&A4)"A'6'%9)$=)68&58)$#89%)b"9%C)#9%D4)69%9)D'#589:)&A)$#89%)=&9(:4

>++$'-B+"*EB"$'"=*N'+&*73VcK

! Y$%)9'58)#9%D2)'JJ%9J'#9)'5%$44):&==9%9A#)=&9(:4S

! !89A)'JJ%9J'#9)'5%$44)#89)#9%D4S

! B(4$;)L49)=&9(:<4N95&=&5)69&J8#4

7$"%59;)-$F9%#4$A)'A:)f'%'J$X')[.//k]

>++$'-B+"*EB"$'"=*N'+&*DQ3!

! ,:9';)Y&9(:<4N95&=&5)69&J8#4)48$"(:)'(4$):9N9A:)$A)4N95&=&5)b"9%C)6$%:l

7$"%59;)H&D2)Z"9)a)+%$=#)[.//d]

>++$'-B+"*EB"$'"=*.@"$*C"R+

*7!*-)[>'4#)9#)'(S)7,U,-).//k];)E8&(9)&A:9W&AJ)'):$5"D9A#)5$((95#&$A2)'::)'%#&=&5&'()49D'A#&5)6$%:4)#$)&AG9%#9:)&A:9W2)9SJS)ON9%4$A;#$AC)F('&%P2)#$)%9=9%9A59)'A)'%#&=&5&'():$5"D9A#)'F$"#)!$AC)>('&%

d.'(*EB"$'"=H*e-.$(*'(*9"#$HXfghi

>++$'-B+"*EB"$'"=*.@"$*C"R+

`'?&'K4)@I*Z;),A:9W)D9'A&AJ):&45$G9%9:)&A)#9W#S

79A#9A59)=$"A:)'4)')%94N$A49)#$)b"9%C)O!"#$%&'()%$'*#$+%"*#&#,"*-P

7$"%59;)8'?&'S5$D)E8&#9N'N9%)$A)79D'A#&5)79'%58)!958A$($JCS)Q'A"'%C)./0/S

>++$'-B+"*EB"$'"=*.@"$*C"R+

Q"="#$%&*6&#44"()"=H

!E8'#)=$%D4)$=)#9W#)'A'(C4&4)'%9)45'('F(9)9A$"J8)=$%)&A#9J%'#&$A)&A#$)'A)&AG9%#9:)&A:9Wm

!`$6)#$)'G$&:)&A:9W)4&X9)F($6<"N!-$F"4#A944;)`$6)#$):9'()6&#8)A$&492)($AJ)#'&()9A#&#&942)9#5S

!`+,;)E8'#)'%9)#89)F94#)[&A#9%'5#&G9])L,)N'%':&JD4m

!"L#(+'%*>(=N"$=*/.$*>$-'+$#$9*EB"$'"=

! JC65Q*X6F56:*C#=8H)U&G9A)')b"9%C2)%9#"%A)')48$%#)4"DD'%C)$=)%9(9G'A#)69F)N'J94S

! OR#LG4"H)6$%:):9=&A&#&$A42)'%#&4#)&A=$%D'#&$A2)9#5S

! 6.(=+$#'(+H)7&X9)(&D&#4)[9SJS)0///)58'%'5#9%4)=$%):94?#$N2).\/)=$%)D$F&(9)%"A]