graph data management lab, school of computer sciencegdm@fudangdm@fudan scalable sparql querying of...

Graph Data Management Lab, School of Computer Science

GDM@FUDAGDM@FUDANN

http://gdm.fudan.edu.cnhttp://gdm.fudan.edu.cn

Scalable SPARQL Querying of Large RDF Graphs

Xu Bo

2012.06.11

In PVLDB, 4(21), 2011


GDM@FUDAGDM@FUDANN


OutlineOutline• About Presenter

• Semantic Web

• Previous Work

• New Problem

• SYSTEM ARCHITECTURE

• EXPERIMENTS

• CONCLUSIONS AND FUTURE WORK

23/4/19 2


GDM@FUDAGDM@FUDANN


About Presenter

• Daniel Abadi• Associate Professor of Computer Science in Yale University• Research

– Column-Oriented Database Systems– Petascale Parallel Database Systems (HadoopDB) – Semantic Web Data Management

23/4/19 3


GDM@FUDAGDM@FUDANN


Semantic WebSemantic Web

• The vision of Semantic Web is to build a "web of data" that enables machines to understand the semantics of information on the Web

23/4/19 4


GDM@FUDAGDM@FUDANN


Google Knowledge Graph

23/4/19 5


GDM@FUDAGDM@FUDANN


Key Technology

• HTML

• XML

23/4/19 6


GDM@FUDAGDM@FUDANN


The Disadvantage of XML David Billington is a lecturer of Discrete Mathematics.

• there is no standard way of assigning meaning to tag nesting

23/4/19 7


GDM@FUDAGDM@FUDANN


The Disadvantage of Xpath

• Suppose we want to collect all academic staff members. A path expression in Xpath might be //academicStaffMember

• XML is semantically unsatisfactory

23/4/19 8


GDM@FUDAGDM@FUDANN


RDF

• Resource Description Framework• 用 Web标识符（称作统一资源标识符， Uniform

Resource Identifiers 或 URIs）来标识事物，用简单的属性（ property）及属性值来描述资源

23/4/19 9


GDM@FUDAGDM@FUDANN


RDF as Triples and a Graph

23/4/19 10


GDM@FUDAGDM@FUDANN


SPARQL

• RDF query language

• A basic graph pattern

• Answering SPARQL can be seen as finding subgraphs in the RDF data that match the graph pattern

23/4/19 11


GDM@FUDAGDM@FUDANN


Example for Star Pattern

• Find the names of the strikers that play for FC Barcelona.

23/4/19 12


GDM@FUDAGDM@FUDANN


Another Example• Find football players playing for clubs in apopulous region where they were born.

23/4/19 13


GDM@FUDAGDM@FUDANN


23/4/19 14


GDM@FUDAGDM@FUDANN


Previous Work• RDF In RDBMSs

• Property Tables

• Vertically Partitioned Approach

23/4/19 15


GDM@FUDAGDM@FUDANN


RDF In RDBMSs• Get the title of the book(s) Joe Fox wrote

in 2001

23/4/19 16


GDM@FUDAGDM@FUDANN


Property Tables

23/4/19 17


GDM@FUDAGDM@FUDANN


Vertically Partitioned Approach

23/4/19 18


GDM@FUDAGDM@FUDANN


New Problem• Single node RDF management systems are

abundant– Sesame– Jena– RDF3X– 3store

• Research in clustered RDF management is less significantly explored: The focus of the talk

23/4/19 19


GDM@FUDAGDM@FUDANN


SYSTEM ARCHITECTURE

23/4/19 20


GDM@FUDAGDM@FUDANN


Graph Partitioning• Hash vs. Graph partitioning

– Hash: Only efficient for star patterns– Graph: Taking advantage of graph model

23/4/19 21


GDM@FUDAGDM@FUDANN


Graph Partitioning• Edge vs. Vertex partitioning

– Edge: Natural but inefficient for query execution

– Vertex: Superior for common graph patterns

23/4/19 22


GDM@FUDAGDM@FUDANN


Vertex Partitioning• Preprocess

– remove triples whose predicate is rdf:type

• METIS partitioner

23/4/19 23


GDM@FUDAGDM@FUDANN


Triple Placement• Minimizing data shuffling/exchange

– Allowing data overlap

• N-hop guarantee– The extent of data overlap– If a vertex is assigned to a machine, any

vertex that is within n-hop of this vertex is also stored in this machine

23/4/19 24


GDM@FUDAGDM@FUDANN


DIRECTED N-HOP GUARANTEE

23/4/19 25


GDM@FUDAGDM@FUDANN


A potential problem• triples (s, p, o) and (o, p’, o’)

– 2-hop guarantee

• triples (s, p, o) and (s’, p’, o)– not guaranteed

• “object-connected” is not unusual

• undirected n-hop guarantee

23/4/19 26


GDM@FUDAGDM@FUDANN


Triple Placement Algorithm

23/4/19 27


GDM@FUDAGDM@FUDANN


Query Processing• Queries are executed in RDF-stores and/or

Hadoop• Query execution is more efficient in RDF-

stores than in Hadoop– Pushing as much of the processing as possible

into RDF-stores– Minimizing the number of Hadoop jobs– The larger the hop guarantee, the more work is

done in RDF-stores23/4/19 28


GDM@FUDAGDM@FUDANN


To Communicate, or not to Communicate

• Given a query and n-hop guarantee, is communication (Hadoop job) between nodes needed?– Choose the “center” of the query graph– Calculate the distance from the “center” to the

furthest edge– If distance > n, communication is needed; not

needed otherwise

23/4/19 29


GDM@FUDAGDM@FUDANN


Determining whether a Query is PWOC

PWOC Query– parallelizable without communication

• DoFE– distance of farthest edge– the vertex in a graph with the smallest DoFE will be

the most central in a graph

23/4/19 30


GDM@FUDAGDM@FUDANN


The algorithm

23/4/19 31


GDM@FUDAGDM@FUDANN


the issue of duplicate results• naive approach

– remove duplicates after the query has completed

• owner-computes model– add triples (v, ‘<isOwned>’, ‘Yes’) to a

partition– For each query issued to the RDF-stores, add

an additional pattern (core, ‘<isOwned>’, ‘Yes’)

23/4/19 32


GDM@FUDAGDM@FUDANN


A query is not PWOC• decompose the query into PWOC

subqueries

• use Hadoop jobs to join the results of the PWOC subqueries

• The number of Hadoop jobs required to complete the query increases as the number of subqueries increases

23/4/19 33


GDM@FUDAGDM@FUDANN


minimal number of subqueries• reduces to the problem of finding minimal

edge partitioning of a graph into subgraphs of bounded diameter

• brute-force

23/4/19 34


GDM@FUDAGDM@FUDANN


Examlple

23/4/19 35

DoFEs for manager, footballClub, Barcelona and club are 2, 2, 2 and 1

the DoFEs for footballer, pop, region, player and club are 3, 3, 2, 2 and 2,


GDM@FUDAGDM@FUDANN


Decompose Example

23/4/19 36


GDM@FUDAGDM@FUDANN


EXPERIMENTS• 20-machine cluster

• Leigh University Benchmark (LUBM): 270 million triples

• Competitors:– Single-node RDF-3X– SHARD: triple-store system in Hadoop– Graph partitioning (the proposed system)– Hash partitioning on subjects

23/4/19 37


GDM@FUDAGDM@FUDANN


Data Load Time

23/4/19 38


GDM@FUDAGDM@FUDANN


Performance Comparison

23/4/19 39


GDM@FUDAGDM@FUDANN


Varying Number of Machines

23/4/19 40


GDM@FUDAGDM@FUDANN


Summary

23/4/19 41


GDM@FUDAGDM@FUDANN


23/4/19 42

graph data management lab, school of computer sciencegdm@fudangdm@fudan scalable sparql querying of...

Documents

graph data management

school of computer sciencegdm

web of data

rdf data

basic graph pattern

google knowledge graph

vision of semantic web

previous work rdf