arX

iv:0

812.

3788

v2 [

cs.D

B]

26 J

an 2

009

Foundations of SPARQL Query Optimization

Michael Schmidt∗ Michael Meier∗ Georg LausenInstitut fur Informatik

Georges-Kohler-Allee, Gebaude 05179110 Freiburg i. Br., Germany

mschmidt, meierm, lausen@informatik.uni-freiburg.de

ABSTRACT

The SPARQL query language is a recent W3C standard forprocessing RDF data, a format that has been developed toencode information in a machine-readable way. We inves-tigate the foundations of SPARQL query optimization and(a) provide novel complexity results for the SPARQL evalu-ation problem, showing that the main source of complexityis operatorOptional alone; (b) propose a comprehensiveset of algebraic query rewriting rules; (c) present a frame-work for constraint-based SPARQL optimization based uponthe well-known chase procedure for Conjunctive Query min-imization. In this line, we develop two novel terminationconditions for the chase. They subsume the strongest condi-tions known so far and do not increase the complexity of therecognition problem, thus making a larger class of both Con-junctive and SPARQL queries amenable to constraint-basedoptimization. Our results are of immediate practical interestand might empower any SPARQL query optimizer.

1. Introduction

The SPARQL Protocol and Query Language is a recentW3C recommendation that has been developed to extractinformation from data encoded using the Resource Descrip-tion Framework (RDF) [14]. From a technical point of view,RDF databases are collections of (subject,predicate,object)triples. Each triple encodes the binary relationpredicate be-tweensubject andobject, i.e. represents a single knowledgefact. Due to their homogeneous structure,RDF databases canbe seen as labeled directed graphs, where each triple definesan edge from thesubject to theobject node under labelpred-icate. While originally designed to encode knowledge in theSemantic Web in a machine-readable format, RDF has foundits way out of the Semantic Web community and entered thewider discourse of Computer Science. Coming along withits application in other areas, such as bio informatics, datapublishing, or data integration, large RDF repositories havebeen created (cf. [29]). It has repeatedly been observed thatthe database community is facing new challenges to copewith the specifics of the RDF data format [7, 18, 21, 31].∗The work of this author was funded by DeutscheForschungsgemeinschaft grant GRK 806/03.

With SPARQL, the W3C has recommended a declarativequery language that allows to extract data from RDF graphs.SPARQL comes with a powerful graph matching facility,whose basic construct are so-called triple patterns. Duringquery evaluation, variables inside these patterns are matchedagainst the RDF input graph. The solution of the evaluationprocess is then described by a set of mappings, where eachmapping associates a set of variables with RDF graph com-ponents. SPARQL additionally provides a set of operators(namelyAnd,Filter, Optional, Select, andUnion),which can be used to compose more expressive queries.

One key contribution in this paper is a comprehensivecom-plexity analysis for fragments of SPARQL. We follow pre-vious approaches [26] and use the complexity of theEval-uation problem as a yardstick: given queryQ, data setD,and candidate solutionS as input, check ifS is contained inthe result of evaluatingQ onD. In [26] it has been shownthat full SPARQL isPSpace-complete, which is bad newsfrom a complexity point of view. We show that yet op-eratorOptional alone makes theEvaluation problemPSpace-hard. Motivated by this result, we further refine ouranalysis and prove better complexity bounds for fragmentswith restricted nesting depth ofOptional expressions.

Having established this theoretical background, we turntowards SPARQL query optimization. The semantics ofSPARQL is formally defined on top of a compact algebraover mapping sets. In the evaluation process, the SPARQLoperators are first translated into algebraic operations, whichare then directly evaluated on the data set. The SPARQLAlgebra (SA) comprises operations such as join, union, leftouter join, difference, projection, and selection, akin totheoperators defined in Relational Algebra (RA). At first glance,there are many parallels between SA and RA; in fact, thestudy in [1] reveals that SA and RA have exactly the sameexpressive power. Though, the technically involved proofin [1] indicates that a semantics-preserving SA-to-RA trans-lation is far from being trivial (cf. [6]). Hence, although bothalgebras provide similar operators, there are still very funda-mental differences between both. One of the most strikingdiscrepancies, as also argued in [26], is that joins in RA arerejecting over null-values, but in SA, where the schema isloose in the sense that mappings may bind an arbitrary setof variables, joins over unbound variables (essentially theequivalent of RA null-values) are always accepting.

One direct implication is that not all equivalences that holdin RA also hold in SA, and vice versa, which calls for a studyof SA by its own. In response, we present an elaborate studyof SA in the second part of the paper. We survey exist-ing and develop new algebraic equivalences, covering vari-ous SA operators, their interaction, and their relation to theRA counterparts. When interpreted as rewriting rules, theseequivalences form the theoretical foundations for transferringestablished RA optimization techniques, such as filter push-ing, into the SPARQL context. Going beyond the adaptionof existing techniques, we also address SPARQL-specific is-sues, e.g. provide rules for simplifying expressions involving(closed-world) negation, which can be expressed in SPARQLsyntax using a combination ofOptional andFilter.

We note that in the past much research effort has beenspent in processing RDF data with traditional systems, suchas relational DBMSs or datalog engines [7, 18, 31, 25, 12, 21,27], thus falling back on established optimization strategies.Some of them (e.g. [31, 12]) work well in practice, but arelimited to small fragments, such asAnd-only queries. Morecomplete approaches (e.g. [7]) suffer from performance bot-tlenecks for complex queries, often caused by poor optimiza-tion results (cf. [18, 21, 22]). For instance, [21] identifiesdeficiencies of existing schemes for queries involving nega-tion, a problem that we tackle in our analysis. This alsoshows that traditional approaches are not laid out for the spe-cific challenges that come along with SPARQL processingand urges the need for a thorough investigation of SA.

In the final part of the paper we study constraint-basedquery optimization in the context of SPARQL, also knownas Semantic Query Optimization (SQO). SQO has been ap-plied successfully in other contexts before, such as Conjunc-tive Query (CQ) optimization (e.g., [3]), relational databases(e.g., [17]), and deductive databases (e.g., [5]). We demon-strated the prospects of SQO for SPARQL in [19], and inthis work we lay the foundations for a schematic semanticoptimization approach. Our SQO scheme builds upon theChase & Backchase (C&B) algorithm [9], an extension ofthe well-known chase procedure for CQ optimization [24, 3,16]. One key problem with the chase is that it might not al-ways terminate. Even worse, it has recently been shown thatfor an arbitrary set of constraints it is undecidable if it termi-nates or not [8]. There exist, however, sufficient conditionsfor the termination of the chase; the best condition known sofar is that ofstratified constraints [8]. The definition of strat-ification uses a former termination condition for the chase,namelyweak acyclicity [28]. In this paper, we present twoprovably stronger termination conditions, making a largerclass of CQs amenable to the semantic optimization processand generalizing the methods introduced in [28, 8].

Our first condition, calledsafety, strictly subsumes weakacyclicity and the second,safe restriction, strictly subsumesstratification. They do not increase the complexity of therecognition problem, i.e. safety is checkable in polynomialtime (like weak acyclicity) and safe restriction by acoNP-algorithm (like stratification). We emphasize that our resultsimmediately carry over to data exchange [28] and integra-tion [20], query answering using views [15], and the implica-

tion problem for constraints. Further, they apply to the corechase introduced in [8] (there it was proven that the termina-tion of the chase implies termination of the core chase).

In order to optimize SPARQL queries, we translateAnd-blocks of the query into CQs, optimize them using the C&B-algorithm, and translate the outcome back into SPARQL. Ad-ditionally, we provide optimization rules that go beyond suchsimple queries, showing that in some casesOptional- andFilter-queries can be simplified. With respect to chase ter-mination, we introduce two alternate SPARQL-to-CQ trans-lation schemes. They differ w.r.t. the termination conditionsthat they exhibit for the subsequent chase, i.e. our sufficientchase termination conditions might guarantee terminationforthe first but not for the second translation, and vice versa.

Our key contributions can be summarized as follows.

• We present previously unknown complexity results forfragments of the SPARQL query language, showing thatthe main source of complexity is operatorOptionalalone. Moreover, we prove there are better bounds whenrestricting the nesting depth ofOptional expressions.• We summarize existent and establish new equivalences

over SPARQL Algebra. Our extensive study character-izes the algebraic operators and their interaction, andmight empower any SPARQL query optimizer. We alsoindicate an erratum in [26] and discuss its implications.• Our novel SQO scheme for SPARQL can be used to

optimizeAnd-only queries under a set of constraints.Further, we provide rules for semantic optimization ofqueries involving operatorsOptional andFilter.• We present two novel sufficient termination conditions

for the chase, which strictly generalize previous condi-tions. This improvement empowers the practicability ofmany important research areas, like e.g. [28, 20, 15, 9].

Structure. We start with some preliminaries in Section 2and present the complexity results for SPARQL fragments inSection 3. The subsequent discussion of query optimizationdivides into algebraic optimization (Section 4) and seman-tic optimization (Section 5). The latter discussion is com-plemented by the chase termination conditions presented inSection 6. Finally, Section 7 contains some closing remarks.

2. Preliminaries

RDF. We follow the notation from [26]. We considerthree disjoint setsB (blank nodes),I (IRIs), andL (literals)and use the shortcutBIL to denote the union of the setsB, I, andL. By convention, we indicate literals by quotedstrings (e.g.“Joe”, “30”) and prefix blank nodes with “:”. AnRDF triple (v1,v2,v3) ∈ BI × I × BIL connects subjectv1 through predicatev2 to objectv3. An RDF database,also called document, is a finite set of triples. We refer theinterested reader to [14] for an elaborate discussion of RDF.

SPARQL Syntax. LetV be an infinite set of variablesdisjoint fromBIL. We start with an abstract syntax forSPARQL, where we abbreviate operatorOptional asOpt.

2

Definition 1. We define SPARQL expressions recursivelyas follows. (1) A triple pattern t ∈ BIV × IV ×BILVis an expression. (2) Let Q1, Q2 be expressions and Ra filter condition. Then Q1 Filter R, Q1 Union Q2,Q1 Opt Q2, and Q1 And Q2 are expressions.

In the remainder of the paper we restrict our discussionto safe filter expressionsQ Filter R, where the variablesoccurring inR form a subset of the variables inQ. As shownin [1], this restriction does not compromise expressiveness.

Next, we define SPARQLqueries on top of expressions.1

Definition 2. Let Q be a SPARQL expression and letS ⊂ V a finite set of variables. A SPARQL query is anexpression of the form SelectS(Q).

SPARQL Semantics. A mapping is a partial functionV → BIL from a subset of variablesV to RDF termsBIL.The domain of a mappingµ,writtendom(µ), is defined as thesubset ofV for whichµ is defined. As a naming convention,we distinguish variables from elements inBIL through aleading question mark symbol. Given two mappingsµ1, µ2,we sayµ1 is compatible with µ2 if µ1(?x) = µ2(?x) forall ?x ∈ dom(µ1) ∩ dom(µ2). We writeµ1 ∼ µ2 if µ1

andµ2 are compatible, andµ1 6∼ µ2 otherwise. Further, wewrite vars(t) to denote all variables in triple patternt and byµ(t) we denote the triple pattern obtained when replacing allvariables?x ∈ dom(µ) ∩ vars(t) in t byµ(?x).

Given variables?x, ?y and constantsc, d, a filter conditionR is either an atomic filter condition of the formbound(?x)(abbreviated asbnd(?x)), ?x = c, ?x =?y, or a combinationof atomic conditions using connectives¬, ∧, ∨. Conditionbnd(?x) applied to a mapping setΩ returns all mappings inΩ for which ?x is bound, i.e.µ ∈ Ω |?x ∈ dom(µ). Theconditions?x = c and?x =?y are equality checks, com-paring the values of?x with c and?y, respectively. Thesechecks fail whenever one of the variables is not bound. Wewrite µ |= R if mappingµ satisfies filter conditionR (seeDefinition 16 in Appendix B.2 for a formal definition). Thesemantics of SPARQL is then formally defined using a com-pact algebra over mapping sets (cf. [26]). The definition ofthe algebraic operators join, union∪, set minus\, leftouter join1 , projectionπ, and selectionσ is given below.

Definition 3. Let Ω, Ωl, Ωr denote mapping sets, R afilter condition, and S ⊂ V a finite set of variables. Wedefine the algebraic operations , ∪, \, 1 , π, and σ:

Ωl Ωr := µl ∪ µr | µl ∈ Ωl, µr ∈ Ωr : µl ∼ µrΩl ∪Ωr := µ | µ ∈ Ωl or µ ∈ ΩrΩl \ Ωr := µl ∈ Ωl | for all µr ∈ Ωr : µl 6∼ µrΩl 1 Ωr:= (Ωl Ωr) ∪ (Ωl \ Ωr)πS(Ω) := µ1 | ∃µ2 : µ1 ∪ µ2 ∈ Ω ∧ dom(µ1) ⊆ S

∧ dom(µ2) ∩ S = ∅

σR(Ω) := µ ∈ Ω | µ |= R

1We do not consider the remaining SPARQL query formsAsk, Construct, and Describe in this paper.

We follow the compositional, set-based semantics pro-posed in [26] and define the result of evaluating SPARQLqueryQ on documentD using operatorJ.KD defined below.

Definition 4. Let t be a triple pattern, Q1, Q2 SPARQLexpressions, R a filter condition, and S ⊂ V a finite setof variables. The semantics of SPARQL evaluation overdocument D is defined as follows.

JtKD := µ | dom(µ) = vars(t) and µ(t) ∈ DJQ1 And Q2KD := JQ1KD JQ2KDJQ1 Opt Q2KD := JQ1KD 1 JQ2KDJQ1 Union Q2KD := JQ1KD ∪ JQ2KDJQ1 Filter RKD := σR(JQ1KD)

JSelectS(Q1)KD := πS(JQ1KD)

Finally, we extend the definition of functionvars. LetQbe a SPARQL expression,A a SPARQL Algebra expression,andR a filter condition. Byvars(A), vars(Q), andvars(R)we denote the set of variables inA, Q, andR, respectively.Further, we define functionsafeVars(A), which denotes thesubset of variables invars(A) that are inevitably bound whenevaluatingA on any documentD.

Definition 5. Let A be a SPARQL Algebra expression,S ⊂ V a finite set of variables, and R a filter con-dition. We define function safeVars(A) recursively onthe structure of expression A as follows.

safeVars(JtKD) := vars(t)

safeVars(A1 A2) := safeVars(A1) ∪ safeVars(A2)

safeVars(A1 ∪ A2) := safeVars(A1) ∩ safeVars(A2)

safeVars(A1 \A2) := safeVars(A1)

safeVars(A1 1 A2) := safeVars(A1)

safeVars(πS(A1)) := safeVars(A1 ) ∩ S

safeVars(σR(A1)) := safeVars(A1 )

Relational Databases, Constraints and Chase.We assume that the reader is familiar with first-order logicand relational databases. We denote bydom(I) the do-main of the relational database instanceI, i.e. the set ofconstants and null values that occur inI. The constraintswe consider, are tuple-generating dependencies (TGD) andequality-generating dependencies (EGD). TGDs have theform ∀x(ϕ(x) → ∃yψ(x, y)) and EGDs have the form∀x(ϕ(x) → xi = xj). A more exact definition of thesetypes of constraints can be found in Appendix D.1. In therest of the paperΣ stands for a fixed set of TGDs and EGDs.If an instanceI is not a model of some constraintα, then wewrite I 2 α.

We now introduce the chase as defined in [8]. A chase

stepIα,a→ J takes a relational database instanceI such that

I 2 α(a)and adds tuples (in case of TGDs) or collapses someelements (in case of EGDs) such that the resulting relationaldatabaseJ is a model ofα(a). If J was obtained fromI inthat kind, we sometimes also writeIa⊕Cα instead ofJ . Achase sequence is a sequence of relational database instancesI0, I1, ... such thatIs+1 is obtained fromIs by a chase step.

3

A chase sequenceI0, ..., In is terminating ifIn |= Σ. Inthis case, we setIΣ := In as the result (IΣ is defined onlyunique up to homomorphicequivalence, but this will suffice).Otherwise,IΣ is undefined.IΣ is also undefined in case thechase fails. More details can be found in Appendix D.1.The chase does not always terminate and there has beendifferent work on sufficient termination conditions. In [28]the following condition, based on the notion of dependencygraph, was introduced. The dependency graph dep(Σ) :=(V,E) of a set of constraintsΣ is the directed graph definedas follows.V is the set of positions that occur inΣ. Thereare two kind of edges inE. Add them as follows: for everyTGD ∀x(ϕ(x)→ ∃yψ(x, y)) ∈ Σ and for everyx in x thatoccurs inψ and every occurrence ofx in ϕ in positionπ1

• for every occurrence ofx in ψ in positionπ2, add an edgeπ1 → π2 (if it does not already exist).• for every existentially quantified variabley and for every

occurrence ofy in a positionπ2, add a special edgeπ1

∗→ π2 (if it does not already exist).

A set Σ of TGDs and EGDs is calledweakly acyclic iffdep(Σ) has no cycles through a special edge. In [8] weakacyclicity was lifted to stratification. Given two TGDs orEGDsα = ∀x1ϕ, β = ∀x2ψ, we defineα ≺ β (meaningthat firing α may causeβ to fire) iff there exist relationaldatabase instancesI, J anda ∈ dom(I), b ∈ dom(J) s.t.

• I 2 ψ(b) (possiblyb is not indom(I)),

• Iα,a−→ J , and

• J 2 ψ(b).

The chase graphG(Σ) = (Σ, E) of a set of constraintsΣcontains a directed edge(α, β) between two constraints iffα ≺ β. We callΣ stratified iff the set of constraints in everycycle ofG(Σ) are weakly acyclic. It is immediate that weakacyclicity implies stratification; further, it was proven in [8]that the chase always terminates for stratified constraint sets.

A Conjunctive Query (CQ) is an expression of the formans(x)← ϕ(x, z), whereϕ is a CQ of relational atoms andx, z are tuples of variables and constants. Every variable inx must also occur inϕ. The semantics of such a query on adatabase instanceI is q(I) := a | I |= ∃zϕ(a, z) .

Let q, q′ be CQs andΣ be a set of constraints. We writeq ⊑Σ q′ if for all database instancesI such thatI |= Σ itholds thatq(I) ⊆ q′(I) and say thatq andq′ areΣ-equivalent(q ≡Σ q′) if q ⊑Σ q′ andq′ ⊑Σ q. In [9] an algorithm waspresented that, givenq andΣ, lists allΣ-equivalent minimal(with respect to the number of atoms in the body) rewritings(up to isomorphism) ofq. This algorithm, called Chase &Backchase, uses the chase and therefore does not necessarilyterminate. We denote its output bycbΣ(q) (if it terminates).

General mathematical notation. The natural num-bersN do not include0; N0 is used as a shortcut forN∪0.Forn ∈ N, we denote by[n] the set1, ..., n. Further, fora setM , we denote by2M its powerset.

3. SPARQL Complexity

We introduce operator shortcutsA := And,F := Filter,O := Opt, U := Union, and denote the class of expres-sions that can be constructed using a set of operators byconcatenating their shortcuts. Further, byE we denote thewhole class of SPARQL expressions, i.e.E := AFOU . Thetermsclass andfragment are used interchangeably.

We first present a complete complexity study for all possi-ble expression classes, which complements the study in [26].We assume the reader to be familiar with basics of complex-ity theory, yet summarize the background in Appendix A.1,to be self-contained. We follow [26] and take the combinedcomplexity of theEvaluation problem as a yardstick:

Evaluation: given a mappingµ, a documentD, and anexpression/queryQ as input: isµ ∈ JQKD?

The theorem below summarizes previous results from [26].

Theorem 1. [26] The Evaluation problem is

• in PTime for class AF ; membership in PTime forclasses A and F follows immediately,

• NP-complete for class AFU , and• PSpace-complete for classes AOU and E .

Our first goal is to establish a more precise characterizationof theUnion operator. As also noted in [26], its design wassubject to controversial discussions in the SPARQL workinggroup2, and we pursue the goal to improve the understandingof the operator and its relation to others, beyond the knownNP-completeness result for classAFU . The following the-orem gives the results for all missingOpt-free fragments.

Theorem 2. The Evaluation problem is

• in PTime for classes U and FU , and• NP-complete for class AU .

The hardness part of theNP-completeness proof for frag-mentAU is a reduction fromSetCover. The interestedreader will find details and other technical results of thissection in Appendix A. Theorems 1 and 2 clarify that thesource of complexity inOpt-free fragments is the combina-tion of And andUnion. In particular, adding or removingFilter-expressions in no case affects the complexity.

We now turn towards an investigation of the complexity ofoperatorOpt and its interaction with other operators. ThePSpace-completeness results for classesAOU andAFOUstated in Theorem 1give only partial answers to the questions.One of the main results in this section is the following.

Theorem 3. Evaluation is PSpace-complete for O.

This result shows that already operatorOpt alone makestheEvaluation problem really hard. Even more, it up-grades the claim in [26] that “the main source of complexityin SPARQL comes from the combination ofUnion and2See the discussion of disjunction in Section 6.1 inhttp://www.w3.org/TR/2005/WD-rdf-sparql-query-20050217/.

4

Opt operators”, by showing thatUnion (andAnd) are notnecessary to obtainPSpace-hardness. The intuition of thisresult is that the algebra operator1 (which is the algebraiccounterpart of operatorOpt) is defined using operators,∪, and\; the mix of these algebraic operations compensatesfor missingAnd andUnion operators at syntax level. Thecorollary below follows from Theorems 1 and 3 and makesthe complexity study of the expression fragments complete.

Corollary 1 . The Evaluation problem for any expres-sion fragment involving Opt is PSpace-complete.

Due to the high complexity ofOpt, an interesting questionis whether we can find natural syntactic conditions that lowerthe complexity of fragments involvingOpt. In fact, a restric-tion of the nesting depth ofOpt expressions constitutes sucha condition. We define theOpt-rankr of an expression as itsdeepest nesting ofOpt expressions: for triple patternt, ex-pressionsQ, and conditionR, we definer(Q) recursively onthe structure ofQ asr(t) := 0, r(Q1 Filter R) := r(Q1),r(Q1 And Q2)=r(Q1 Union Q2) := max (r(Q1), r(Q2)),andr(Q1 Opt Q2) := max (r(Q1), r(Q2)) + 1.

By E≤n we denote the class of expressionsQ ∈ E withr(Q) ≤ n. The following theorem shows that, when restrict-ing theOpt-rank of expressions, theEvaluation problemfalls into a class in the polynomial hierarchy.

Theorem 4. For any n ∈ N0, the Evaluation problemis ΣPn+1-complete for the SPARQL fragment E≤n.

Observe thatEvaluation for classE≤0 is complete forΣP1 =NP, thus obtaining the result forOpt-free expressions(cf. Theorem 1). With increasing nesting-depth ofOpt ex-pressions we climb up the polynomial hierarchy (PH). This isreminiscent of theValidity-problemfor quantified booleanformulae, where the number of quantifier alternations fixesthe complexity class in the PH. In fact, the hardness proof(see Appendix A.3) makes these similarities explicit.

We finally extend our study to SPARQL queries, i.e. frag-ments involving top-level projection in the form of aSe-lect-operator (see Def. 2). We extend the notation forclasses as follows. LetF be an expression fragment. Wedenote byF+ the class of queries of the formSelectS(Q),whereS ⊂ V is a finite set of variables andQ ∈ F is an ex-pression. The next theorem shows that we obtain (top-level)projection for free in fragments that are at leastNP-complete.

Theorem 5. Let C be a complexity class and F a classof expressions. If Evaluation is C-complete for F andC ⊇ NP then Evaluation is also C-complete for F+.

In combination with Corollary 1 we immediately obtainPSpace-completeness for query classes involving opera-tor Opt. Similarly, all Opt-free query fragments involv-ing bothAnd andUnion areNP-complete. We concludeour complexity analysis with the following theorem, whichshows that top-level projection makes theEvaluationproblem forAnd-only expressions considerably harder.

Theorem 6. Evaluation is NP-complete for A+.

4. SPARQL Algebra

We next present a rich set of algebraic equivalences forSPARQL Algebra. In the interest of a complete survey weinclude equivalences that have been stated before in [26].3

Our main contributions in this section are (a) a systematicextension of previous rewriting rules, (b) a correction of anerratum in [26], and (c) the development and discussion ofrewriting rules for SPARQL expressions involving negation.

We focus on two fragments of SPARQL algebra, namelythe full class of algebra expressionsA (i.e., algebra expres-sions build using operators∪, , \, 1 , π, andσ) and theunion- and projection-free expressionsA

− (build using onlyoperator, \, 1 , andσ). We start with a property thatseparatesA− fromA, calledincompatibility property.4

Proposition 1. Let Ω be the mapping set obtained fromevaluating an A

−-expression on any document D. Allpairs of distinct mappings in Ω are incompatible.

Figure 1(I-IV) surveys rewriting rules that hold with re-spect to common algebraic laws (we writeA ≡ B if A isequivalent toB on any documentD). GroupI containsresults obtained when combining an expression with itselfusing the different operators. It is interesting to see that(JI-dem) and(LIdem) hold only for fragmentA−; in fact, itis the incompatibility property that makes the equivalencesvalid. The associativity and commutativity rules were in-troduced in [26] and we list them for completeness. Mostinteresting is distributivity. We observe that, \, 1 areright-distributive over∪, and is also left-distributive over∪. The listing in Figure 1 is complete in the following sense:

Lemma 1. Let O1 := , \, 1 and O2 := O1 ∪ ∪.

• The two equivalences (JIdem) and (LIdem) in gen-eral do not hold for fragments larger than A

−.

• Associativity and Commutativity do not hold foroperators \ and 1 .

• Neither \ nor 1 are left-distributive over ∪.• Let o1 ∈ O1, o2 ∈ O2, and o1 6= o2. Then o2 isneither left- nor right-commutative over o1.

Cases (3) and (4) rule out distributivity for all operatorcombinations different from those listed in Figure 1. Thisresult implies that Proposition 1(3) in [26] is wrong:

Example 1. We show that the SPARQL equivalence

A1 Opt (A2 Union A3) ≡ (A1 Opt A2) Union (A1 Opt A3)

stated in Proposition 1(3) in [26] does not hold inthe general case. We choose database D=(0, c, 1) andset A1=(0, c, ?a), A2=(?a, c, 1), and A3=(0, c, ?b). ThenJA1 Opt (A2 Union A3)KD = ?a 7→ 1, ?b 7→ 1,but J(A1 Opt A2) Union (A1 Opt A3)KD evaluates to?a 7→ 1, ?a 7→ 1, ?b 7→ 1. The results differ.

3Most equivalences in [26] were established at the syntacticlevel. In summary, rule (MJ) in Proposition 2 and abouthalf of the equivalences in Figure 1 are borrowed from [26].We indicate these rules in the proofs in Appendix B.4Lemma 2 in [26] also builds on this observation.

5

I. Idempotence and Inverse

A ∪ A ≡ A (UIdem)A−

A− ≡ A− (JIdem)A−1 A− ≡ A− (LIdem)

A \A ≡ ∅ (Inv)

II. Associativity

(A1 ∪ A2) ∪A3 ≡ A1 ∪ (A2 ∪ A3) (UAss)(A1 A2) A3 ≡ A1 (A2 A3) (JAss)

III. Commutativity

A1 ∪ A2 ≡ A2 ∪ A1 (UComm)A1 A2 ≡ A2 A1 (JComm)

IV. Distributivity

(A1 ∪ A2) A3 ≡ (A1 A3) ∪ (A2 A3) (JUDistR)A1 (A2 ∪ A3) ≡ (A1 A2) ∪ (A1 A3) (JUDistL)(A1 ∪ A2) \A3 ≡ (A1 \A3) ∪ (A2 \A3) (MUDistR)(A1 ∪ A2) 1 A3 ≡ (A1 1 A3) ∪ (A2 1 A3) (LUDistR)

V. Filter Decomposition and Elimination

σR(A1 ∪ A2) ≡ σR(A1) ∪ σR(A2) (SUPush)σR1∧R2

(A) ≡ σR1(σR2

(A)) (SDecompI)σR1∨R2

(A) ≡ σR1(A) ∪ σR2

(A) (SDecompII)σR1

(σR2(A)) ≡ σR2

(σR1(A)) (SReord)

σbnd(?x)(A1) ≡ A1, if ?x ∈ safeVars(A1) (BndI)σbnd(?x)(A1) ≡ ∅, if ?x 6∈ vars(A1) (BndII)σ¬bnd(?x)(A1) ≡ ∅, if ?x ∈ safeVars(A1) (BndIII)σ¬bnd(?x)(A1) ≡ A1, if ?x 6∈ vars(A1) (BndIV)

If ?x ∈ safeVars(A2) \ vars(A1), thenσbnd(?x)(A1 1 A2) ≡ A1 A2 (BndV)

VI. Filter PushingThe following rules hold if vars(R) ⊆ safeVars(A1 ).

σR(A1 A2) ≡ σR(A1) A2 (SJPush)σR(A1 \A2) ≡ σR(A1) \A2 (SMPush)σR(A1 1 A2) ≡ σR(A1) 1 A2 (SLPush)

Figure 1: SA equivalences for A-expr. A, A1, A2, A3; A−-expr. A−; filter condition R; variable ?x.

Remark 1. This erratum calls the existence of the unionnormal form stated in Proposition 1 in [26] into ques-tion, as it builds upon the invalid equivalence. We ac-tually do not see how to fix or compensate for this rule,so it remains an open question if such a union normalform exists or not. The non-existence would put differ-ent results into perspective, since – based on the claimthat Union can always be pulled to the top – the au-thors restrict the subsequent discussion to Union-freeexpressions. For instance, results on well-defined pat-terns, normalization, and equivalence between compo-sitional and operational semantics are applicable onlyto queries that can be brought into union normal form.Arguably, this class may comprise most of the SPARQLqueries that arise in practice (queries without union orwith union only at the top-level also constitute very fre-quent patterns in other query languages, such as SQL).Still, a careful reinvestigation would be necessary to ex-tend the results to queries beyond that class.

Figure 1(V-VI) presents rules for decomposing, eliminat-ing, and rearranging (parts of) filter conditions. In combina-tion with rewriting rulesI-IV they provide a powerful frame-work for manipulatingfilter expressions in the style of RA fil-ter rewriting and pushing. Most interesting is the use ofsafe-Vars as a sufficient precondition for(SJPush), (SMPush),and(SLPush).5 The need for this precondition arises fromthe fact that joins over mappings are accepting for unboundvariables. In RA, where joins over null values are rejecting,the situation is less complicated. For instance, given two RArelationsA1,A2 and a (relational) filterR,(SJPush) is appli-cable whenever the schema ofA1 contains all attributes inR.We conclude this discussion with the remark that, for smallerfragments of SPARQL conditions, weaker preconditions forthe rules in groupVI exist. For instance, ifR = e1∧· · ·∧enis a conjunction of atomic equalitiese1, . . . , en, then theequivalences in groupVI follow from the (weaker) conditionvars(R) ⊆ vars(A)∧ vars(B)∩ vars(R) ⊆ safeVars(A).

5A variant of rule (SJPush), restricted to And-only queries,has been stated (at syntax level) in Lemma 1(2) in [26].

We pass on a detailed discussion of operatorπ, also be-cause – when translating SPARQL queries into algebra ex-pression – this operator appears only at the top-level. Still,we emphasize that also for this operator rewriting rules exist,e.g. allowing to project away unneeded variables at an earlystage. Instead, in the remainder of this section we will presenta thorough discussion of operator\. The latter, in contrastto the other algebraic operations, has no direct counterpart atthe syntactic level. This complicates the encoding of queriesinvolving negation and, as we will see, poses specific chal-lenges to the optimization scheme. We start with the remarkthat, as shown in [1], operator\ can always be encoded atthe syntactic level through a combination of operatorsOpt,Filter, andbnd. We illustrate the idea by example.

Example 2. The following SPARQL expression Q1 andthe corresponding algebra expression A1 select all per-sons for which no name is specified in the data set.

Q1 = Filter¬bnd(?n)((?p, type, P erson) Opt

((?p, type, P erson) And (?p, name, ?n)))A1 = σ¬bnd(?n)(J(?p, type, P erson)K 1

(J(?p, type, P erson)K J(?p, name, ?n)K))

From an optimization point of view it would be desirable tohave a clean translation of this constellation using operator\,but the semantics mapsQ1 intoA1, which contains operatorsσ, 1 , , and predicatebnd, rather than\. In fact, a bettertranslation (based on\) exists for a class of practical queriesand we will provide rewriting rules for such a transformation.

Proposition 2. Let A1, A2 be A-expressions and A−1 ,

A−1 beA−-expressions. The following equivalences hold.

(A1 \A2) \A3 ≡ (A1 \A3) \A2 (MReord)(A1 \A2) \A3 ≡ A1 \ (A2 ∪ A3) (MMUCorr)A1 \A2 ≡ A1 \ (A1 A2) (MJ)

A−

1 1 A−

2 ≡ A−

1 1 (A−

1 A−

2 ) (LJ)

Rules (MReord) and (MMUCorr) are general-purposerewriting rules, listed for completeness. Most important inour context is rule(LJ). It allows to eliminate redundantsubexpressions in the right side of1 -expressions (forA−

6

expressions), e.g. the application of(LJ) simplifiesA1 toA′

1= σ¬bnd(?n)(J(?p, type, Person)K1 J(?p, name, ?n)K).The following lemma allows for further simplification.

Lemma 2. Let A−1 , A

−2 be A

−-expressions, R a filtercondition, and ?x ∈ safeVars(A2)\ vars(A1) a variable.Then σ¬bnd(?x)(A

−1 1 A−

2 ) ≡ A−1 \A

−2 holds.

The application of the lemma to queryA′1yields the expres-

sionA′′1 = J(?p, type, Person)K \ J(?p, name, ?n)K. Com-

bined with rule(LJ), we have established a powerful mech-anism that often allows to make simulated negation explicit.

5. Semantic SPARQL Query Optimization

This chapter complements the discussion of algebraic opti-mization with constraint-based, semantic query optimization(SQO). The key idea of SQO is to find semantically equiv-alent queries over a database that satisfies a set of integrityconstraints. These constraints might have been specified bythe user, extracted from the underlying database, or hold im-plicitly when SPARQL is evaluated on RDFS data coupledwith an RDFS inference system.6 More precisely, given aqueryQ and a set of constraintsΣ over an RDF databaseD s.t.D |= Σ, we want to enumerate (all) queriesQ′ thatcompute the same result onD. We writeQ ≡Σ Q′ if Qis equivalent toQ′ on each databaseD s.t.D |= Σ. Fol-lowing previous approaches [9], we focus on TGDs andEGDs, which cover a broad range of practical constraintsover RDF, such as functional and inclusion dependencies.When talking about constraints in the following we alwaysmean TGDs or EGDs. We refer the interested reader to [19]for motivating examples and a study of constraints for RDF.We represent each constraintα ∈ Σ by a first-order logicformula over a ternary relationTD(s, p, o) that stores alltriples contained in RDF databaseD and useT as the cor-responding relation symbol. For instance, the constraint∀x1, x2(T (x1, p1, x2) → ∃y1T (x1, p2, y1)) states that eachRDF resource with propertyp1 also has propertyp2. Like inthe case of conjunctive queries we call aA+ query minimalif there is no equivalentA+ query with fewer triple patterns.

Our approach relies on the Chase & Backchase (C&B)algorithm for semantic optimization of CQs proposed in [9].Given a CQq and a setΣ of constraints as input, the algorithmoutputs all semantically equivalent and minimalq′ ≡Σ qwhenever the underlying chase algorithm terminates. Wedefer the discussion of chase termination to the subsequentsection and use the C&B algorithm as a black box withthe above properties. Our basic idea is as follows. First, wetranslateAnd-only blocks (or queries), so-called basic graphpatterns (BGPs), into CQs and then apply C&B to optimizethem. We introduce two alternate translation schemes below.

Definition 6. Let S ⊂ V be a finite set of variables andQ ∈ A+ be a SPARQL query defined as

Q = SelectS((s1, p1, o1) And . . . And (sn, pn, on)).

6Note that the SPARQL semantics disregards RDFS infer-ence, but assumes that it is realized in a separate layer.

We define the translation C1(Q) := q, where

q : ans(s)← T (s1, p1, o1), . . . , T (sn, pn, on),

and s is a vector of variables containing exactly thevariables in S. We define C−1

1 (q) as follows. It takes aCQ in the form of q as input and returnsQ if it is a validSPARQL query, i.e. if si ∈ BIV , pi ∈ IV , oi ∈ BILVfor all i ∈ [n]; otherwise, C−1

1 (q) is undefined.

Definition 7. Let Σ be a set of RDF constraints, D anRDF database, and α = ∀x(φ(x) → ∃yψ(x, y)) ∈ Σ.We use h(T (a1, a2, a3)) := a2(a1, a3) if a2 is not a vari-able, otherwise we set it to the empty string. For a con-junction

∧ni=1 T (ai) of atoms, we set h(

∧ni=1 T (ai)) :=∧n

i=1 h(T (ai)). Then, we define the constraint α′ as∀x(h(φ(x))→ ∃yh(ψ(x, y))). We set Σ′ := α′ | α ∈ Σif all α′ are constraints, otherwise Σ′ := ∅.

Let S ⊂ V be a set of variables, Q ∈ A+ defined as

Q = SelectS((s1, p1, o1) And ... And (sn, pn, on)),

and assume that pi is never a variable. We define thetranslationC2(Q) := ans(s)← p1(s1, o1), ..., pn(sn, on),where vector s contains exactly the variables in S. Fora CQ q: ans(s)← R1(x11, x12), ..., Rn(xn1, xn2), we de-note by C−1

2 (q) the expression

SelectS((x11, R1, x12) And ... And (xn1, Rn, xn2))

if it is a SPARQL query, else C−12 (q) is undefined.

C1(Q) andC−11 (Q) constitute straightforward translations

from SPARQLAnd-only queries to CQs and back. The defi-nition ofC2(Q)andC−1

2 (Q)was inspired by the work in [13]and is motivated by the observation that in many real-worldSPARQL queries variables do not occur in predicate position;it is applicable only in this context. Given that the secondtranslation scheme is not always applicable, the reader maywonder why we introduced it. The reason is that the trans-lation schemes are different w.r.t. the termination conditionsfor the subsequent chase that they exhibit. We will comeback to this issue when discussing termination conditions forthe chase in the next section (see Proposition 4).

The translation schemes, although defined forA+ queries,directly carry over toA-expressions, i.e. each expressionQ ∈ A can be rewritten into the equivalentA+-expressionSelectvars(Q)(Q). Coupled with the C&B algorithm, theyprovide a sound approach to semantic query optimization forAnd-only queries whenever the underlying chase algorithmterminates, as stated by the following lemma.

Lemma 3. Let Q an A+-expression, D a database, andΣ a set of EGDs and TGDs.

• If cbΣ(C1(Q)) terminates then ∀Q′ ∈ A+:Q′ ∈ C−1

1 (cbΣ(C1(Q)))⇒ Q′ ≡Σ Q andQ′ minimal.

• If C2(Q) is defined, |Σ′| = |Σ| and cbΣ′(C2(Q)) ter-minates then so does cbΣ(C1(Q)).

• If C2(Q) is defined, |Σ′| = |Σ| and cbΣ′(C2(Q)) ter-minates then ∀Q′ ∈ A+:Q′ ∈ C−1

1 (cbΣ(C1(Q)))⇒ Q′ ≡Σ Q.

7

The converse direction in bullets one and three does nothold in general, i.e. the scheme is not complete. Before weaddress this issue, we illustrate the problem by example.

Example 3. Let the two expressions Q1 := (?x, b,′ l′),Q2 := (?x, b,′ l′) And (?x, a, c), and let the constraint∀x1, x2, x3(T (x1, x2, x3) → T (x3, x2, x1)) be given. Bydefinition, there are no RDF databases that contain aliteral in a predicate position because, according to theconstraint, such a literal would also occur in the subjectposition, which is not allowed. Therefore, the answerto both expressions Q1 and Q2 is always the empty set,which implies Q1 ≡Σ Q2. But it is easy to verify thatC1(Q1) ≡Σ C1(Q2) does not hold. The reason for thisdiscrepancy is that the universal plan [9] of the queriesis not a valid SPARQL query.

We formalize this observation in the next lemma, i.e. pro-vide a precondition that guarantees completeness.7 For aCQ q, we denote byU(q) its universal plan [9], namely theconjunctive queryq′ obtained fromq by chasing its body.

Lemma 4. LetD be a database andQ anA+-expressionsuch that C−1

1 (U(C1(Q))) ∈ A+.

• If cbΣ(C1(Q)) terminates then ∀Q′ ∈ A+ such thatC−1

1 (U(C1(Q′))) ∈ A+:

Q′ ∈ C−11 (cbΣ(C1(Q)))⇔ Q′ ≡Σ Q andQ′ minimal.

• If C2(Q) is defined, |Σ′| = |Σ| and cbΣ′(C2(Q)) ter-minates then ∀Q′∈A+ s.t. C−1

1 (U(C1(Q′))) ∈ A+:

Q′ ∈ C−11 (cbΣ(C1(Q)))⇔ Q′ ≡Σ Q andQ′ minimal.

By now we have established a mechanism that allows us toenumerate equivalent queries of SPARQLAnd-only queries,or BGPs inside queries. Next, we provide extensions that gobeyondAnd-only queries. The first rule in the followinglemma shows that sometimesOpt can be replaced byAnd;informally spoken, it applies when the expression in theOptclause is implied by the constraints. The second rule can beused to eliminate redundant BGPs inOpt-subexpressions.

Lemma 5. Let Q1, Q2, Q3 ∈ A and S ⊂ V a finite setof variables.

• If Q1 ≡Σ Selectvars(Q1)(Q1 And Q2) thenQ1 Opt Q2 ≡Σ Q1 And Q2.

• If Q1 ≡Σ Q1 And Q2 then(Q1 Opt (Q2 And Q3)) ≡Σ Q1 Opt Q3.

Note that the preconditions are always expressed in termsof And-only queries and projection, thus can be checkedusing our translation schemes and the C&B algorithm. Weconclude our discussion of SQO with a lemma that givesrules for the elimination of redundant filter expressions.

Lemma 6. Let Q1, Q2 ∈ A, S ⊂ V \?y a set of vari-ables, ?x, ?y ∈ vars(Q2), Σ a set of constraints, and Da documents s.t. D |= Σ. Further let Q2

?x?y be obtained

from Q2 by replacing each occurrence of ?y by ?x.7We might expect that situations as the one sketched in Ex-ample 3 occur rarely in practice, so the condition in Lemma 4may guarantee completeness in most practical scenarios.

• If Q1 ≡Σ Selectvars(Q1)(Q1 And Q2) thenJFilter¬bnd(?x)(Q1 Opt Q2)KD = ∅.

• If SelectS(Q2) ≡Σ SelectS(Q2?x?y ) then

SelectS(Filter?x=?y(Q2)) ≡Σ SelectS(Q2?x?y ).

• If SelectS(Q2) ≡Σ SelectS(Q2?x?y ) then

JFilter¬?x=?y(Q2)KD = ∅.

We conclude this section with some final remarks. First,we note that semantic optimization strategies are basicallyorthogonal to algebraic optimizations,hence both approachescan be coupled with each other. For instance, we mightget better optimization results when combining the rules forfilter decomposition and pushing in Figure 1(V-VI) with thesemantic rewriting rules for filter expressions in the lemmaabove. Second, as discussed in [9], the C&B algorithm canbe enhanced by a cost function, which makes it easy to factorin cost-based query optimization approaches for SPARQL,e.g. in the style of [23]. This flexibility strengthens theprospectives and practicability of our semantic optimizationscheme. The study of rewriting heuristics and the integrationof a cost function, though, is beyond the scope of this paper.

6. Chase Termination

The applicability of the C&B algorithm, and hence of ourSQO scheme presented in the previous section, depends onthe termination of the underlying chase algorithm. Givenan arbitrary set of constraints it is in general undecidableifthe chase terminates for every database instance [8]; still, inthe past several sufficient termination conditions have beenpostulated [16, 10, 9, 28, 8]. The strongest sufficient con-ditions known so far areweak acyclicity [28], which wasstrictly generalized tostratification in [8], raising the recog-nition problem fromP to coNP. Our SQO approach on topof the C&B algorithm motivated a reinvestigation of thesetermination conditions, and as a key result we present twonovel chase termination conditions for the classical frame-work of relational databases, which empower virtually allapplications that rely on the chase. Whenever we men-tion a database or a database instance in this section, wemean a relational database. We will start our discussionwith a small example run of the chase algorithm. The ba-sic idea is simple: given a database and a set of constraintsas input, it fixes constraint violations in the database in-stance. Consider for example databaseR(a, b) and con-straint∀x1, x2(R(x1, x2) → ∃yR(x2, y)). The chase firstaddsR(b, y1) to the instance, wherey1 is a fresh null value.The constraint is still violated, becausey1 does not occurin the first position of anR-tuple. So, the chase will addR(y1, y2), R(y2, y3), R(y3, y4), . . . in subsequent steps,wherey2, y3, y4, . . . are fresh null values. Obviously, thechase algorithm will never terminate in this toy example.

Figure 2 summarizes the results of this section and putsthem into context. First, we will introduce the novel classof safe constraints, which guarantees the termination of thechase. It strictly subsumes weak acyclicity, but is differentfrom stratification. Building upon the definition of safety,we

8

Figure 2: Chase termination conditions.

then presentsafely restricted constraints as a consequent ad-vancement of our ideas. The latter class strictly subsumes allremaining termination conditions known so far. Finally, wewill show that, based on our framework, we can easily defineanother class calledsafely stratified constraints, which isstrictly contained in the class of safely restricted constraints,but also subsumes weak acyclicity and safeness.

Safe Constraints. The basic idea of the first termina-tion condition is to keep track of positions a newly introducedlabeled null may be copied to. Consider for instance con-straintR(x1, x2, x3), S(x2)→ ∃yR(x2, y, x1), which is notweakly acyclic. Its dependency graph is depicted in Figure3 (left). As illustrated in the toy example in the beginningof this section, a cascading of labeled nulls (i.e. new labelednull values that are created over and over again) may cause anon-terminating chase sequence. However, we can observethat for the constraint above such a cascading of fresh labelednulls cannot occur, i.e. no fresh labeled null can repeatedlycreate new labeled nulls in positionR2 while copying itselfto positionR1. The reason is that the constraint cannot beviolated with a fresh labeled null inR2, i.e. if R(a1, a2, a3)andS(a2) hold, but∃yR(a2, y, a1) does not, thena2 is nevera newly created labeled null. This is due to the fact thata2must also occur in relationS, which is not modified whenchasing only with this single constraint. Consequently, thechase sequence always terminates. We will later see that thisis not a mere coincidence: the constraint is safe.

To formally define safety, we first introduce the notion ofaffected positions. Intuitively, a position is affected if, duringthe application of the chase, a newly introduced labeled nullcan be copied or created in it. Thus, the set of affectedpositions is an overestimation of the positions in which anull value that was introduced during the chase may occur.

Definition 8. [4] Let Σ be a set of TGDs. The set ofaffected positions aff(Σ) of Σ is defined inductively asfollows. Let π be a position in the head of an α ∈ Σ.

• If an existentially quantified variable appears in π,then π ∈ aff(Σ).

• If the same universally quantified variable X ap-pears both in position π, and only in affected po-sitions in the body of α, then π ∈ aff(Σ).

Although we borrow this definition from [4] our focus isdifferent. We extend known classes of constraints for whichthe chase terminates. The focus in [4] is on query answeringin cases the chase may not terminate. Our work neither

Figure 3: Left: Dependency graph. Right: Cor-responding propagation graph (it has no edges).

subsumes [4] nor the other way around. Like in the case ofweak acyclicity, we define the safety condition with the helpof the absence of cycles containing special edges in somegraph. We call this graph propagation graph.

Definition 9. Given a set of TGDs Σ, the propagationgraph prop(Σ) := (aff(Σ), E) is the directed graph de-fined as follows. There are two kinds of edges in E. Addthem as follows: for every TGD ∀x(φ(x)→ ∃yψ(x, y)) ∈Σ and for every x in x that occurs in ψ and every oc-currence of x in φ in position π1

• if x occurs only in affected positions in φ then, forevery occurrence of x in ψ in position π2, add anedge π1 → π2 (if it does not already exist).

• if x occurs only in affected positions in φ then, forevery existentially quantified variable y and for ev-ery occurrence of y in a position π2, add a special

edge π1∗→ π2 (if it does not already exist).

Definition 10. A set Σ of constraints is called safe iffprop(Σ) has no cycles going through a special edge.

The intuition of these definitions is that we forbid an un-restricted cascading of null values, i.e. with the help of thepropagation graph we impose a partial order on the affectedpositions such that any newly introduced null value can onlybe created in a position that has a higher rank in that partialorder in comparison to null values that may occur in the bodyof a TGD. To state this more precisely, assume a TGD of theform ∀x(φ(x) → ∃yψ(x, y)) is violated. Then,I |= φ(a)andI 2 ∃yψ(a, y)) must hold. The safety condition ensuresthat any position in the body that has a newly created labelednull from a in itself and also occurs in the head of the TGDhas a strictly lower rank in our partial order than any positionin which some element fromy occurs. The main differencein comparison to weak acyclicity is that we look in a refinedway (see affected positions) on where a labeled null can bepropagated to. We note that given a set of constraints it canbe decided in polynomial time whether it is safe.

Example 4. Consider the TGD R(x1, x2, x3), S(x2) →∃yR(x2, y, x1) from before. The dependency graph isdepicted in Figure 3 on the left side and its propagationgraph on the right side. The only affected position isR2. From the respective definitions it follows that thisconstraint is safe, but not weakly acyclic.

Note that ifΣ is safe, then every subset ofΣ is safe, too.We will now compare safety to other termination conditions.In the example, the propagation graph is a subgraph of thedependency graph. This is not a mere coincidence.

9

Theorem 7. Let Σ be a set of constraints.

• Then, prop(Σ) is a subgraph of dep(Σ). It holdsthat if Σ is weakly acyclic, then it is also safe.

• There is some Σ that is safe, but not stratified.

• There is some Σ that is stratified, but not safe.

The next result shows that safety guarantees terminationwhile retaining polynomial time data complexity.

Theorem 8. Let Σ be a fixed set of safe constraints.Then, there exists a polynomial Q ∈ N[X ] such that forany database instance I, the length of every chase se-quence is bounded by Q(||I||), where ||I|| is the numberof distinct values in I.

Safely Restricted Constraints. In this section wegeneralize the method of stratification from [8] to a conditionwhich we callsafe restriction. The chase graph from [8]will be a special case of our new notion. We then definethe notion of safe restriction and show that the chase alwaysterminates for constraints obeying it.

Let α := S(x2, x3), R(x1, x2, x3)→ ∃yR(x2, y, x1) andβ := R(x1, x2, x3)→ S(x1, x3). It can be seen thatα ≺ βandβ ≺ α. Further,α, β is not weakly acyclic, so itfollows that α, β is not stratified. Still, the chase willalways terminate: A firing ofα may cause a null value toappear in positionR2, but a firing ofβ will never introducenull values in the head ofβ althoughβ ≺ α holds. This isthe key observation for the upcoming definitions. First, wewill refine the relation≺ from [8]. This refinement helps usto detect if during the chase null values might be copied tothe head of some constraint. Letpos(Σ) denote the set ofpositions that occur in the body of some constraint inΣ.

Definition 11. Let Σ a set of constraints and P ⊆ pos(Σ).For all α, β ∈ Σ, we define α ≺P β iff there are tuplesa, b and a database instance I s.t.

• I 2 α(a),

• β is not applicable on b and I,

• Ia⊕ Cα 2 β(b),

• null values in I occur only in positions from P , and

• the firing of β in the case of bullet three copies somenull value from Ia⊕ Cα to the head of β.

We next introduce a notion for affected positions relativeto a constraint and a set of positions.

Definition 12. For any set of positions P and tgd α letaff-cl(α, P ) be the set of positions π from the head of αsuch that either

• the variable in π occurs in the body of α only inpositions from P or

• π contains an existentially quantified variable.

The latter definition and the refinement of≺ will help usto define the notion of a restriction system, which is a strictgeneralization of the chase graph introduced in [8].

Definition 13. A restriction system is a pair (G′(Σ), f),where G′(Σ) := (Σ, E) is a directed graph and f : Σ→2pos(Σ) is a function such that

• forall TGDs α and forall (α, β) ∈ E:aff-cl(α, f(α)) ∩ pos(β) ⊆ f(β),• forall EGDs α and forall (α, β) ∈ E:f(α) ∩ pos(β) ⊆ f(β), and• forall α, β ∈ Σ: α ≺f(α) β =⇒ (α, β) ∈ E.

We illustrate this definition by an example. It also showsthat restriction systems always exist.

Example 5. Let Σ a set of constraints. Then, (G(Σ), f),where f(α) := pos(α) for all α ∈ Σ is a restrictionsystem for Σ.

Based on the novel technical notion of restriction systemswe can easily define a new class of constraints.

Definition 14. Σ is called safely restricted if and only ifthere is a restriction system (G′(Σ), f) for Σ such thatevery strongly connected component in G′(Σ) is safe.

The next theorem shows that safe restriction strictly ex-tends the notion of stratification and safety.

Theorem 9. If Σ is stratified or safe, then it is also safelyrestricted. There is some Σ that is safely restricted butneither safe nor stratified.

Definition 14 implies that safely restricted constraints canbe recognized by aΣP2 -algorithm. However, with the helpof a canonical restriction system, we can show that saferestriction can be decided incoNP (like stratification).

Theorem 10. Given constraint set Σ it can be checkedby a coNP-algorithm whether Σ is safely restricted.

The next theorem is the main contribution of this section.It states that the chase will always terminate in polynomialtime data complexity for safely restricted constraints.

Theorem 11. Let Σ be a fixed set of safely restrictedconstraints. Then, there exists a polynomial Q ∈ N[X ]such that for any database instance I, the length ofevery chase sequence is bounded by Q(||I||), where ||I||is the number of distinct values in I.

To the best of our knowledge safe restriction is the mostgeneral sufficient termination condition for TGDs and EGDs.We finally compare the chase graph to restriction systems.The reader might wonder what happens if we substitute weakacyclicity with safety in the definition of stratification (in thepreliminaries).

Definition 15. We call Σ safely stratified iff the con-straints in every cycle of G(Σ) are safe.

We obtain the following result, showing that with the helpof restriction systems, we strictly extended the method of thechase graph from [8].

10

Theorem 12. Let Σ be a set of constraints.

• If Σ is weakly acyclic or safe, then it is safely strat-ified.

• If Σ is safely stratified, then it is safely restricted.

• There is some set of constraints that is safely re-stricted, but not safely stratified.

Note that we used safety instead of safe stratification inthe definition of safe restrictedness although safely stratifiedconstraints are the provably larger class. This is due to thefact that safety is easily checkable and would not change theclass of constraints. The next proposition clarifies this issue.

Proposition 3. Σ is safely restricted iff there is a restric-tion system (G′(Σ), f) for Σ such that every stronglyconnected component in G′(Σ) is safely stratified.

In the previous section we proposed two SPARQL transla-tion schemes and it is left to explain why we introduced twoalternative schemes. The next proposition states that the twoschemes behave differently with respect to safe restriction.

Proposition 4. Let Σ be a non-empty set of constraintset over a ternary relation symbol T .

• There is some Σ that is safely restricted, but Σ′ = ∅,i.e. the second translation scheme is not applicable.

• There is some Σ such that |Σ| = |Σ′| and Σ′ is safelyrestricted, but Σ is not.

Referring back to Lemma 3, this means we might checkbothΣ orΣ′ for safe restrictedness, and can guarantee termi-nation of the chase if at least one of them is safely restricted.

7. Conclusion

We have discussed several facets of the SPARQL querylanguage. Our complexity analysis extends prior investi-gations [26] and (a) shows that the combination ofAndandUnion is the main source of complexity inOpt-freeSPARQL fragments and (b) clarifies that yet operatorOptalone makes SPARQL evaluationPSpace-complete. Wealso show that, when restricting the nesting depth ofOpt-expressions, we obtain better complexity bounds.

The subsequent study of SPARQL Algebra lays the foun-dations for transferring established Relational Algebra opti-mization techniques into the context of SPARQL. Addition-ally, we considered specifics of the SPARQL query language,such as rewriting of SPARQL queries involving negation.The algebraic optimization approach is complemented by apowerful framework for semantic query optimization. Weargue that a combination of both algebraic and semanticoptimization will push the limits of existing SPARQL im-plementations and leave the study of a schematic rewritingapproach and good rewriting heuristics as future work.

Finally, our results on chase termination empower the prac-ticability of SPARQL query optimization in the presence ofconstraints and directly carry over to other applications thatrely on the chase, such as [28, 20, 15, 9].

8. References

[1] R. Angles and C. Gutierrez. The Expressive Power ofSPARQL. In ISWC, pages 114–129, 2008.

[2] S. Arora and B. Barak. Computational Complexity: AModern Approach. Cambridge University Press, 2007.

[3] C. Beeri and M. Y. Vardi. A Proof Procedure for DataDependencies. J. ACM, 31(4):718–741, 1984.

[4] A. Calı, G. Gottlob, and M. Kifer. Taming the InfiniteChase: Query Answering under Expressive RelationalConstraints. In Descr. Logics, volume 353, 2008.

[5] U. S. Chakravarthy, J. Grant, and J. Minker. Logic-basedApproach to Semantic Query Optimization. TODS,15(2):162–207, 1990.

[6] R. Cyganiac. A relational algebra for SPARQL. TechnicalReport, HP Laboratories Bristol, 2005.

[7] D. Abadi et al. Scalable Semantic Web Data ManagementUsing Vertical Partitioning. In VLDB, pages 411–422, 2007.

[8] A. Deutsch, A. Nash, and J. Remmel. The Chase Revisited.In PODS, pages 149–158, 2008.

[9] A. Deutsch, L. Popa, and V. Tannen. Query Reformulationwith Constraints. SIGMOD Record, 35(1):65–73, 2006.

[10] A. Deutsch and V. Tannen. XML queries and constraints,containment and reformulation. Theor. Comput. Sci.,336(1):57–87, 2005.

[11] F. Bry et al. Foundations of Rule-based Query Answering.In Reasoning Web, pages 1–153, 2007.

[12] G. H. L. Fletcher and P. W. Beck. A Role-free Approach toIndexing Large RDF Data Sets in Secondary Memory forEfficient SPARQL Evaluation. CoRR, abs/0811.1083, 2008.

[13] G. Serfiotis et al. Containment and Minimization of RDF/SQuery Patterns. In ISWC, pages 607–623, 2005.

[14] C. Gutierrez, C. A. Hurtado, and A. O. Mendelzon.Foundations of Semantic Web Databases. In PODS, pages95–106, 2004.

[15] A. Y. Halevy. Answering Queries Using Views: A Survey.VLDB Journal, pages 270–294, 2001.

[16] D. S. Johnson and A. Klug. Testing Containment ofConjunctive Queries under Functional and InclusionDependencies. In PODS, pages 164–169, 1982.

[17] J. J. King. QUIST: a system for semantic queryoptimization in relational databases. In VLDB, pages510–517, 1981.

[18] L. Sidirourgos et al. Column-store Support for RDF DataManagement: not all swans are white. In VLDB, pages1553–1563, 2008.

[19] G. Lausen, M. Meier, and M. Schmidt. SPARQLingConstraints for RDF. In EDBT, pages 499–509, 2008.

[20] M. Lenzerini. Data Integration: A Theoretical Perspective.In PODS, pages 233–246, 2002.

[21] M. Schmidt et al. An Experimental Comparison of RDFData Management Approaches in a SPARQL BenchmarkScenario. In ISWC, pages 82–97, 2008.

[22] M. Schmidt et al. SP2Bench: A SPARQL PerformanceBenchmark. In ISWC, 2009.

[23] M. Stocker et al. SPARQL Basic Graph PatternOptimization Using Selectivity Estimation. In WWW, 2008.

[24] D. Maier, A. Mendelzon, and Y. Sagiv. Testing Implicationsof Data Dependencies. In SIGMOD, pages 152–152, 1979.

[25] T. Neumann and G. Weikum. RDF-3X: a RISC-styleengine for RDF. PVLDB, 1(1):647–659, 2008.

[26] J. Perez, M. Arenas, and C. Gutierrez. Semantics andComplexity of SPARQL. CoRR, cs/0605124, 2006.

11

[27] A. Polleres. From SPARQL to Rules (and back). In WWW,pages 787–796, 2007.

[28] R. Fagin et al. Data Exchange: Semantics and QueryAnswering. Theor. Comput. Sci., 336(1):89–124, 2005.

[29] Semantic Web Challenge. Billion triples dataset.http://www.cs.vu.nl/~pmika/swc/btc.html.

[30] L. J. Stockmeyer. The polynomial-time hierarchy. Theor.Comput. Sci., 3:1–22, 1976.

[31] C. Weiss, P. Karras, and A. Bernstein. Hexastore: SextupleIndexing for Semantic Web Data Management. In VLDB,pages 1008–1019, 2008.

APPENDIX

A. Proofs of the Complexity Results

This section contains the complexity proofs of theEval-uation problem for the fragments studied in Section 3. Werefer the interested reader to [26] for the proof of Theorem 1.We start with some basics from complexity theory.

A.1 Background from Complexity Theory

Complexity Classes. As usual, we denote byPTime(orP, for short) the complexity class comprising all problemsthat can be decided by a deterministic Turing Machine (TM)in polynomial time, byNP the set of problems that can bedecided by a non-deterministic TM in polynomial time, andby PSpace the class of problems that can be decided by adeterministic TM within polynomial space bounds.

The Polynomial Hierarchy

Given a complexity classC we denote bycoC the set ofdecision problems whose complement can be decided by aTM in classC. Given complexity classesC1 andC2, theclassCC2

1 captures all problems that can be decided by a TMM1 in classC1 enhanced by an oracle TMM2 for solvingproblems in classC2. Informally, engineM1 can useM2

to obtain ayes/no-answer for a problem inC2 in a singlestep. We refer the interested reader to [2] for a more formaldiscussion of oracle machines. Finally, we define the classesΣPi andΠPi recursively as

ΣP0 = ΠP0 :=P andΣPn+1:=NPΣP

n , and put

ΠPn+1:=coNPΣP

n .

The polynomial hierarchyPH [30] is then defined as

PH =⋃i∈N0

ΣPi

It is folklore that ΣPi = coΠPi , and thatΣPi ⊆ ΠPi+1

andΠPi ⊆ ΣPi+1 holds. Moreover, the following inclusionhierarchies forΣPi andΠPi are known.

P = ΣP0 ⊆ NP = ΣP1 ⊆ ΣP2 ⊆ · · · ⊆ PSPACE, andP = ΠP0 ⊆ coNP = ΠP1 ⊆ ΠP2 ⊆ · · · ⊆ PSPACE.

Complete Problems

We consider completeness only with respect to polynomial-time many-one reductions. QBF, the tautology test for quan-tified boolean formulas, is known to bePSpace-complete[2]. Forms of QBF with restricted quantifier alternation arecomplete for classesΠPi orΣPi depending on the question ifthe first quantified of the formula is∀ or∃. A more thoroughintroduction to complete problems in the polynomial hierar-chy can be found in [2]. Finally, theNP-completeness oftheSetCover-problem and the3Sat-problem is folklore.

12

A.2 OPT-free Fragments (Theorem 2)

FragmentU : UNION (Theorem 2(1))

For aUnion-only expressionP and data setD it sufficesto check ifµ ∈ JtKD for any triple patternt in P . This caneasily be achieved in polynomial time.

FragmentFU : FILTER + UNION (Theorem 2(1))

We present aPTime-algorithm that solves theEvaluationproblem for this fragment. It is defined on the structure ofthe input expressionP and returnstrue if µ ∈ JP KD, falseotherwise. We distinguish three cases. (a) IfP = t isa triple pattern, we returntrue if and only if µ ∈ JtKD.(b) If P = P1 Union P2 we (recursively) check ifµ ∈JP1KD ∨ µ ∈ JP2KD holds. (c) IfP = P1 Filter R forany filter conditionR we returntrue if and only if µ ∈JP1KD ∧ R |= µ. It is easy to see that the above algorithmruns in polynomial time. Its correctness follows from thedefinition of the algebraic operators∪ andσ.

FragmentAU : AND + UNION (Theorem 2(2))

In order to show thatEvaluation for this fragment isNP-complete we have to show membership and hardness.

Membership in NP. Let P be a SPARQL expressioncomposed of operatorsAnd andUnion, D a document,andµ a mapping. We provide anNP-algorithm that returnstrue if µ ∈ JP KD, and false otherwise. Our algorithm isdefined on the structure ofP . (a) If P = t return true ifµ ∈ JtKD , false otherwise. (b) IfP = P1 Union P2, wereturn the truth value ofµ ∈ JP1KD ∨ µ ∈ JP2KD. (c) IfP = P1 And P2, we guess a decompositionµ = µ1 ∪ µ2

and return the truth value ofµ1 ∈ JP1KD ∧ µ2 ∈ JP2KD.Correctness of the algorithm follows from the definition ofthe algebraic operators and∪. It can easily be realized bya non-deterministic TM that runs in polynomial time, whichproves membership inNP.

NP-Hardness. We reduce theSetCover problem to theEvaluation problem for SPARQL (in polynomial time).SetCover is known to beNP-complete, so the reductiongives us the desired hardness result.

The decision version ofSetCover is defined as follows.Let U = u1, . . . , uk be a universe,S1, . . . Sn ⊆ U besets overU , and letk be positive integer. Is there a setI ⊆ 1, . . . , n of size| I |≤ k s.t.

⋃i∈I Si = U?

We use the fixed databaseD := (a, b, 1) for our en-coding and represent each setSi = x1, x2, . . . , xm by aSPARQL expression of the form

PSi:= (a, b, ?X1) And . . . And (a, b, ?Xm).

The setS = S1, . . . , Sn of all Si is then encoded as

PS := PS1Union . . . Union PSn

.

Finally we define the SPARQL expression

P := PS And . . . And PS ,wherePS appears exactlyk times.

It is straightforward to show thatSetCover is true ifand only ifµ = ?U1 7→ 1, . . . , ?Uk 7→ 1 ∈ JP KD. Theintuition of the encoding is as follows.PS encodes all subsetsSi. A set element, sayx, is represented in SPARQL by amapping from variable?X to value1. The encoding ofPallows us to merge (at most)k arbitrary setsSi. We finallycheck if the universeU can be constructed this way.

Remark 2. The proof above relies on the fact that map-ping µ is part of the input of the Evaluation problem.In fact, when fixing µ the resulting (modified) versionof the Evaluation problem can be solved in PTime.

A.3 Fragments Including Operator OPT

We now discuss several fragments includingOpt. Onegoal here is to show that fragmentO is PSpace-complete(c.f. Theorem 3);PSpace-completeness for all fragmentsinvolving Opt then follows (cf. Corollary 1). Given thePSpace-completeness results for fragmentE = AFOU , itsuffices to provehardness for all smaller fragments; mem-bership is implicit. Our road map is as follows.

1. We first showPSpace-hardness for fragmentAFO.

2. We then showPSpace-hardness for fragmentAO.

3. Next, a rewriting of operatorAnd byOpt is presented,which can be used to eliminate allAnd operators inthe proof of (2).PSpace-completeness forO then isshown using this rewriting rule.

4. Finally, we prove Theorem 4, i.e. show that fragmentE≤n isΣPn+1-complete, making use of part (1).

FragmentAFO: AND + FILTER + OPT

We present a (polynomial-time) reduction fromQBF toEvaluation for fragmentAFO. TheQBF problem isknown to bePSpace-complete, so this reduction gives us thedesiredPSpace-hardness result. Membership inPSpace,and hencePSpace-completeness, then follows from Theo-rem 1(3).QBF is defined as follows.

QBF: given a quantified boolean formula of the form

ϕ = ∀x1∃y1∀x2∃y2 . . .∀xm∃ymψ,

whereψ is a quantifier-free boolean formula,as input: isϕ valid?

The following proof was inspired by the proof of Theo-rem 3 in [26]: we encode the inner formulaψ usingAndandFilter, and then adopt the translation scheme for thequantifier sequence∀∃∀∃ . . . proposed in [26].

13

First note that, according to the problem statement,ψ is aquantifier-free boolean formula. We assume w.l.o.g. thatψis composed of∧, ∨ and¬.8 We use the fixed database

D := (a, tv, 0), (a, tv, 1), (a, false, 0), (a, true, 1)

and denote byV = v1, . . . vl the set of variables appear-ing in ψ. Formulaψ then is encoded as

Pψ:=((a, tv, ?V1) And (a, tv, ?V2) And . . .And (a, tv, ?Vl)) Filter f (ψ),

wheref (ψ) is a function that generates a SPARQL condi-tion that mirrors the boolean formulaψ. More precisely,f isdefined recursively on the structure ofψ as

f(vi) := ?Vi = 1f(ψ1 ∧ ψ2) := f(ψ1) ∧ f(ψ2)f(ψ1 ∨ ψ2) := f(ψ1) ∨ f(ψ2)f(¬ψ1) := ¬ f(ψ1)

In our encodingPψ, theAnd-block generates all possiblevaluations for the variables, while theFilter-expressionretains exactly those valuations that satisfy formulaψ. It isstraightforward to show thatψ is satisfiable if and only ifthere exists a mappingµ ∈ JPψKD and, moreover, for eachmappingµ ∈ JPψKD there is a truth assignmentρµ definedas ρµ(x) = µ(?X) for all variables?Xi, ?Yi ∈ dom(µ)such thatµ ∈ JPψKD if and only if ρµ satisfiesψ. GivenPψ , we can encode the quantifier-sequence using a series ofnestedOpt statements as shown in [26]. To make the proofself-contained, we shortly summarize this construction.

SPARQL variables?X1, . . . , ?Xm and ?Y1, . . . Ym areused to represent variablesx1, . . . xm and y1, . . . , ym, re-spectively. In addition to these variables, we use fresh vari-ables?A0, . . .?Am, ?B0, . . .?Bm, and operatorsAnd andOpt to encode the quantifier sequence∀x1∃y1 . . . ∀xm∃ym.For eachi ∈ [m] we define two expressionsPi andQi

Pi := ((a, tv, ?X1) And . . . And (a, tv, ?Xi) And(a, tv, ?Y1) And . . . And (a, tv, ?Yi−1) And(a, false, ?Ai−1) And (a, true, ?Ai)),

Qi := ((a, tv, ?X1) And . . . And (a, tv, ?Xi) And(a, tv, ?Y1) And . . . And (a, tv, ?Yi) And(a, false, ?Bi−1) And (a, true, ?Bi)),

and encodePϕ as

Pϕ:= ((a, true, ?B0)Opt (P1 Opt (Q1

Opt (P2 Opt (Q2

. . .Opt (Pm Opt (Qm And Pψ)) . . . )))))

8In [26] ψ was additionally restricted to be in CNF. We relaxthis restriction here.

It can be shown thatµ = ?B0 7→ 1 ∈ JPϕKD iff ϕ isvalid, which completes the reduction. We refer the reader tothe proof of Theorem 3 in [26] for this part of the proof.

Remark 3. The proof for this fragment (AFO) is sub-sumed by the subsequent proof, which shows PSpace-hardness for a smaller fragment. It was included toillustrate how to encode quantifier-free boolean formu-las that are not in CNF. Some of the following proofsbuild upon this construction.

FragmentAO: AND + OPT

We reduce theQBF problem toEvaluation for classAO.We encode a quantified boolean formula of the form

ϕ = ∀x1∃y1∀x2∃y2 . . . ∀xm∃ymψ,

whereψ is a quantifier-free formula in conjunctive normalform (CNF), i.e.ψ is a conjunction of clauses

ψ = C1 ∧ · · · ∧ Cn,

where theCi, 1 ≤ i ≤ n, are disjunctions of literals.9

By V we denote the variables inψ and byVCithe variables

appearing in clauseCi (either as positive of negative literals).We use the following database, which is polynomial in thesize of the query.

D :=(a, tv, 0), (a, tv, 1), (a, false, 0), (a, true, 1) ∪(a, vari, v) | v ∈ VCi

∪ (a, v, v) | v ∈ V

For eachCi = v1∨· · ·∨vj∨¬vj+1∨· · ·∨¬vk, where thev1 . . . vj are positive and thevj+1 . . . vk are negated variables(contained inVCi

), we define a separate SPARQL expression

PCi:=(. . . ((. . . (

(a, vari, ?vari)Opt ((a, v1, ?vari) And (a, true, ?V1))). . .Opt ((a, vj , ?vari) And (a, true, ?Vj)))Opt ((a, vj+1, ?vari) And (a, false, ?Vj+1))). . .Opt ((a, vk, ?vari) And (a, false, ?Vk)))

and encode formulaψ as

Pψ := PC1And . . . And PCn

.

It is straightforward to verify thatψ is satisfiable if andonly if there is a mappingµ ∈ JPψKD and, moreover, foreachµ ∈ JPψKD there is a truth assignmentρµ defined asρµ(x) = µ(?X) for all variables?Xi, ?Yi ∈ dom(µ) suchthatµ ∈ JPψKD if and only if ρµ satisfiesψ. Now, given9In the previous proof (for fragment AFO) there was nosuch restriction for formula ψ. Still, it is known that QBF isalso PSpace-complete when restricting to formulae in CNF.

14

Pψ , we encode the quantifier-sequence using only operatorsOpt andAnd, as shown in the previous proof for fragmentAFO. For the resulting encodingPϕ, it analogously holdsthatµ = ?B0 7→ 1 ∈ JPϕKD iff ϕ is valid.

We provide a small example that illustrates the translationscheme for QBF presented in in the proof above.

Example 6. We show how to encode the QBF

ϕ = ∀x1∃y1(x1 ⇔ y1)= ∀x1∃y1((x1 ∨ ¬y1) ∧ (¬x1 ∨ y1)),

where ψ = ((x1 ∨ ¬y1) ∧ (¬x1 ∨ y1)) is in CNF. Itis easy to see that the QBF formula ϕ is a tautology.The variables in ψ are V = x1, y1; further, we haveC1 = x1 ∨ ¬y1, C2 = ¬x1 ∨ y1, and VC1

= VC2= V =

x1, y1. Following the construction in the proof we setup the database

D := (a, tv, 0), (a, tv, 1), (a, false, 0), (a, true, 1),(a, var1, x1), (a, var1, y1), (a, var2, x1),(a, var2, y1), (a, x1, x1), (a, y1, y1)

and define expression Pψ = PC1And PC2

, where

PC1:=((a, var1, ?var1)

Opt ((a, x1, ?var1) And (a, true, ?X1)))Opt ((a, y1, ?var1) And (a, false, ?Y1))

PC2:=((a, var2, ?var2)

Opt ((a, y1, ?var2) And (a, true, ?Y1)))Opt ((a, x1, ?var2) And (a, false, ?X1)).

When evaluating these expressions we get:

JPC1KD = (?var1 7→ x1, ?var1 7→ y1

1 ?var1 7→ x1, ?X1 7→ 1)1 ?var1 7→ y1, ?Y1 7→ 0

= ?var1 7→ x1, ?X1 7→ 1, ?var1 7→ y1, ?Y1 7→ 0

JPC2KD = ?var2 7→ x1, ?var2 7→ y1

1 ?var2 7→ y1, ?Y1 7→ 11 ?var2 7→ x1, ?X1 7→ 0

= ?var2 7→ x1, ?X1 7→ 0, ?var2 7→ y2, ?Y2 7→ 1

JPψKD = JPC1And PC2

KD= ?var1 7→ x1, ?var2 7→ y1, ?X1 7→ 1, ?Y1 7→ 1,

?var1 7→ y1, ?var2 7→ x1, ?X1 7→ 0, ?Y1 7→ 0

Finally, we set up the expressions P1 and Q1, as de-scribed in the proof for fragment AOF

P1 := ((a, tv, ?X1) And (a, false , ?A0)And (a, true, ?A1))

Q1 := ((a, tv, ?X1) And (a, tv, ?Y1)And (a, false , ?B0) And (a, true, ?B1))

and encode the quantified boolean formula ϕ as

Pϕ := (a, true, ?B0) Opt (P1 Opt (Q1 And Pψ))

We leave it as an exercise to verify that the mappingµ = ?B0 7→ 1 is contained in JPϕKD. This resultconfirms that the original formula ψ is valid.

FragmentO: OPT-only (Theorem 3)

We start with a transformation rule for operatorAnd; itessentially expresses the key idea of the subsequent proof.

Lemma 7. Let

• Q,Q1, Q2, . . . , Qn (n ≥ 2) be SPARQL expressions,

• S = vars(Q)∪vars(Q1 )∪vars(Q2 )∪· · ·∪vars(Qn),denote the set of variables in Q,Q1, Q2, . . . , Qn

• D = (a, true, 1), (a, false, 0), (a, tv, 0), (a, tv, 1)be a fixed database,

• ?V2, ?V3, . . . , ?Vn be a set of n − 1 fresh variables,i.e. S ∩ ?V2, ?V3, . . . , ?Vn = ∅ holds.

Further, we define

Q′:=((. . . ((Q Opt V2) Opt V3) . . . ) Opt Vn),Q′′:=((. . . ((Q1 Opt (Q2 Opt V2))

Opt (Q3 Opt V3)). . .Opt (Qn Opt Vn))),

Vi:=(a, true, ?Vi), andV i:=(a, false, ?Vi).

The following claims hold.

(1) JQ′KD = µ ∪ ?V2 7→ 1, . . . , ?Vn 7→ 1 | µ ∈ JQKD,(2) JQ′ Opt (Q1 And Q2 And . . . And Qn)KD= JQ′ Opt (. . . ((Q′′ Opt V 2)

Opt V 3). . .Opt V n)KD

The second part of the lemma provides a way to rewrite anAnd-only expression that is encapsulated in the right sideof anOpt-expression by means of anOpt-only expression.Before proving the lemma, we illustrate the construction bymeans of a small example.

Example 7. LetD be the database given in the previouslemma and consider the expressions

Q := (a, tv, ?a) ,i.e. JQKD = ?a 7→ 0, ?a 7→ 1Q1 := (a, true, ?a) ,i.e. JQ1KD= ?a 7→ 1Q2 := (a, false, ?b) ,i.e. JQ2KD= ?b 7→ 0

As for the part (1) of the lemma we observe that

JQ′KD= JQ Opt V2KD= JQ Opt (a, true, ?V2)KD= ?a 7→ 0, ?V2 7→ 1, ?a 7→ 1, ?V2 7→ 1.

Concerning part (2) it holds that the left side

JQ′ Opt (Q1 And Q2)KD= JQ′KD 1 ?a 7→ 1, ?b 7→ 0= ?a 7→ 0, ?V2 7→ 1, ?a 7→ 1, ?b 7→ 0, ?V2 7→ 1

is equal to the right side

JQ′ Opt ((Q1 Opt (Q2 Opt V2)) Opt V2)KD= JQ′KD 1 (?a 7→ 1, ?b 7→ 0, ?V2 7→ 1 1 JV2KD)= JQ′KD 1 ?a 7→ 1, ?b 7→ 0, ?V2 7→ 1= ?a 7→ 0, ?V2 7→ 1, ?a 7→ 1, ?b 7→ 0, ?V2 7→ 1.

15

Proof of Lemma 7. We omit some technical details,but instead give the intuition of the encoding. (1) The firstclaim follows trivially from the definition ofQ′, the ob-servations that eachVi evaluates to?Vi 7→ 1, and thefact that all?Vi are unbound inQ′ (recall that, by assump-tion, the ?Vi are fresh variables). To prove (2), we con-sider the evaluation of the right side expression, in order toshow that it yields the same result as the left side. Firstconsider subexpressionQ′′ and observe that the result ofevaluatingQi Opt Vi is exactly the result of evaluatingQiextended by the binding?Vi 7→ 1. In the sequel, we useQVi as an abbreviation forQi Opt Vi, i.e. we denoteQ′′ as((. . . ((Q1 Opt QV2 ) Opt QV3 ) Opt . . . ) Opt QVn ). Ap-plying semantics, we can rewriteJQ′′KD into the form

JQ′′KD= J((. . . ((Q1 Opt QV2 ) Opt QV3 ) Opt . . . ) Opt QVn )KD= J(Q1 And QV2 And QV3 And . . . And QVn )KD ∪ PD,

where we call the left subexpression of the unionjoinpart, andPD at the right side is an algebra expression (overdatabaseD) with the following property: for each mappingµ ∈ PD there is at least one?Vi (2 ≤ i ≤ n) s.t. ?Vi 6∈dom(µ). We observe that, in contrast, for each mappingin the join partdom(µ) ⊇ ?V2, . . . , ?Vn holds and, evenmore,µ(?Vi) = 1, for 2 ≤ i ≤ n. Hence, the mappings inthe result of the join part are identified by the property that?V2, ?V3, . . . , ?Vn are all bound to 1.

Let us next consider the evaluation of the larger expression(on the right side of the original equation)

R := ((. . . ((Q′′ Opt V 2) Opt V 3) Opt . . . ) Opt V n)).

When evaluatingR, we obtain exactly the mappings fromJQ′′KD, but each mappingµ ∈ JQ′′KD is extended by bind-ings?Vi 7→ 0 for all ?Vi 6∈ dom(µ) (cf. the argumentationin for claim (1)). As argued before, all mappings in the joinpart ofQ′′ are complete in the sense that all?Vi are bound,so these mappings will not be affected. The remaining map-pings (i.e. those originating fromPD) will be extended bybindings?Vi 7→ 0 for at least one?Vi. The resulting situationcan be summarized as follows: all mappings resulting fromthe join part ofQ′′ bind all variables?Vi to 1; all mappingsin PD bind all?Vi, but at least one of them is bound to 0.

From part (1) we know that each mapping inJQ′KD mapsall ?Vi to 1. Hence, when computingJQ′ Opt RKD =JQ′KD 1 JRKD, the bindings?Vi 7→ 1 for all µ ∈ JQ′KDserves as a filter that removes the mappings inJRKD origi-nating fromPD. This means

JQ′ Opt RKD= JQ′KD 1 JRKD= JQ′KD 1 J(Q1 And QV2 And QV3 And . . . And QVn )KD= JQ′ Opt (Q1 And QV2 And QV3 And . . . And QVn )KD.

Even more, we observe that all?Vi are already bound inQ′ (all of them to1), so the following rewriting is valid.

JQ′ Opt RKD= JQ′ Opt (Q1 And QV2 And QV3 And . . . And QVn )KD= JQ′ Opt (Q1 And Q2 And Q3 And . . . And Qn)KD

Thus, we have shown that the equivalence holds. Thiscompletes the proof.

Given Lemma 7 we are now in the position to provePSpace-completeness for fragmentO. As in previousproofs it suffices to show hardness; membership follows asbefore from thePSpace-completeness of fragmentE .

The proof idea is the following. We show that, in theprevious reduction fromQBF toEvaluation for fragmentAO, eachAnd expression can be rewritten using onlyOptoperators. We start with a QBF of the form

ϕ = ∀x1∃y1∀x2∃y2 . . . ∀xm∃ymψ,

whereψ is a quantifier-free formula in conjunctive normalform (CNF), i.e.ψ is a conjunction of clauses

ψ = C1 ∧ · · · ∧ Cn,

where theCi, 1 ≤ i ≤ n, are disjunctions of literals.By V we denote the set of variables insideψ and byVCi

the variables appearing in clauseCi (either in positive ofnegative form) and use the same database as in the proof forfragmentAO, namely

D :=(a, tv, 0), (a, tv, 1), (a, false, 0), (a, true, 1) ∪(a, vari, v) | v ∈ VCi

∪ (a, v, v) | v ∈ V

The first modification of the proof for classAO concernsthe encoding of clausesCi = v1∨· · ·∨vj∨¬vj+1∨· · ·∨¬vk,where thev1 . . . vj are positive andvj+1 . . . vk are negatedvariables. In the prior encoding we used bothAnd andOptoperators to encode them. It is easy to see that we can sim-ply replace eachAnd operator there throughOpt withoutchanging semantics. The reason is that, for all subexpres-sionsAOpt B in the encoding, it holds thatvars(A) ∩vars(B) = ∅ andJBKD 6= ∅; hence, all mappings inA arecompatible with all mappings inB and there is at least onemapping inB. When applying this modification, we obtainthe followingO-encoding for clausesCi.

PCi:=(. . . ((. . . (

(a, vari, ?vari)Opt ((a, v1, ?vari) Opt (a, true, ?V1))). . .Opt ((a, vj , ?vari) Opt (a, true, ?Vj)))Opt ((a, vj+1, ?vari) Opt (a, false, ?Vj+1))). . .Opt ((a, vk, ?vari) Opt (a, false, ?Vk))),

Let us next consider thePi andQi used for simulatingthe quantifier alternations. The original definition of theseexpression was given in the proof for fragmentAFO. Witha similar argumentation as before we can replace each oc-currence of operatorAnd throughOpt without changing

16

the semantics of the whole expression. This results in thefollowingO encodings forPi andQi, i ∈ [m].

Pi := ((a, tv, ?X1) Opt . . . Opt (a, tv, ?Xi) Opt(a, tv, ?Y1) Opt . . . Opt (a, tv, ?Yi−1) Opt(a, false, ?Ai−1) Opt (a, true, ?Ai))

Qi := ((a, tv, ?X1) Opt . . . Opt (a, tv, ?Xi) Opt(a, tv, ?Y1) Opt . . . Opt (a, tv, ?Yi) Opt(a, false, ?Bi−1) Opt (a, true, ?Bi)),

In the underlying proof forAO, the conjunctionψ isencoded asPC1

And . . . And PCn, thus we have not yet

eliminated allAnd-operators. We shortly summarize whatwe have achieved so far:

Pϕ:= ((a, true, ?B0)Opt (P1 Opt (Q1

. . .Opt (Pm−1 Opt (Qm−1

Opt (P ′))) . . . ))), where

P ′ = Pm Opt (Qm And (Pψ))= Pm Opt (Qm And PC1

And . . . And PCn)

Note thatP ′ is the only expression that still containsAndoperators (whereQm,PC1

, . . . ,PCnare alreadyAnd-free).

We now exploit the rewriting given in Lemma 7. In particular,we replaceP ′ in Pϕ by the expressionP ′

∗ defined as

P ′∗ := Q′ Opt

((. . . ((Q′′ Opt V 2) Opt V 3) Opt . . . ) Opt V n+1)),

whereQ′:=((. . . ((Pm Opt V2) Opt V3) . . . ) Opt Vn+1),Q′′:=((. . . ((Qm Opt (PC1

Opt V2))Opt (PC2

Opt V3)). . .Opt (PCn

Opt Vn+1))),Vi:=(a, true, ?Vi),V i:=(a, false, ?Vi),and the?Vi (i ∈ 2, . . . , n+ 1) are fresh variables.

The resultingPϕ is now anO-expression. From Lemma 7it follows that the result of evaluatingP ′

∗ is obtained fromthe result ofP ′ by extending each mapping inP ′ by ad-ditional variables, more precisely by?V2 7→ 1, ?V3 7→1, . . . , ?Vn+1 7→ 1, i.e. the results are identical modulothis extension. It is straightforward to verify that these ad-ditional bindings do not harm the construction, i.e. it holdsthat?B0 7→ 1 ∈ JPϕKD iff ϕ is valid.

ΣPn+1-completeness of FragmentE≤n (Theorem 4)

We start with two lemmas that will be used in the proof.

Lemma 8. Let

D = (a, tv, 0), (a, tv, 1), (a, true, 1), (a, false, 0)

be an RDF database and F = ∀x1∃y1 . . . ∀xm∃ymψ(m ≥ 1) be a QBF, where ψ is a quantifier-free booleanformula. There is an E≤2m encoding enc(F ) of F s.t.

1. F is valid exactly if ?B0 7→ 1 ∈ Jenc(F )KD

2. F is invalid exactly if all mappings µ′ ∈ Jenc(F )KDare of the form µ′ = µ′

1 ∪ µ′2, where µ

′1 ∼ µ′

2 andµ′1 = ?B0 7→ 1, ?A1 7→ 1.

Proof: The lemma follows from thePSpace-hardnessproof for fragmentAFO, where we have shown how toencodeQBF for a (possibly non-CNF) inner formulaψ.

Lemma 9. Let A and B SPARQL expressions for whichthe evaluation problem is in ΣPi , i ≥ 1, and let R aFilter condition. The following claims hold.

1. The Evaluation problem for the SPARQL expres-sion A Union B is in ΣPi .

2. The Evaluation problem for the SPARQL expres-sion A And B is in ΣPi .

3. The Evaluation problem for the SPARQL expres-sion A Filter R is in ΣPi .

Proof: 1. According to the SPARQL semantics we havethatµ ∈ JA Union BK if and only if µ ∈ JAK or µ ∈ JBK.By assumption, both conditions can be checked individuallyin ΣPi , and so can both checks in sequence.

2. It is easy to see thatµ ∈ JA And BK iff µ canbe decomposed into two compatible mappingsµ1 andµ2

s.t.µ = µ1 ∪ µ2 andµ1 ∈ JAK andµ2 ∈ JBK. By assump-tion, testingµ1 ∈ JAK (µ2 ∈ JBK) is in ΣPi . Sincei ≥ 1,this complexity class is at leastΣP1 = NP. So we can guessa decompositionµ = µ1 ∪µ2 and test for the two conditionsone after the other. Hence, the whole procedure is inΣPi .

3. µ ∈ JA Filter RK holds iff µ ∈ JAK, which can betested inΣP1 by assumption, andR satisfiesµ, which can betested in polynomial time. SinceΣPi ⊇ NP ⊇ P for i ≥ 1,the whole procedure is still inΣPi .

We are now ready to prove Theorem 4. The proof dividesinto two parts, i.e. hardness and membership. The hard-ness proof is a reduction fromQBF with a fixed number ofquantifier alternations. Second, we prove by induction ontheOpt-rank that there exists aΣPn+1-algorithm to solve theEvaluation problem forE≤n expressions.

Hardness. We consider a QBF of the form

ϕ = ∃x0∀x1∃x2 . . .Qxn ψ,

wheren ≥ 1, Q = ∃ if n is even,Q = ∀ if n is odd,andψ is a quantifier-free boolean formula. It is known thattheValidity problem for such formulae isΣPn+1-complete.We now present a (polynomial-time) reduction from theVa-lidity problem for these quantified boolean formulae totheEvaluation problem for theE≤n fragment, to proveΣPn+1-hardness. We distinguish two cases.

Case 1: LetQ = ∃, so the formula is of the form

17

F = ∃y0∀x1∃y1 . . . ∀xm∃ymψ.

The formulaF has2m + 1 quantifier alternations, so weneed to find anE≤2m encoding for this expressions. WerewriteF into an equivalent formulaF = F1 ∨ F2, where

F1 =∀x1∃y1 . . . ∀xm∃ym(ψ ∧ y0), andF2 =∀x1∃y1 . . . ∀xm∃ym(ψ ∧ ¬y0).

According to Lemma 8 there is a fixed documentD andE≤2m encodingsenc(F1) andenc(F2) (for F1 andF2, re-spectively) s.t.Jenc(F1)KD (Jenc(F2)KD) contains the map-pingµ = ?B0 7→ 1 if and only ifF1 (F2) is valid. Then theexpressionenc(F ) = enc(F1) Union enc(F2) containsµif and only if F1 or F2 is valid, i.e. iff F is valid. Clearly,enc(F ) is anE≤2m expression, soenc(F ) constitutes thedesiredE≤2m encoding of theEvaluation problem.

Case 2: LetQ = ∀, so the formula is of the form

F = ∃x0∀y0∃x1∀x1 . . .∃xm∀ymψ.

F has2m+2 quantifier alternations, so we need to providea reduction into theE≤2m+1 fragment. We eliminate theouter∃-quantifier by rewritingF asF = F1 ∨ F2, where

F1 =∀y0∃x1∀y1 . . . ∃xm∀ym(ψ ∧ y0), andF2 =∀y0∃x1∀y1 . . . ∃xm∀ym(ψ ∧ ¬y0).

Abstracting from the details of the inner formula, bothF1

andF2 are of the form

F ′ = ∀y0∃x1∀y1 . . . ∃xm∀ymψ′,

whereψ′ is a quantifier-free boolean formula. We nowshow (*) that we can encodeF ′ by E≤2m+1 expressionsenc(F ′) that, evaluated on a fixed documentD, yields a fixedmappingµ exactly ifF ′ is valid. This is sufficient, becausethen the expressionenc(F1) Union enc(F2) is anE≤2m+1

that containsµ exactly if the original formulaF = F1 ∨ F2

is valid. We again start by rewritingF ′:

F ′ = ∀y0∃x1∀y1 . . . ∃xm∀ymψ′

= ¬∃y0∀x1∃y1 . . .∀xm∃ym¬ψ′

= ¬(F ′1 ∨ F

′2), where

F ′1 = ∀x1∃y1 . . . ∀xm∃ym(¬ψ′ ∧ y0), andF ′2 = ∀x1∃y1 . . . ∀xm∃ym(¬ψ′ ∧ ¬y0).

According to Lemma 8, eachF ′i can be encoded by an

E≤2m expressionsenc(F ′i ) s.t., on the fixed databaseD

given there, (1)µ = ?B0 7→ 1 ∈ JF ′i KD iff F ′

i is validand (2) if F ′

i is not valid, then all mappingsJenc(F ′i )KD

bind both variables?A1 and?B0 to 1. Then the same condi-tions (1) and (2) hold forenc(F ′

1) Union enc(F ′2) exactly

if F1 ∨ F2 is valid. Now consider the expressionQ =((a, false , ?A1) Opt (enc(F1) Union enc(F2)). This ex-pression containsµ′ = ?A1 7→ 0 (when evaluated on thedatabase given in Lemma 8) if and only ifF ′

1 ∨ F′2 is not

valid (since otherwise, there is a compatible mapping forµ′,namely?B0 7→ 1 in Jenc(F1) Union enc(F2)KD). Insummary, this meansµ′ ∈ JQKD if and only if¬(F ′

1∨F′2) =

F ′ holds. Since bothenc(F1) andenc(F2) areE≤2m ex-pressions,Q is contained inE≤2m+1, so (*) holds.

Membership. We next prove membership ofE≤n expres-sions inΣPn+1 by induction on theOpt-rank. Let us assumethat for eachE≤n expression (n ∈ N0) Evaluation is inΣPn+1. As stated in Theorem 1(2),Evaluation is ΣP1 =NP-complete forOpt-free expressions, so the hypothesisholds for the basic case. In the induction step we increasetheOpt-rank fromn to n+ 1.

We distinguish several cases, depending on the structureof the expression, sayA, with Opt-rankn+ 1.

Case 1: Checking if µ ∈ JA Opt BK. First note thatAOpt B is in E≤n+1, and from the definition ofOpt-rankr it follows immediately that bothA andB are in E≤n.Hence, by induction hypothesis, bothA andB can be eval-uated inΣPn+1. By semantics, we have thatJAOpt BK =JA And BK ∪ (JAK \ JBK), soµ is in JAOpt BK iff it isgenerated by the (a) left or (b) right side of the union. Fol-lowing Lemma 9, part (a) can be checked inΣPn+1. (b) Themore interesting part is to check ifµ ∈ JAK \ JBK. Ac-cording to the semantics of operator\, this check can beformulated asC = C1 ∧ C2, whereC1 = µ ∈ JAK andC2 = ¬∃µ′ ∈ JBK : µ and µ′ are compatible. By induc-tion hypothesis,C1 can be checked inΣPn+1. We argue thatalso¬C2 = ∃µ′ ∈ JBK : µ and µ′ are compatible can beevaluated inΣPn+1: we can guess a mappingµ′ (in NP), thencheck ifµ ∈ JBK (in ΣPn+1), and finally check ifµ andµ′ arecompatible (in polynomial time). SinceP ⊆ NP ⊆ ΣPn+1,all these checks in sequence can be done inΣPn+1. Check-ing if the inverse problem, i.e.C2, holds is then possible incoΣPn+1 = ΠPn+1. Summarizing cases (a) and (b) we ob-serve that (a)ΣPn+1 and (b)ΠPn+1 are contained inΣPn+2,so both checks in sequence can be executed inΣPn+2, whichcompletes case 1.

Case 2: Checking if µ ∈ JA And BK.Figure 4(a) showsthe structure of a sampleAnd expression, where the• sym-bols represent non-Optoperators (i.e.And,Union, orFil-ter), andt stands for triple patterns. There is an arbitrarynumber ofOpt-subexpression (which might, of course, con-tainOpt subexpression themselves). Each of these subex-pressions hasOpt-rank≤ n + 1. Using the same argu-mentation as in case (1), the evaluation problem for all ofthem is inΣPn+2. Moreover, there might be triple leaf nodes;the evaluation problem for such patterns is inPTime, andclearlyP ⊇ ΣPn+2. Figure 4(b) illustrates the situation whenall Opt-expressions and triple patterns have been replacedby the complexity of theirEvaluation problem.

We then proceed as follows. We apply Lemma 9 re-peatedly, folding the remainingAnd, Union, andFiltersubexpressions bottom up. The lemma guarantees that thesefolding operations do not increase the complexity class, andit is easy to prove thatEvaluation remains inΣPn+2 for

18

And

Opt

. . . . . .

•

•

Opt

. . . . . .

t

•

Opt

. . . . . .

And

Evaluation in ΣPn+2 •

•

Evaluation in ΣPn+2 Evaluation in PTime

•

Evaluation in ΣPn+2

Figure 4: (a) AND-expression with increased OPT-rank; (b) The OPT-expressions and leaf nodeshave been replaced by the complexity class of their EVALUATION problem.

the whole expression.

Case 3: Checking if µ ∈ JA Union BK and Case 4:Checking if µ ∈ JA Filter RK. Similar to case 2.

It is worth mentioning that the structural induction is poly-nomially bounded by the size of the expression when thenesting depth ofOpt-operators is fixed (which holds by as-sumption), i.e. comprises only polynomially many steps. Inall cases except case 1 the recursive calls concern subexpres-sions of the original expression. In case 1,µ ∈ JAOpt BKgenerates two checks, namelyµ ∈ JA And BK andµ ∈JAK \ JBK. These two checks might trigger recursive checksagain. But since the nesting depth ofOpt-expressions isrestricted, it is easy to see that there is a polynomial thatbounds the number of recursive call.

A.4 Queries: Fragments Including SELECT

Proof of Theorem 5

LetF be a fragment for which theEvaluation problem isC-complete, whereC is a complexity class s.t.NP ⊆ C. Weshow that, for a queryQ ∈ F+, documentD, and mappingµ,testing ifµ ∈ JQKD is contained inC (C-hardness followstrivially from C-hardness of fragmentF ). By definition,each query inF+ is of the formQ = SelectS(Q

′), whereS ⊂ V is a finite set of variables andQ′ is anF -expression.By definition of operatorSelect, µ ∈ JQKD holds if andonly if there exists a mappingµ′ ∈ JQ′KD s.t. µ′ = µ ∪ µ′′,for any mappingµ′′ ∼ µ. We observe that the domain ofcandidate mappingsµ′′ is bounded by the set of variablesin Q′, i.e. dom(µ′′) ⊆ vars(Q′). Hence, we can guess amappingµ′′ (this is possible since we are at least inNP) andsubsequently check ifµ′ = µ ∪ µ′′ ∈ JQ′KD, which is alsopossible inC. The whole algorithm is inC.

Proof of Theorem 6

First, we show thatEvaluation for A+-queries is con-tained inNP. By definition, each query inA+ is of the formQ = SelectS(Q

′), whereS ⊂ V is a finite set of variablesandQ′ is anAnd-only expression. Further letD a docu-ment andµ a mapping. To prove membership, we follow

the approach taken in the previous proof (of Theorem 5) andeliminate theSelect-clause. More precisely, we guess amappingµ′′ ∼ µ and check ifµ′′ ∪ µ ∈ JQ′KD (we refer thereader to the proof of Theorem 5 before for more details).Again, the size of the mapping to be guessed is bounded, andit is easy to see that the resulting algorithm is inNP.

To prove hardness we reduce3Sat, a prototypicalNP-complete problem, to our problem. The proof was inspiredby the reduction of3Sat to the evaluation problem for con-junctive queries in [11]. The3Sat problem is defined asfollows. Letψ a boolean formula

ψ = C1 ∧ · · · ∧ Cn

in CNF, where each clauseCi is of the form

Ci = li1 ∨ li2 ∨ li3,

i.e. contains exactly three, possibly negated, literals: isψsatisfiable? For our encoding we use the fixed database

D := (1, 1, 1), (1, 1, 0), (1, 0, 1), (1, 0, 0), (0, 1, 1),(0, 1, 0), (0, 0, 1), (0, 0, 0), (0, c, 1), (1, c, 0),

where we assume that0, 1 ∈ I are any IRIs. Further letV = x1, . . . xm denote the set of variables occurring informulaψ. We set up the SPARQL core expression

P ′ := (L∗11, L

∗12, L

∗13) And . . . And (L∗

n1, L∗n2, L

∗n3)

And (?X1, c, ?X1) And . . . And (?Xm, c, ?Xm)And (0, c, ?A), where

L∗ij :=?Xk if lij = xk, andL∗

ij :=?Xk if lij = ¬xk.

Finally, setP :=Select?A(P′). It is straightforward to

verify thatµ = ?A 7→ 1 ∈ JP KD iff ψ is satisfiable.

B. Algebraic Results

B.1 Proofs of the Equivalences in Figure 1(I-IV)

19

I. Idempotence and Inverse

The two equivalences(UIdem) and (Inv) follow directlyfrom the definition of operators∪ and\, respectively.

(JIdem). LetA− be anA−-expression. We show that bothdirections of the equivalence hold.⇒: Consider a mappingµ ∈ A−

A−. Thenµ = µ1 ∪ µ2 whereµ1, µ2 ∈ A−

and µ1 ∼ µ2. Each pair of distinct mappings inA− isincompatible, soµ1 = µ2 and, consequently,µ1 ∪ µ2 =µ1. By assumption,µ1 ∈ A− holds and we are done.⇐:Consider a mappingµ ∈ A−. Chooseµ for both the left andright expression inA−

A−. By assumption,µ = µ ∪ µ iscontained in the left side expression of the equation.

(LIdem). LetA− be anA− expression. Then

A−1 A−= (A−

A−) ∪ (A− \A−) [semantics]= (A−

A−) ∪ ∅ [(Inv)]= A−

A−

= A− [(JIdem)],

which proves the equivalence.

II. Associativity

(UAss) and(JAss) are trivial (cf. [26]).

III. Commutativity

(UComm) and(JComm) are trivial (cf. [26]).

IV. Distributivity

(JUDistR). We show that both directions of the equivalencehold. ⇒: First assume thatµ ∈ (A1 ∪ A2) A3. Then(according to the definition of) µ is of the formµ12 ∪ µ3

whereµ12 ∈ A1 ∪ A2, µ3 ∈ A3, andµ12 ∼ µ3. Moreprecisely,µ12 ∈ A1 ∪ A2 meansµ12 in A1 or in A2, so wedistinguish two cases. If (a)µ12 ∈ A1 then the subexpressionA1 A3 on the right side generatesµ = µ12 ∪ µ3 (chooseµ12 from A1 andµ3 from A3); similarly, if (b) µ12 ∈ A2,then the expressionA2 A3 on the right side generatesµ.⇐: Consider a mappingµ ∈ (A1 A3) ∪ (A2 A3).Thenµ is of the form (a)µ = µ1 ∪ µ3 or of the form (b)µ = µ2 ∪ µ3 with µ1 ∈ A1 µ2 ∈ A2, µ3 ∈ A3 (where(a) µ1 ∼ µ3 or (b) µ2 ∼ µ3 holds, respectively). Case (a):µ1 is contained inA1, so it is also contained inA1 ∪ A2.Hence, on the left-hand side we chooseµ1 from A1 ∪ A2

andµ3 from A3. By assumption they are compatible andgenerateµ = µ1 ∪ µ3. Case (b) is symmetrical.

(JUDistL). The equivalence follows from(JComm) and(JUDistR) (cf. [26]).

(MUDistR). We show that both directions of the equationhold.⇒: Consider a mappingµ ∈ (A1 ∪ A2) \A3. Hence,µ is contained inA1 or in A2 and there is no compatible

mapping inA3. If µ ∈ A1 then the right side subexpressionA1 \ A3 generatesµ, in the other caseA2 \ A3 generatesdoes.⇐: Consider a mappingµ in (A1 \A3) ∪ (A2 \A3).Thenµ ∈ (A1 \ A3) or µ ∈ (A2 ∪ A3). In the first case,µ is contained inA1 and there is no compatible mappingin A3. Clearly,µ is then also contained inA1 ∪ A2 and(A1 ∪ A2) \A3. The second case is symmetrical.

(LUDistR). The following rewriting proves the equivalence.

(A1 ∪A2)1 A3

= ((A1 ∪A2) A3) ∪ ((A1 ∪ A2) \A3)(1)= ((A1 A3) ∪ (A2 A3)) ∪ ((A1 \A3) ∪ (A2 \A3))(2)= ((A1 A3) ∪ (A1 \A3)) ∪ ((A2 A3) ∪ (A2 \A3))= (A1 1 A3) ∪ (A2 1 A3)

Step (1) is an application of(JUDistR) and(MUDistR);in step (2) we applied(UAss) and(UComm).

B.2 Proofs of the Equivalences in Figure 1(V-VI)

In the paper satisfaction was defined informally; to beself-contained, we repeat the formal definition from [26].

Definition 16. Given a mapping µ, filter conditions R,R1, R2, variables ?x, ?y, and constant c, we say that µsatisfies R (denoted as µ |= R), if

1. R is bnd(?x ) and ?x ∈ dom(µ),

2. R is ?x = c, ?x ∈ dom(µ), and µ(?x) = c,

3. R is ?x =?y, ?x, ?y ⊆ dom(µ), and µ(?x)=µ(?y),

4. R is ¬R1 and it is not the case that µ |= R1,

5. R is R1 ∨R2 and µ |= R1 or µ |= R2,

6. R is R1 ∧R2 and µ |= R1 and µ |= R2.

Recall that, given a set of mappingsΩ and filter conditionR, σR(Ω) is defined as the subset of mappings inΩ thatsatisfy conditionR, i.e.σR(Ω) = µ ∈ Ω | µ |= R.

The following proposition states that functionsafeVars(A)returns only variables that are bound in each mapping whenevaluatingA on any documentD. It will be required in someof the subsequent proofs.

Proposition 5. Let A be a SPARQL Algebra expressionand let ΩA denote the mapping set obtained from evalu-ating A on any document. Then ?x ∈ safeVars(A) =⇒∀µ ∈ ΩA :?x ∈ dom(µ).

Proof. The proof is by induction on the structure ofA andapplication of Definition 5. We omit the details.

Proofs of Equivalences in Figure 1(V-VI)

(SUPush). Follows directly from Proposition 1(5) in [26].

(SDecompI). Follows directly from Lemma 1(1) in [26].

20

(SDecompII). Follows directly from Lemma 1(2) in [26].

(SReord). Follows from the application of(SDecompI) andthe commutativity of the boolean operator∧.

(BndI), (BndII), (BndIII), and(BndIV) are trivial.

(BndV). Recall that by assumption?x 6∈ vars(A1). Thefollowing rewriting proves the equivalence.

σbnd(?x)(A1 1 A2)= σbnd(?x)(A1 A2) ∪ (A1 \A2)) [semantics]= σbnd(?x)(A1 A2) ∪ σbnd(?x)(A1 \A2) [(SUPush)]= σbnd(?x)(A1 A2) [∗1]= A1 A2 [∗2]

∗1 follows immediately from assumption?x 6∈ vars(A1);∗2 follows from the observation that?x ∈ safeVars(A2).

(SJPush). ⇒: Let µ ∈ σR(A1 A2). By semantics,µ |= R. Furthermore,µ is of the formµ1 ∪ µ2, whereµ1 ∈ A1, µ2 ∈ A2, andµ1 ∼ µ2. Recall that by as-sumptionvars(R) ⊆ safeVars(A1), so we always have thatdom(µ1) ⊆ vars(R) (cf. Proposition 5), i.e. each variablethat occurs inR is bound in mappingµ1. It is easy to verifythatR |= µ impliesR |= µ1, since both mappings coincidein the variables that are relevant for evaluatingR. Conse-quentlyσR(A1) on the right side generatesµ1, and clearlyσR(A1) A2 generatesµ1 ∪ µ2 = µ. ⇐: Consider a map-pingµ ∈ σR(A1) A2. Thenµ is of the formµ = µ1 ∪µ2,µ1 ∈ A1, µ2 ∈ A2, µ1 ∼ µ2, andµ1 |= R. It is easy to seethat then alsoµ1∪µ2 |= R, becausedom(µ1) ⊆ vars(R) (asargued in case⇒) andµ1 ∪µ2 coincides withµ1 on all vari-ables that are relevant for evaluatingR. Hence,µ = µ1 ∪µ2

is generated by the left side of the equation.

(SMPush). ⇒: Let µ ∈ σR(A1 \ A2). By semantics,µ ∈ A1 and there is noµ2 ∈ A2 compatible withµ1 andµ |= R. From these preconditions it follows immediatelythatµ ∈ σR(A1) \ A2. ⇐: Let µ ∈ σR(A1) \ A2. Thenµ ∈ A1, µ |= R, and there is no compatible mapping inA2.Clearly, then alsoµ ∈ A1 \A2 andµ ∈ σR(A1 \A2).

(SLPush). The following rewriting proves the equivalence.

σR(A1 1 A2)= σR((A1 A2) ∪ (A1 \A2)) [semantics]= σR(A1 A2) ∪ σR(A1 \A2) [(SUPush)]= (σR(A1) A2) ∪ (σR(A1) \A2) [*]= σR(A1)1 A2 [semantics]

* denotes application of(SJPush) and(SMPush).

B.3 Proofs of the Remaining Technical Results

Proof of Proposition 1

Proof. Let A− be anA− expression. The proof is byinduction on the structure ofA− The basic case isA− =

JtK. By semantics, all mappings inA− bind exactly thesame set of variables, and consequently the values of eachtwo distinct mappings must differ in at least one variable,which makes them incompatible.10 (Case 1) We assumethat the hypothesis holds and consider an expressionA− =A−

1 A−2 . Then each mappingµ ∈ A− is of the form

µ = µ1 ∪ µ2 with µ1 ∈ A−1 , µ2 ∈ A−

2 , andµ1 ∼ µ2.We fix µ and show that each mappingµ′ ∈ A− differentfrom µ is incompatible. Any mapping inµ′ ∈ A− that isdifferent fromµ is of the formµ′

1 ∪ µ′2 with µ′

1 ∈ A−1 ,

µ′2 ∈ A−

2 andµ′1 different fromµ1 or µ′

2 different fromµ2. Let us w.l.o.g. assume thatµ′

1 is different fromµ1.By induction hypothesis,µ1 is incompatible withµ′

1. It iseasy to verify that thenµ = µ1 ∪ µ2 is incompatible withµ′ = µ′

1 ∪ µ′2, sinceµ1 and µ′

1 disagree in the value ofat least one variable. (Case 2) LetA− = A−

1 \ A−2 . By

induction hypothesis, each two mappings inA−1 are pairwise

incompatible. By semantics,A− is a subset ofA−1 , so the

incompatibility property still holds forA−. (Case 3) LetA− = A−

1 1 A−2 . We rewrite the left outer join according

to its semantics:A− = A−1 1 A−

2 = (A−1 A−

2 ) ∪ (A−1 \

A−2 ). As argued in cases (1) and (2), the incompatibility

property holds for both subexpressionsA = A−1 A−

2

andA\ = A−1 \A

−2 , so it suffices to show that the mappings

in A are pairwise incompatible to those inA\. We observethatA\ is a subset ofA−

1 . Further, each mappingµ ∈ A

is of the formµ = µ1 ∪ µ2, whereµ1 ∈ A−1 , µ2 ∈ A

−2 , and

µ1 ∼ µ2. By assumption, each mapping inA−1 , and hence

each mappingµ′1 ∈ A

\ is either identical to or incompatiblewith µ1. (a) Ifµ1 6= µ′

1, thenµ′1 is incompatible withµ1, and

consequently incompatible withµ1∪µ2 = µ, so we are done.(b) Let µ1 = µ′

1. We observe that, by assumption, there isa compatible mapping (namelyµ2 in A−

2 ). This means thatA−

1 \ A−2 does not generateµ′

1, so we have a contradiction(i.e., the assumptionµ1 = µ′

1 was invalid). (Case 4) LetA = σC(A

−1 ). Analogously to case 2,A− is a subset ofA−

1 ,for which the property holds by induction hypothesis.

Proof of Lemma 1

We provide an exhaustive set of counterexamples.

Proof of Claim 1. In this part, we give counterexamplesfor two fragments (a)A,\, 1 ,σ,∪ and (b)A,\, 1 ,σ,π.The result for the full algebra (i.e.,A,\, 1 ,σ,∪,π) follows.

(1a) FragmentA,\, 1 ,σ,∪. We use the fixed databaseD = (0, c, 1). Consider the algebra expressionA =J(?x, c, 1)KD ∪ J(0, c, ?y)KD. It is easy to see that bothA A = ?x 7→ 0, ?y 7→ 1, ?x 7→ 0, ?y 7→ 1andA1 A = ?x 7→ 0, ?y 7→ 1, ?x 7→ 0, ?y 7→ 1differ fromA = ?x 7→ 0, ?y 7→ 1, which shows thatneither(JIdem) nor (LIdem) holds for this fragment.

(1b) FragmentA,\, 1 ,σ,π. We use the fixed databaseD = (0, f, 0), (1, t, 1), (a, tv, 0), (a, tv, 1). Consider thealgebra expression

10Recall that we assume set semantics.

21

A = π?x,?y((t1 1 t2)1 t3), where

t1def:= J(a, tv, ?z)KD,

t2def:= J(?z, f, ?x)KD, and

t3def:= J(?z, t, ?y)KD.

It is easy to verify thatA = ?x 7→ 0, ?y 7→ 1.ForA A andA1 A we then get exactly the same resultsas in part (1a), and we conclude that neither(JIdem) nor(LIdem) holds for the fragment under consideration.

Proof of Claim 2. Trivial.

Proof of Claims 3 + 4. We provide counterexamples foreach possible operator constellations. All counterexamplesare designed for the databaseD = (0, c, 1).

Distributivity over ∪ (Claim 3 in Lemma 1):

• A1 \ (A2 ∪ A3) ≡ (A1 \ A2) ∪ (A1 \ A3) does nothold, e.g.A1 = J(0, c, ?a)KD,A2 = J(?a, c, 1)KD, andA3 = J(0, c, ?b)KD violates the equation.

• A1 1 (A2 ∪ A3) ≡ (A1 1 A2) ∪ (A1 1 A3) doesnot hold, e.g.A1 = J(0, c, ?a)KD,A2 = J(?a, c, 1)KD,andA3 = J(0, c, ?b)KD. violates the equation.

Distributivity over (Claim 4 in Lemma 1):

• A1 ∪ (A2 A3) ≡ (A1 ∪A2) (A1 ∪A3) does nothold, e.g.A1 = J(?a, c, 1)KD,A2 = J(?b, c, 1)KD, andA3J(0, c, ?b)KD violates the equation.

• (A1 A2)∪A3 ≡ (A1 ∪A3) (A2 ∪A3) does nothold (symmetrical to the previous one).

• A1 \ (A2 A3) ≡ (A1 \A2) (A1 \A3) does nothold, e.g.A1 = J(?a, c, 1)KD,A2 = J(?b, c, 1)KD, andA3J(0, c, ?b)KD violates the equation.

• (A1 A2) \A3 ≡ (A1 \A3) (A2 \A3) does nothold, e.g.A1 = J(0, c, ?a)KD,A2 = J(0, c, ?b)KD, andA3J(?a, c, 1)KD violates the equation.

• A1 1 (A2 A3) ≡ (A1 1 A2) (A1 1 A3) doesnot hold, e.g.A1 = J(?a, c, 1)KD,A2 = J(?b, c, 1)KD,andA3J(0, c, ?a)KD violates the equation.

• (A1 A2)1 A3 ≡ (A1 1 A3) (A2 1 A3) doesnot hold, e.g.A1 = J(0, c, ?a)KD,A2 = J(0, c, ?b)KD,andA3J(?a, c, 1)KD violates the equation.

Distributivity over \ (Claim 4 in Lemma 1):

• A1 ∪ (A2 \ A3) ≡ (A1 ∪ A2) \ (A1 ∪ A3) does nothold, e.g.A1 = J(?a, c, 1)KD,A2 = J(0, c, ?a)KD, andA3J(?a, c, 1)KD violates the equation.

• (A1 \ A2) ∪ A3 ≡ (A1 ∪ A3) \ (A2 ∪ A3) does nothold (symmetrical to the previous one).

• A1 (A2 \A3) ≡ (A1 A2) \ (A1 A3) does nothold, e.g.A1 = J(?a, c, 1)KD,A2 = J(?b, c, 1)KD, andA3J(0, c, ?a)KD violates the equation.

• (A1 \A2) A3 ≡ (A1 A3) \ (A2 A3) does nothold (symmetrical to the previous one).

• A1 1 (A2\A3) ≡ (A1 1 A2)\(A1 1 A3)does nothold, e.g.A1 = J(?a, c, 1)KD, A2 = J(?b, c, 1)KD, andA3J(?b, c, 1)KD violates the equation.

• (A1\A2)1 A3 ≡ (A1 1 A3)\(A2 1 A3)does nothold, e.g.A1 = J(?a, c, 1)KD, A2 = J(?b, c, 1)KD, andA3J(0, c, ?b)KD violates the equation.

Distributivity over 1 (Claim 4 in Lemma 1):

• A1∪(A2 1 A3) ≡ (A1∪A2)1 (A1∪A3)does nothold, e.g.A1 = J(?a, c, 1)KD, A2 = J(c, c, c)KD, andA3J(?b, c, 1)KD violates the equation.

• (A1 1 A2)∪A3 ≡ (A1∪A3)1 (A2∪A3)does nothold (symmetrical to the previous one).

• A1 (A2 1 A3) ≡ (A1 A2)1 (A1 A3) doesnot hold, e.g.A1 = J(?a, c, 1)KD,A2 = J(?b, c, 1)KD,andA3J(0, c, ?a)KD violates the equation.

• (A1 1 A2) A3 ≡ (A1 A3)1 (A2 A3) doesnot hold (symmetrical to the previous one).

• A1 \ (A2 1 A3) ≡ (A1 \A2)1 (A1 \A3) does nothold, e.g.A1 = J(?a, c, 1)KD, A2 = J(?b, c, 1)KD, andA3J(0, c, ?a)KD violates the equation.

• (A1 1 A2) \A3 ≡ (A1 \A3)1 (A2 \A3) does nothold, e.g.A1 = J(?a, c, 1)KD, A2 = J(?b, c, 1)KD, andA3J(0, c, ?b)KD violates the equation.

The list of counterexamples is exhaustive.

Proof of Proposition 2

(MReord). We consider all possible mappingsµ. Clearly,if µ is not contained inA1, it will be neither contained inthe right side nor in the left side of the expressions (both aresubsets ofA1). So we can restrict our discussion to mappingsµ ∈ A1. We distinguish three cases. Case (1): consider amappingµ ∈ A1 and assume there is a compatible mappingin A2. Thenµ is not contained inA1 \ A2, and also not in(A1 \A2)\A3, which by definition is a subset of the former.Now consider the right-hand side of the equation and let usassume thatµ ∈ A1 \ A3 (otherwise we are done). Then,as there is a compatible mapping toµ in A2, the expressionµ ∈ (A1 \ A3) \ A2 will not containµ. Case (2): The caseof µ ∈ A1 being compatible with any mapping fromA3 issymmetrical to (2). Case (3): Letµ ∈ A1 be a mapping thatis not compatible with any mapping inA2 andA3. Thenboth(A1 \A2) \ A3 on the left side and(A1 \A3) \ A2 onthe right side containµ. In all cases,µ is contained in theright side exactly if it is contained in the left side.

(MMUCorr). We show both directions of the equivalence.⇒: Letµ ∈ (A1\A2)\A3. Thenµ ∈ A1 and there is neithera compatible mappingµ2 ∈ A2 nor a compatible mappingµ3 ∈ A3. Then bothA2 andA3 contain only incompatiblemappings, and clearlyA2 ∪ A3 contains only incompatiblemappings. Hence, the right sideA1 \ (A2 ∪ A3) producesµ. ⇐: Let µ ∈ A1 \ (A2 ∪ A3). Thenµ ∈ A1 and there isno compatible mapping inA2 ∪ A2, which means that thereis neither a compatible mapping inA2 nor inA3. It follows

22

thatA1 \ A2 containsµ (as there is no compatible mappinginA2 andµ ∈ A1). From the fact that there is no compatiblemapping inA3, we deduceµ ∈ (A1 \A2) \A3.

(MJ). See Lemma 3(2) in [26].

(LJ). Let A−1 , A−

2 beA−-expressions. The following se-

quence of rewriting steps proves the equivalence.

A−1 1 A−

2

= (A−1 A−

2 ) ∪ (A−1 \A

−2 ) [sem.]

= (A−1 (A−

1 A−2 )) ∪ (A−

1 \ (A−1 A−

2 )) [*]= (A−

1 1 (A−1 A−

2 )) [sem.]

* denotes application of(JIdem), (JAss), and(MJ).

Proof of Lemma 2

Let A−1 , A−

2 beA−-expressions,R a filter condition, and

?x ∈ safeVars(A2 )\vars(A1)a variable that is contained inthe set of safe variables ofA2, but not inA1. We transform theleft side expression into the right side expression as follows.

σ¬bnd(?x)(A−1 1 A−

2 )= σ¬bnd(?x)((A

−1 A−

2 ) ∪ (A−1 \A

−2 )) [semantics]

= σ¬bnd(?x)(A−1 A−

2 ) ∪σ¬bnd(?x)(A

−1 \A

−2 ) [(SUPush)]

= σ¬bnd(?x)(A−1 \A

−2 ) [∗1]

= A−1 \A

−2 [∗2]

We first show that rewriting step∗1 holds. Observe that?x ∈ safeVars(A−

2 ), which implies that (following Proposi-tion 5) variable?x is bound in each mapping generated byA2.Consequently,?x is also bound in each mapping generatedby A1 A2 and the condition¬bnd(?x ) is never satisfiedfor the join part, so it can be eliminated. Concerning step∗2 we observe that?x ∈ safeVars(A2) \ vars(A1) implies?x 6∈ vars(A1). It follows immediately that?x is unboundin any mapping generated byA−

1 \ A−2 , so the surrounding

filter condition always holds and can be dropped.

C. Proofs of the SQO Results

Proof of Lemma 3

• Let Q′ ∈ C−11 (cbΣ(C1(Q))) ∩ A+. ThenC1(Q

′) ∈cbΣ(C1(Q)). This impliesC1(Q

′) ≡Σ C1(Q). It fol-lows thatQ′ ≡Σ Q.

• Follows directly from the definition of the second trans-lation scheme.

• Follows from the last two points.

Proof of Lemma 4

• LetQ′ ≡Σ Q. We have thatC−11 (U(C1(Q

′))),

C−11 (U(C1(Q))) ∈ A+, thereforeC1(Q

′) ≡Σ C1(Q).

Then, it follows thatC1(Q′) ∈ cbΣ(C1(Q)) andQ′ ∈

C−11 (cbΣ(C1(Q))).

• Follows from the last point and bullet two in lemma 3.

Proof of Lemma 5

• We transformQ systematically. LetD be an RDFdatabase such thatD |= Σ.

JQKD= J(Q1 Opt Q2)KD= J(Q1 And Q2)KD ∪ (JQ1KD \ JQ2KD)= J(Q1 And Q2)KD ∪

(πvars(Q1)J(Q1 And Q2)KD \ JQ2KD)

It is easy to verify that each mapping inJQ1 And Q2KDis compatible with at least one mapping inQ2, and thesame still holds for the projectionπvars(Q1)J(Q1 And Q2)KD.Hence, the right side of the union can be dropped and theelimination simplifies toQ ≡Σ (Q1 And Q2).• LetD be an RDF database such thatD |= Σ. Then we

have that

JQKD = J(Q1 Opt (Q2 And Q3))KD= J(Q1 And Q2 And Q3)KD ∪

(JQ1KD \ JQ2 And Q3KD)= J(Q1 And Q3)KD ∪

(JQ1 And Q2KD \ JQ2 And Q3KD).

We now show that(JQ1 And Q2KD\JQ2 And Q3KD) =(JQ1 And Q2KD \ JQ3KD). Assume that there is someµ ∈ (JQ1 And Q2KD \ JQ2 And Q3KD). Then, for allµ′ ∈ JQ2 And Q3KD it holds thatµ′ is incompatibleto µ. Asµ is, by choice, compatible to some element inJQ2KD, it must be in compatible to all elements inJQ3KD.This impliesµ ∈ (JQ1 And Q2KD \ JQ3KD). Assumewe haveν ∈ (JQ1 And Q2KD \ JQ3KD). Chooseν′ ∈JQ2 And Q3KD. It follows that the projection ofν′ inthe variables inQ3 is not compatible toν, thereforeν′ isnot compatible toν. This impliesν ∈ (JQ1 And Q2KD\JQ2 And Q3KD). Consequently

JQKD= πS(J(Q1 And Q3)KD ∪ (JQ1 And Q2KD \ JQ3KD))= πS(J(Q1 And Q3)KD ∪ (JQ1KD \ JQ3KD))= JQ1 Opt Q3KD.

Proof of Lemma 6

• Let µ ∈ JQ1 Opt Q2KD. Then,µ(?x) is defined be-cause ofQ1 ≡Σ Selectvars(Q1)(Q1 And Q2). So,JFilter¬bnd(?x)(Q1 Opt Q2)KD = ∅.• The proof of this claim is straightforward.• Assume that there is someµ ∈ JFilter¬?x=?y(Q2)KD.

So,µ|S ∈ JSelectS(Q2)KD. It holds thatSelectS(Q2)≡Σ SelectS(Q2

?x?y )≡Σ SelectS(Filter?x=?y(Q2)).

It follows that µ ∈ JSelectS(Filter?x=?y(Q2))KD,which is a contradiction.

23

D. Proofs of the Chase Termination Results

D.1 Additional Definitions

Databases. We choose three pairwise disjoint infinitesets∆,∆null andV . We will refer to∆ as the set ofcon-stants, to∆null as the set oflabelled nulls and toV as theset of variables. Adatabase schema R is a finite set of re-lational symbolsR1, ..., Rn. To everyRi ∈ R we assigna natural numberar(Ri) ∈ N, which we call thearity ofRi. The arity ofR, denoted byar(R), is defined asmaxar(Ri) | i ∈ [n] . Throughout the rest of the paper, weassume the database schema, the set of constants and the setof labelled nulls to be fixed. This is why we will suppressthese sets in our notation.

A database instance I is ann-tuple (I1, ..., In), whereIi ⊆ (∆ ∪ ∆null)

ar(Ri) for every i ∈ [n]. We will de-note(c1, ..., car(Ri)) ∈ Ii by thefact Ri(c1, ..., car(Ri)) andtherefore represent the instanceI as the set if its facts. Abus-ing notation, we writeI = Ri(t) | t ∈ Ii, i ∈ [n] .

A position is a position in a predicate, e.g. a three-arypredicateR has three positionsR1, R2, R3. We say that avariable, labelled null or constantc appears e.g. in a positionR1 if there exists a factR(c, ...).

Constraints. Let x, y be tuples of variables. We con-sider two types of database constraints, i.e.tuple-generatingandequality generating dependencies. A tuple-generatingdependency (TGD) is a first-order sentence

ϕ := ∀x(φ(x)→ ∃yψ(x, y)),

such that (a) bothφ andψ are conjunctions of atomic formu-las (possibly with parameters from∆), (b)ψ is not empty, (c)φ is possibly empty, (d) bothφ andψ do not contain equalityatoms and (e) all variables fromx that occur inψ must alsooccur inφ. We denote bybody(ϕ) the set of atoms inφ andby head(ϕ) the set of atoms inψ.

An equality generating dependency (EGD) is a first-ordersentence

ϕ := ∀x(φ(x)→ xi = xj),

wherexi, xj occur inφ andφ is a non-empty conjunction ofequality-freeR-atoms (possibly with parameters from∆).We denote bybody(ϕ) the set of atoms inφ and byhead(ϕ)the setxi = xj.

For brevity, we will often omit the∀-quantifier and therespective list of universally quantified variables.

Constraint satisfaction. Let |= be the standard first-order model relationship andΣ be a set of TGDs and EGDs.We say that a database instanceI = (I1, ..., In) satisfiesΣ,denoted byI |= Σ, if and only if (∆∪∆null, I1, ..., In) |= Σin the sense of anR-structure.

It is folklore that TGDs and EGDs together are expressiveenough to express foreign key constraints, inclusion, func-tional, join, multivalued and embedded dependencies. Thus,we can capture all important semantic constraints used indatabases. Therefore, in the rest of the paper, all sets ofconstraints are a union of TGDs and EGDs only.

Homomorphisms. A homomorphism from a set ofatomsA1 to a set of atomsA2 is a mapping

µ : ∆ ∪∆null ∪ V → ∆ ∪∆null ∪ V

such that the following conditions hold: (a) ifc ∈ ∆, thenµ(c) = c, (b) if c ∈ ∆null, thenµ(c) ∈ ∆ ∪∆null and (c) ifR(c1, ..., cn) ∈ A1, thenR(µ(c1), ..., µ(cn)) ∈ A2.

Chase. Let Σ be a set of TGDs and EGDs andI aninstance, represented as a set of atoms. We say that a TGD∀xϕ ∈ Σ is applicable toI if there is a homomorphismµfrom body(∀xϕ) to I andµ cannot be extended to a homo-morphismµ′ ⊇ µ from head(∀xϕ) to I. In such a case the

chase stepI∀xϕ,µ(x)−→ J is defined as follows. We define a

homomorphismν as follows: (a)ν agrees withµ on all uni-versally quantified variables inϕ, (b) for every existentiallyquantified variabley in ∀xϕ we choose a "fresh" labellednull ny ∈ ∆null and defineν(y) := ny. We setJ to beI ∪ ν(head(∀xϕ)). We say that an EGD∀xϕ ∈ Σ is appli-cable toI if there is a homomorphismµ from body(∀xϕ) to

I andµ(xi) 6= µ(xj). In such a case the chase stepI∀xϕ,a−→ J

is defined as follows. We setJ to be

• I except that all occurrences ofµ(xj) are substituted byµ(xi) =: a, if µ(xj) is a labelled null,• I except that all occurrences ofµ(xi) are substituted byµ(xj) =: a, if µ(xi) is a labelled null,• undefined, if bothµ(xj) andµ(xi) are constants. In this

case we say that the chase fails.

A chase sequence is an exhaustive application of applicableconstraints

I0ϕ0,a0−→ I1

ϕ1,a1−→ . . .,

where we impose no strict order what constraint must beapplied in case several constraints apply. If this sequenceis finite, sayIr being its final element, the chase terminatesand its resultIΣ0 is defined asIr. The length of this chasesequence isr. Note that different orders of application ofapplicable constraints may lead to a different chase result.However, as proven in [28], two different chase orders leadto homomorphically equivalent results, if these exist. There-fore, we writeIΣ for the result of the chase on an instanceI under constraintsΣ. It has been shown in [24, 3, 16] thatIΣ |= Σ. In case that a chase step cannot be performed(e.g., because a homomorphism would have to equate twoconstants) the chase result is undefined. In case of an infinitechase sequence, we also say that the result is undefined.

24

Provisio. We will make a simplifying assumption. LetIbe a database instance andΣ some constraint set. Withoutloss of generality we can assume that whenever two labellednulls, sayy1, y2, are equated by the chase andy1 ∈ dom(I),then all occurrences ofy2 are mapped toy1 in the chase step.This does not affect chase termination as substitutingy1 withy2 would lead to an isomorphic instance.

D.2 Previous Results

In the following we are only interested in constraints forwhich any chase sequence is finite. In [28] weak acyclicitywas introduced, which is the starting point for our work.

Definition 17. (see [28]) Given a set of constraints Σ,its dependency graph dep(Σ) := (V,E) is the directedgraph defined as follows. V is the set of positions thatoccur in the TGDs in Σ. There are two kind of edgesin E. Add them as follows: for every TGD

∀x(φ(x)→ ∃yψ(x, y)) ∈ Σ

and for every x in x that occurs in ψ and every occur-rence of x in φ in position π1

• for every occurrence of x in ψ in position π2, add anedge π1 → π2 (if it does not already exist).

• for every existentially quantified variable y and forevery occurrence of y in a position π2, add a special

edge π1∗→ π2 (if it does not already exist).

A set Σ of TGDs and EGDs is called weakly acyclic iffdep(Σ) has no cycles through a special edge.

Then, in [8] stratification was set on top of the definitionof weak acyclicity. The main idea is that we can test if aconstraint can cause another constraint to fire, which is theintuition of the following definition.

Definition 18. (see [8]) Given two TGDs or EGDs α =∀x1ϕ, β = ∀x2ψ, we define α ≺ β iff there exist databaseinstances I, J and a ∈ dom(I), b ∈ dom(J) such that

• I 2 ϕ(b), possibly b is not in dom(I),

• Iα,a−→ J and

• J 2 ψ(b).

The actual definition of stratification then relies on weakacyclicity.

Definition 19. (see [8]) The chase graph G(Σ) = (Σ, E)of a set of TGDs Σ contains a directed edge (α, β) be-tween two constraints iff α ≺ β. We call Σ stratified iffthe set of constraints in every cycle of G(Σ) are weaklyacyclic.

Theorem 13. (see [8]) If a set of constraints of weaklyacyclic, then it is also stratified. It can be decided bya coNP-algorithm whether a set of constraints is strat-ified.

The crucial property of stratification is that it guaranteesthe termination of the chase in polynomially many chasesteps.

Theorem 14. (see [8]) Let Σ be a fixed and stratifiedset of constraints. Then, there exists a polynomial Q ∈N[X ] such that for any database instance I, the lengthof every chase sequence is bounded by Q(||I||), where||I|| is the number of distinct values in I. Thus, thechase terminates in polynomial time data complexity.

D.3 Proofs of the Technical Results

Proof of Theorem 7

• Follows directly from the definition of the propagationgraph. In the propagation graph stronger conditions haveto be satisfied than in the dependency graph in order toadd special or non-special edges.• Letα := S(X2, X3), R(X1, X2, X3)→ ∃Y R(X2, Y,X1)

andβ := R(X1, X2, X3)→ S(X1, X3). It can be seenthat α ≺ β andβ ≺ α. Together with the fact thatα, β is not weakly acyclic it follows thatα, β is notstratified. However,α, β is safe.• (see [8]) Letγ := T (X1, X2), T (X2, X1) → ∃ Y1, Y2T (X1, Y1), T (Y1, Y2), T (Y2, X1). It was argued in [8]that γ is stratified. However, it is not safe becausebothT 1 andT 2 are affected and therefore dep(γ) =prop(γ) and it was argued in [8] that it is not weaklyacyclic.

Proof of Theorem 8

First we introduce some additional notation. We denote con-straints in the formφ(x1, x2, u) → ∃yψ(x1, x2, y), wherex1, x2, u are all the universally quantified variables and

• u are those variables that do not occur in the head,• every element inx1 occurs in a non-affected position in

the body, and• every element inx2 occurs only in affected positions in

the body.

The proof is inspired by the proof of Theorem 3.8 in [28],especially the notation and some introductory definitions aretaken from there. In a first step we will give the proof forTGDs only, i.e. we do not consider EGDs. Later, we will seewhat changes when we add EGDs again.

Note thatΣ is fixed. Let(V,E) be the propagation graphprop(Σ). For every positionπ ∈ V an incoming path is a,possibly infinite, path ending inπ. We denote byrank(π) themaximum number of special edges over all incoming paths.It holds thatrank(π) < ∞ because prop(Σ) contains nocycles through a special edge. Definer := max rank(π) |π ∈ V andp := |V |. It is easily verified thatr ≤ p, thusr is bounded by a constant. This allows us to partition thepositions into setsN0, ..., Np such thatNi contains exactlythose positionsπ with rank(π) = i. Letn be the number ofvalues inI. We definedom(Σ) as the set of constants inΣ.

25

Choose someα := φ(x1, x2, u) → ∃yψ(x1, x2, y) ∈ Σ.

Let I → . . . → Gα,a1a2b−→ G′ and letc be the newly created

null values in the step fromG toG′. Then

1. newly introduced labelled nulls occur only in affectedpositions,

2. a1 ⊆ dom(I) ∪ dom(Σ) and3. for every labelled nullY ∈ a2 that occurs inπ in φ

and everyc ∈ c that occurs inρ in ψ it holds thatrank(π) < rank(ρ).

This intermediate claim is easily proved by induction onthe length of the chase sequence. Now we show by inductionon i that the number of values that can occur in any positionin Ni in G′ is bounded by some polynomialQi in n thatdepends only oni (and, of course,Σ). As i ≤ r ≤ p, thisimplies the theorem’s statement because the maximal arityar(R) of a relation is fixed. We denote bybody(Σ) the num-ber of characters of the largest body of all constraints inΣ.

Case 1:i = 0.We claim thatQ0(n):=n+|Σ|·nar(R)·body(Σ)

is sufficient for our needs. We consider a positionπ ∈ N0

and an arbitrary TGD fromΣ such thatπ occurs in the head ofα. For simplicity we assume that it has the syntactic form ofα. In case that there is a universally quantified variable inπ,there can occur at mostn distinct elements inπ. Therefore,we assume that some existentially quantified variable occursin π inψ. Note that asi = 0 it must hold that|x2| = 0. Everyvalue inI can occur inπ. But how many labelled nulls canbe newly created inπ? For every choice ofa1 ⊆ dom(G)such thatG |= φ(a1, λ, b) andG 2 ∃yψ(a1, λ, y) at mostone labelled null can be added toπ by α. Note that in thiscase it holds thata1 ⊆ dom(I) due to (1). So, there are atmostnar(R)·body(Σ) such choices. Over all TGDs at most|Σ| · nar(R)·body(Σ) labelled nulls are created inπ.

Case 2:i→ i + 1.We claim thatQi+1(n) :=∑ij=0Qi(n)+

|Σ| · (∑i

j=0Qi(n))ar(R)·body(Σ) is such a polynomial. Con-

sider the fixed TGDα. Let π ∈ Ni+1. Values inπ maybe either copied from a position inN0 ∪ ... ∪ Ni or maybe a new labelled null. Therefore w.l.o.g. we assumethat some existentially quantified variable occurs inπ inψ. In case a TGD, sayα, is violated inG′ there must exista1, a2 ⊆ domG′(N0, ..., Ni) and b ⊆ dom(G′) such thatG′ |= φ(a1, a2, b), butG′

2 ∃yψ(a1, a2, y). If newly intro-duced labelled null occurs ina2, say in some positionρ, thenρ ∈

⋃ij=0Nj. As there are at most(

∑ij=0Qi(n))

ar(R)·body(Σ)

many such choices fora1, a2, at most(∑i

j=0Qi(n))ar(R)·body(Σ)

many labelled nulls can be newly created inπ.

When we allow EGDs among our constraints, we have thatthe number of values that can occur in any position inNi inG′

can be bounded by the same polynomialQi because equatinglabelled nulls does not increase the number of labelled nullsand the fact that EGDs preserve valid existential conclusionsof TGDs.

Proof of Theorems 9 and 10

Theorem 9 Follows from Theorem 12. Before we proveTheorem 10, we introduce some additional tool.

In general, a set of constraints may have several restrictionsystems. A restriction system isminimal if it is obtainedfrom ((Σ, ∅),(α, ∅) | α ∈ Σ) by a repeated applicationof the constraints from bullets one to three in Definition 13(until all constraints hold) s.t., in case of the first and secondbullet, the image off(β) is extended only by those positionsthat are required to satisfy the condition. Thus, a minimalrestriction system can be computed by a fixedpoint iteration.

Lemma 10. Let Σ be a set of constraints, (G′(Σ), f) arestriction system for Σ and (G′

min(Σ), fmin) its mini-mal one.

• Let P be a set of positions and α, β constraints.Then, the mapping (P, α, β) 7→ α ≺P β? can becomputed by an NP-algorithm.

• The minimal restriction system for Σ is unique. Itcan be computed from Σ in non-deterministic poly-nomial time.

• It holds that Σ is safely restricted if and only if everystrongly connected component in G′

min(Σ) is safe.

Proof. The proof of part one of the lemma proceeds likethe proof of Theorem 3 in [8]. It is enough to considercandidate databases forA of size at most|α|+ |β|, i.e. unionsof homomorphic images of the premises ofα andβ s.t. nullvalues occur only in positions fromP . This concludes partone.

Uniqueness holds by definition. It can be computed viasuccessive application of the constraints (note thatf andEare changed in each step) in definition 13by a Turing machinethat guesses answers to the questionα ≺P β?. As themapping(P, α, β) 7→ α ≺P β? can be computed by an NP-algorithm and the fixedpoint is reached after polynomiallymany applications of the constraints from definition 13, thisimplies the second claim.

Concerning the second claim, observe that every stronglyconnected component inG′

min(Σ) is contained in a singlestrongly connected component of any other restriction sys-tem. This implies the third claim.

Now we turn towards the proof of Theorem 10. By theprevious lemma it suffices to check the conditions from def-inition 14 only for the minimal restriction system. To decidewhetherΣ is not safely restricted, compute the minimal re-striction system, guess a strongly connected component andcheck if it is not safe. Clearly, this can be done in non-deterministic polynomial time.

Proof of Theorem 11 (Sketch)

Before proving this theorem, we need a technical lemma. Itstates the most important property of restriction systems.

Lemma 11. Let α ∈ Σ and (G(Σ), f) a restriction sys-tem for Σ and I be a database instance. If during the

26

chase it occurs that J1α,a→ J2, then the set of positions

in which null values from a that are not in dom(I) occurin the body of α is contained in f(α).

The proof of this lemma is by induction on the length ofthe chase sequence with whichJ1 was obtained fromI and isstraightforward. Note that it uses the simplifying assumptionthat we introduced at the end of appendix D.1.

Let us now turn to the proof of Theorem 11. Let(G′(Σ), f) =((G,E), f) be the minimal restriction system ofΣ. LetC1, ..., Cm be all the pairwise different strongly connectedcomponents of the reflexive closure ofG′(Σ). The graphHis defined as the quotient graph with respect to (α, β) ∈ E| ∃i ∈ [m] : α, β ∈ Ci , i.e. H := G′(Σ)/ (α, β) ∈ E |∃i ∈ [m] : α, β ∈ Ci . H is acyclic and depends only onΣ. We show the claim by induction on the number of nodesn in H .

Case: n = 1: G′(Σ) has a single strongly connectedcomponent. This component is safe by prerequisite. It fol-lows from Theorem 8 that the chase terminates in polynomialtime data complexity in this case.

Case: n 7→ n + 1: Let h be a node inH that has nosuccessors andH− the union of constraints from all othernodes inH . The chase withH− terminates by inductionhypothesis, say that the number of distinct value in this resultis bounded by some polynomialQ−. Chasing the constraintsin h alone terminates, too, say that the number of distinctvalue in this result is bounded by the polynomialq. Thefiring of constraints fromH− can cause some constraintsfrom h to copy null values in their heads. Yet, the firingof constraints inh cannot enforce constraints fromH− tocopy null values to their head (by construction of the minimalrestriction system). IfI is the database instance to be chased,then the number of distinct value in this result is boundedby Q−(||I||) + q(||I|| + Q−(||I||)). As Σ is fixed we canconclude that the chase terminates in polynomial time datacomplexity.

Proof of Theorem 12

• Let Σ be weakly acyclic. Every cycle inG(Σ) is safe,becauseΣ is safe and weak acyclicity implies safety. LetΣ be safe. Every cycle inG(Σ) is safe, becauseΣ is.• Follows from Example 5 and the following proposition.

Proposition 6. Let P ⊆ P ′ ⊆ pos(Σ). If α ≺P β,then α ≺P ′ β. It holds that if α ≺P β, then α ≺ β.

The proof follows from the definition of≺P and≺.• Consider the following TGDs.Σ := α, β, χ, δ.α := R1(x1, x2)→ ∃yS(x1, x2, y),β := R1(x1, x2)→ ∃yT (x1, x2, y),χ := S(x1, x2, x3), T (x4, x5, x6)→ T (x5, x1, x4) andδ := S(x1, x2, x3), T (x4, x5, x3)→ T (x1, x3, x3),R1(x3, x1), R2(x3, x1).It can be seen thatα ≺ χ, β ≺ χ, χ ≺ δ, δ ≺ α andδ ≺ β holds. Thus , there is a cycle in the chase graph thatinvolves all constraints. Unfortunately, the constraint set

is not safe. Therefore, it is also not safely stratified.The minimal restriction system is((Σ, E), f), whereE = ∅ and f = (γ, ∅) | γ ∈ Σ . Obviously, ev-ery cycle in(Σ, E) is safe. Hence,Σ is safely restricted.

Proof of Proposition 3

Let (G′(Σ), f) be a restriction system forΣ such that everystrongly connected component inG′(Σ) is safely stratified.Choose some strongly connected componentC and two con-straintsα, β ∈ C such thatα ≺P β for some set of positionsP . By Proposition 6,α ≺ β holds. AsC is safely stratified,this means thatC must also be safe. So, every cycle inG′(Σ)is also safe.

Proof of Proposition 4

• LetΣ := ∀x1, x2(T (e, x1, x2), T (x2, d, d)→ T (x1, x2, x1)),whered is a constant andxi are variables. Then,Σ′ = ∅.• An example for such a set of constraintsC is constituted

as follows.T (x1, d, x2)→ ∃yT (g, e, y), T (f, d, y)T (x1, e, x2)→ ∃yT (g, e, y), T (f, d, y)T (x1, d, x2)→ T (x2, e, x1)T (x1, e, x2)→ ∃yT (x2, d, y)Note thatd, e, f, g are constants.

27