intelligent information systems - uni-bonn.de · © 2017 prof. dr. r. manthey intelligent...

© 2017 Prof. Dr. R. Manthey Intelligent Information Systems 1

Chapter 5

WS 2016/17

5. Update Propagation over Deductive Databases

Intelligent Information Systems

5


Foundations of Update Propagation

WS 2016/17



5.1

5.1 Foundations of Update Propagation5.2 Updates, Transactions, and Constraints in Datalog5.3 The Magic Updates Method


Update Propagation: What is it All About?

+

+

+

−

−

Base data changes

Inducedchanges ofderived relations

Update propagationis the process of „extra-polating“ changes ofbase relations to those derived relations dependingon them.

The resulting induced changesare not actually performed(except for materialized views)but are purely virtual.

Nevertheless, it may be important to compute them!

5


Update Propagation: What For? (1)

− +

Materializedviews

Integrityconstraints

Determining induced updates is required in order to be able to efficiently („incrementally“) check integrity constraints over viewsand toadapt materialized views.

5


Update Propagation: What For? (2)

− +

Continuousqueries

Another motivation for update propagation can be drawn from the application area ofdatastream monitoring and analysis, which we discussed in the very beginning of thesemester. Here, continuously evaluated queries– expressed as views – are monitored for any induced change that indicates the occurrence of some relevant event.

5


Motivating Example (1)

e_cyclic ← s(X).s(X) ← p(X,X).p(X,Y) ← e(X,Y).p(X,Y) ← e(X,Z), p(Z,Y).

R:F:

e(1,2).e(2,3).e(3,4).

1

2

3

4I : constraint not e_cyclic.

materialized

virtual

k:

p(1,2). p(1,3).p(2,3). p(2,4).p(3,4). p(1,4).

F*:

U1: { + e(4,1) } !

U1:U2: { − e(2,3) } !

U3: { + e(4,1) ; − e(2,3) } !

5




R:F:

e(1,2).e(2,3).e(3,4).

1

2

3

4I : constraintnot e_cyclic.

materialized

virtual

k:

p(1,2). p(1,3).p(2,3). p(2,4).p(3,4). p(1,4).

F*:

U1: { + e(4,1) } !

Intuitive computation of the follow-up changes induced on derived relations:

+ p(4,1)+ p(4,2)+ p(4,3)+ p(4,4)

+ p(3,1)+ p(3,2) + p(3,3)

+ p(2,1)+ p(2,2)

+ p(1,1)

+ s(1)+ s(2)+ s(3)+ s(4)

+ e-cyclic

inconsistent

5




R:F:

e(1,2).e(2,3).e(3,4).

1

2

3


materialized

virtual

k:

p(1,2). p(1,3).p(2,3). p(2,4).p(3,4). p(1,4).

F*:

U1:U2: { − e(2,3) } !

− p(2,3)− p(2,4)

− p(1,3)− p(1,4)

No furtherinduced changes

5

Strictly speaking, integrity checking is completelyuseless here, because an acyclic graph will neverbecome cyclic by canceling an edge!

But the materialized p has to be adapted!




R:F:

e(1,2).e(2,3).e(3,4).

1

2

3


materialized

virtual

k:

p(1,2). p(1,3).p(2,3). p(2,4).p(3,4). p(1,4).

F*:

U3: { + e(4,1) ; − e(2,3) } !

Combining both elementary updates in a single transaction does not lead to acombination of the induced effects, as there are side effectsof the deletion com-

pensating certain effects of the insertion:

+ p(4,1)+ p(4,2)

+ p(3,1)+ p(3,2)

− p(2,3)− p(2,4)

− p(1,3)− p(1,4)

Consistent, too,as there are nofurther inducedchanges.

2

5


Update Propagation: Principle (1)

p(X,Y) ← e(X,Z), p(Z,Y).

Each individual update propagation step obviously is a special form of Forward Reasoning,where – in contrast with our previous T-operators – new changes are derived from facts and previous changes, e.g.:

+ e(4,1) p(1,2). p(1,3).p(2,3). p(2,4).p(3,4). p(1,4).

Y∈{2,3,4}

+ p(4,2)+ p(4,3)+ p(4,4)

1. 2.3.

Three phasesof a single propagation step:

1.

2.

3.

"Matching": U-literal matched with "suitable" body literal

Residual evaluation: Evaluation of the remaining body literals

Residual

Propagation: Transfer of the variable bindings resulting from 1 and 2 tothe rule head and combination of the body literal with thesign of U

UChanges induced

by U

6.2



+ p(4,1)+ p(4,2)+ p(4,3)+ p(4,4)

+ p(3,1)+ p(3,2) + p(3,3)

+ p(2,1)+ p(2,2) + p(1,1)

+ s(1)+ s(3)+ s(3)+ s(4)

+ e-cyclic

If summarizing all steps that are possible in a particular state of propagation into a set,a set-oriented iteration processcan be discovered again, which continues until no furtherderivation is possible anymore:

+ e(4,1)

Update propagation can be viewed as fixpoint iteration, too !

6.2


Update Propagation à la Magic Sets

R RupdateRule Compiler

Fixpoint iteration

F

U !

Update

Induced updatesand intermediateresults

Internal Rules

per change pattern

This observation motivates an approach organizingupdate propagationverysimilarlyto query evaluation based on theMagic Sets approach:

6.2


Updates, Transactions, and Constraints in Datalog

WS 2016/17



5.2

5.1 Foundations of Update Propagation5.2 Updates, Transactions, and Constraints in Datalog5.3 The Magic Updates Method


Schemas and Integrity Constraints in Datalog 5.2

• Integrity checkingis one of the „classical“ motivations for update propagationtechniques. Therefore we have tointroduce a suitable syntaxfor integrity constraintsinto (our version of) Datalog. Again, there isno agreementamong Datalog research-ers about the way how constraints ought to be expressed in Datalog.

• In SQL, there is a heterogeneous situation wrt constraint syntax within thestandard:• There are few special syntactic features for those forms of (single) table cons-

traintsmost commonly used in practice:• PRIMARY KEY, UNIQUE• FOREIGN KEY• NOT NULL• CHECK

• There is the concept ofASSERTIONSenabling schema designers to expressany kind of table constraints as well as all other forms of multi-table constraintsusing the full expressive power of SQL‘s query language. Assertions alonewould be sufficient for expressing all (static) integrity constraints.

• We do so in the context of a general syntax for expressingschemasin Datalog (againjust our „Bonn“ syntax, not shared by most others.


Schemas in Datalog (1)

• As for each database: Declarationof every relation is required, before the DBcan be opened and facts be inserted (or rules be defined).

• In Datalog: There is no agreed notationor convention for schemas; most authorssimply don‘t mention a schema explicitly – we have our own notation.

• Datalog is an untyped language(like Prolog), i.e., there are no attributes and no explicit value domains. However, we will use types implicitly throughout.

• In a schema: Only the arity of a relation and its nameare declared.

• The "form of definition"of the relation, however, is to be fixed in the schema, too:• Relation defined by facts only: stored• Relation defined by rules only: derived

• Both variants can be combined, i.e., there are three relation types:• Base relation: stored (but not derived)• virtual relation: derived (but not stored)• materialized relation: both, stored and derived

Database Schema

engl. "to store": speichernengl. "to derive": ableiten

5.2



m ← p(X, X).

p(X, Y) ← q(X, Y), r(Y, X).

q(X, Y) ← s(X, Y, Z).r(Y, X) ← s(X, Y), not t(Y).

Example of a simple Datalog schema:

derived m/0.derived p/2.

stored p/2.derived q/2.derived r/2.

stored s/3.stored s/2.stored t/1.

Relation declarations

Derivation rules

Materialized

Rules must „fit“ with relation declarations!

Virtual

Baserelations

Derivedrelations

(Of course, rules – like view definitions in SQL –are part of the schema, too!)

Same name with different arity is admissible in Datalog!

5.2



m ← p(X, X).

p(X, Y) ← q(X, Y), r(Y, X).

q(X, Y) ← s(X, Y, Z).r(Y, X) ← s(X, Y), not t(Y).

Surprisingly, there is no agreed notation for integrity constraintsin Datalog either – thus,we introduce our own style for expressing normative rules, too:

derived m/0.derived p/2.

stored p/2.derived q/2.derived r/2.

stored s/3.stored s/2.stored t/1.

constraint t(c).

constraintnot m.constraintnot q(a, b).

Integrity constraints

Positive condition: „Must always be true!"

Negative condition:„May never be true!"

Constraints may apply to both,stored and derived relations.

5.2


Integrity Constraints in the Genealogy DB (1)

In our sample database representing a family tree a lot of „nonsense“ can be accomodatedwithout integrity constraints, e.g.:

born_in('John', 1978).born_in('Mary', 1933).born_in('Mary', 1968).

died_in('Jim', 1999).died_in('John', 1945).died_in('Jim', 2000).

sex_of('Jim', n).sex_of('male', m).

child_of('William', 'Diana', 'Charles').child_of('John', 'Caroline', 'William').child_of('Charles', 'Christine', 'John').

married_to('Charles', 'Diana', 1981).married_to('Charles', 'Anne', 2004).

divorced_from('Diana', 'Charles', 1996).divorced_from('Elizabeth', 'Charles', 2002).

title_of('Charles', 'King of Great Britain').title_of('William', 'King of Great Britain').

Each of these facts is somehow "inadmissible"! Why? And prevent them in Datalog?

5.2


Integrity Constraints in the Genealogy DB (2)

ICs for the genealogy DB: (formulated in natural language first)

IC 1 Each person who has died must have been born before. IC 2 There is just one person per name.IC 3 Each person‘s sex is (either) male or female.IC 4 If two people divorce they have to be married at the time of divorce.IC 5 In the 'married_to'-relation, the 1st column always contains the name of

the wife, whereas the husband is recorded in the 2nd column. IC 6 (analogously for 'divorced_from')IC 7 (analogously for 'child_of')IC 8 The 1st column in the 'sex_of'-relation contains the name of a person.IC 9 Nobody must be a descendant of himself or of one of his own descendants

(i.e., the 'descendant_of'-relation is acyclic).IC 10 Nobody must be married to a relative of degree 1 or 2.IC 11 Each title must be held by at most one living person a time.IC 12 Nobody may get married before being born.

5.2


Integrity Constraints as Universal Axioms

IC 1 Each person who has died must have been born before.

• Most integrity constraints are universal statements (like IC1):"For all" objects of a particular type a certain condition holds!

• "For all" is circumscribed in natural language most of the time (even negatively): "each", "only", "all", "always", "nobody"

(or is not made explicit at all, as in IC 4 and IC 8)• In (full) predicate logic, such conditions are expressed using a universal quantifier,

in most cases coupled with implication for restricting the scope of the statement:

• Like SQL, Datalog does not offer any universal quantifier, but only the (implicit)existential quantifier, so that a transformation using Boolean algebra is required:

∀ X, Y1: died_in(X,Y1) ⇒ ∃ Y2 (born_in(X,Y2) ∧ Y2 ≤ Y1)

¬ ∃ X, Y1: (died_in(X,Y1) ∧ ¬ ∃ Y2 (born_in(X,Y2) ∧ Y2 ≤ Y1 ))

5.2


Expressing ICs in Datalog (1)

¬ ∃ X, Y1: (died_in(X,Y1) ∧ ¬ ∃ Y2 ( born_in(X,Y2) ∧ Y2 ≤ Y1 ))

• Universal conditions are thus turned into negated existential conditions.

• Existential conditions in Datalog are expressible via parameterless relations(arity 0)defined (once again – a very important concept!) by auxiliary rules:

p1 ← died_in(X,Y1), not ( . . . . )

• As nesting is banned from Datalog, the conjunction has to be moved "behind" the negation operator and pushed into another auxiliary rule:

p1 ← died_in(X,Y1), not p2(X, Y1)p2(X, Y1) ← born_in(X, Y2), Y2 =< Y1

∃ Y2

∃ X, Y1

5.2


Expressing ICs in Datalog (2)

¬ ∃ X, Y1: ( died_in(X,Y1) ∧ ¬ ∃ Y2 ( born_in(X,Y2) ∧ Y2 ≤ Y1 ))

p1 ← died_in(X,Y1), not p2(X, Y1)p2(X, Y1) ← born_in(X, Y2), Y2 =< Y1

Thus, the positive partof the existential condition has been properly formulated in Datalog:

What is missing is just the integrity constraint itself (including the negation), which we will identify from now on by the keyword constraint:

constraint notp1

5.2


Integrity Constraints vs. Test Queries

• Important: It is not sufficient to introduce a parameterless relation defined by aderivation rule for expressing a constraint!

• Reaon: Parameterless relations may be queried (as tests), but are notautomaticallycombined with the request to the DBMS to automatically enforce this test!

derived consistent/0.

consistent ← p(X), not q(X).

constraint consistentwith consistent ← p(X), not q(X).

derived consistent/0.

constraint consistent.

consistent ← p(X), not q(X).

1)

2)

3)

1) and 2): Test query "consistent ?"can be posed

3): Relation "consistent/0" is not queriable(as local to the integrity constraint)

1): Answer may be true or false.2): Answer is always true.

5.2


Datalog Schemas: Summary

• A Datalog schemaconsists of the following parts:

• Relation declarationsof the form [derived | stored ] <Name> / <Arity>• Relation names are strings beginning with a lower case character.• Arities are positive integers or zero.• Per couple name/arity occurring in the DB there must be at least one

declaration; double declarations (derived and stored) are possible.

• Derivation rulesof the form <Head> ← <Body>.• The rule head is a positive literal for a relation declared as derived.• The rule head is a conjunction of literals (or just a single literal).• Each derived relation is defined by at least one rule.

• Integrity contraintsof the form constraint <Literal>.• The literal must be variable-free (a ground literal, positive or negative).

• In addition, various "wellformedness conditions" have to be respected.

5.2


Updates and Transactions in Datalog 5.2

• The most important syntax extension in this chapter, of course, is concerned withchanges of database states.

• Again, thereis no agreed common style of expressing change related commandsin Datalog (as compared to the INSERT, DELETE, UPDATE style in SQL).

• We will again introduce „our own“ syntactic version of change commands (comingwith a related, ratherintuitive semantics, close to previous decisions about semantics of rules and queries).

• Moreover, a transactionconcept (resembling that of SQL) is introduced, as a commonsyntactic basis of update propagation methods.

• Even though semantical issues are again discussed ratherformally, the formal treat-ment doesn‘t matter for the exam (this semester), but the basic concepts haveto beunderstood as far as syntax and intuitive semantics is concerned.


Updates in Datalog

• For updates(DB changes, i.e., insertions, deletions and modifications of facts) in Datalog there is no generally accepted syntax and semantics in the literature either.Again we need a notation of our own, which seemlessly fits with the rest of thelanguage. (Note that we don‘t consider schema changes in this lecture.)

• We will introduceinsertions(+) und deletions(−) only:Modificationsof individual attribute values can be simulated by combining+/−.

• For now: Only updates of base relations are considered !! The topic ofviewupdateswill be informally treated at the end of this chapter briefly only.

• Syntactical building block of all Datalog updates:(Literals with a sign+ / − )

• All update statements are delimited by an (imperative) exclamation mark.

Dynamic Literal

e.g.:+ successor('Elizabeth', 'William')!− successor('Elizabeth', 'Charles')!

5.2


Updates: Syntax

• Elementary updates(insertions/deletions) of individual facts in a relation are expressed by means of dynamic ground literals, e.g.:

• Conditional updates: Several, simultaneously executable elementary updates depending on a condition (syntactically like a rule body)

+ title(X, 'Sir') : member_of(X, 'Beatles')!

Dynamic literal Condition

• Safety requirementfor conditional updates: In analogy to rules and queries

• For conditional updates, local rules are possible, too:

+ p(X) : r(X), not s(X) with s(X) ← t (X,Y) !

+ title('Paul McCartney', 'Sir') !

5.2


Analogy Updates ⇔ Queries ⇔ Rules

• The analogyalready observed for queries and rules carries over to conditional updates:

dynamic literal ⇔ target list ⇔ rule head

condition ⇔ qualification ⇔ rule body

{ (X, Y) : father_of(X, Z), child_of(Y,Z)} ?

grandfather_of (X, Y) ← father_of(X, Z), child_of(Y,Z) .

e.g.:

+ grandfather_of (X, Y) : father_of(X, Z), child_of(Y,Z) !

• For elementary updates/facts/test queries similarly:

+ born_in('Paul', 1943) ! born_in('Paul', 1943) . born_in('Paul', 1943) ?

5.2


Updates: Semantics Intuitively

• Conditional updates are interpreted in a set-orientedmanner (so are retrieval queriesand derivation rules!):

+ p(X) : q(X,Y) ! • All p-insertions are performed simultaneously.

• The condition is completely evaluated before any change takes place.q(1,2).

q(1,3).q(2,3).

+ p(1) !+ p(2) !

• Condition evaluation and DB-update are performed strictly separately in two phases.Thus, an update may not cause any retroactive effectson the evaluation of its own condition:

+ p(X) : q(X,Y), p(Y) !

5.2


Redundant Updates

• Elementary updates which don‘t have any changing effect on the current state areaccepted: Redundant updates

e.g.: p(1).p(3).p(4).

+ p(1) ! (similarly for - p(2) ! )

• This is useful in particular for conditional updates where some of the variablebindings resulting from condition evaluation have an effect, whereas others don‘t:

p(1). q(1).q(2).

p(3).p(4). q(4).

- p(X) : q(X) !e.g.:

5.2

© 2017 Prof. Dr. R. Manthey

• Context: Deductive database D = (R, F), F*: Semantics of D, U: safe conditional update,F: Current state of D before the update (Fold)

(Conditions again without negation at first – extensions will be obvious)

Intelligent Information Systems 31

Semantics of Individual Update Statements (1)

+ A : B1, . . ., Bn !−U ≡

• Assumptions: • Unconditional updates are treated as special case of conditional updates

(condition true).• Local rules contained in R (for simplicity‘s sake, at least internally).

• Evaluation of update statements happens in three stepsmodelized by means of three mappings:

• Complete evaluation of the condition partover the current state resultingin a set of facts δ [U] (F)

• Elimination of all redundant changesresulting in a reducedfact set ∆ [U] (F)

• Simultaneous application ofall remaining true changes to Fold resulting in a new state (set of facts) Fnew =def τ [U] (F)

5.2



• Condition evaluation (in analogy with T-operator for individual rules):

• Elimination of redundant changes: (Insertion of facts into F resp. deletion of facts not in F)

δ [U] (F) =def { A σ | σ consisten substitution and∀ 1 ≤ i ≤ n: Bi σ ∈ F*}

+ A : B1, . . ., Bn !−U ≡

∆ [U] (F) =def

δ [U] (F) − F, if U is an insertion

δ [U] (F) ∩ F, if U is a deletion

5.2



τ [U] (F) =def

F ∪ ∆ [U] (F), if U is an insertion

F − ∆ [U] (F), if U is a deletion

• Simultaneous application of true updates (redundancies eliminated):

• τ formalizes the effect of an individual state transition on the base data:τ for transition

F old = F F new = τ [U] (F)

−+

τ

5.2


Transactions

• Transactionsfor databases are " blocks of changes "either executed entirely, or not at all.

⇒ State transitions between DB states are expressible viatransactions which could not be executed individuallydue to integrity violations.

• In SQL: Transactions aresequences, introduced and concluded by special keywords.

• For Datalog: "Set-oriented" transaction conceptseems to be appropriate, i.e.:

Simultaneousexecution of several changeswithout fixing any ordering of steps.

• Syntax formatfor transactions: { U1; U2 ; . . . . ; Un } !

(Ui elementary or conditional updates without '!')

5.2


Contradictory Updates and Net Effect

• If using conditional updates withín a transaction, it may happen thatcontradictoryelementary updates result from evaluating different parts of the transaction. Here,contradictory means that the same elementary update appears with positive andnegative sign, e.g.:

• Datalog transactions do not fix any ordering of execution, but are set-oriented(in order to remain compatible with other Datalog conventions).Therefore, it is necessary to eliminate elementary updates mutually compensatingeach other. We call this step"net-effect" computation

{ − p(a); + p(X) : q(X)} !

− p(a)+ p(a)+ p(b)+ p(d)

q(a). p(a).q(b). p(c).q(d).

q(a). p(a).q(b). p(b).q(d). p(c).

p(d).

5.2


Net Effect Computation and Duplicates

{ + p(X) : r(X) ; + p(X) : q(X) } !

• It may happen that different parts of a transaction appear to cause the same elementaryupdateto be executed several times:

r(a). q(a).r(b).

q(c).

+ p(a) + p(a)

• This kind of "redundancy" among dynamic literals is to be treated according to thegeneral design principles of Datalog, too. As duplicates are eliminated in queries andduring rule application (due to sets being free of duplicates), we will eliminate alldynamic literals appearing more than once before computing contradictory updatesand net effect.

• Common, recurring principle underlying all these conventions:• No duplicatesin Datalog (all results are sets).• No orderingin Datalog.

5.2


Semantics of Transactions (1)

• Context: D = (R, F) as before

• Safe transaction { U1; U2 ; . . . . ; Un } !T ≡

• Transaction execution is performed in three phasesagain, modelized by means of three mappings generalizing those we already introduced for individual changes:

• Determination of the individual effects of insertions and deletions:∆+ [T] (F) and ∆− [T] (F)

• Mutual influences, i.e., net effect computation: Π + [T] (F) and Π − [T] (F)

• Simultaneous application of all partial changes: τ [T] (F)

• First: Separate the transaction into a positive and a negative partT+ : Set of all insertions in TT− : Set of all deletions in T

5.2



• Separate determination of the individual effects of insertions and deletions:

∆+ [T] (F) =def δ [U] (F)

∆− [T] (F) =def δ [U] (F)

∪

∪

U ∈ T+

U ∈ T−

• Net effect adjustment:

Π + [T] (F) =def ∆+ [T] (F) − ∆− [T] (F)

Π − [T] (F) =def ∆− [T] (F) − ∆+ [T] (F)

5.2



• Transitioncaused by the transaction:

τ [T] (F) =def ( F − Π − [T] (F) ) ∪ Π + [T] (F)

F old = F F new = τ [T] (F)

−+

τ

• Quite analogously:

5.2


Changes and Derived Relations

F old = F F new = τ [T] (F)

−+

τ

+

−

(F old )* (F new )*

τ*

The induced changescaused by a transaction or an individual update statement affectingbase data are computed by determining the fixpoint semantics of the new set of base factsafter the change:

By comparing (F old )* and (F new )*, the differences between both states can be determined.Again, doing so in practice would be a very naive way of computing induced updates!

5.2


Changes and Integrity

• Extensionof the context: D = (R, F,I ) with I being a set of integrity constraintssucht that F* satisfies I (consistent old state)

• I = I+ ∪ I− : I+ Set of all positive integrity constraints of the form 'constraint L'I − Set of all negative integrity constraints of the form 'constraint not L'

• Semantics of a transaction T in presence of integrity constraints:

τI [T] (F) =def

τ[T] (F),

F, else

if ∀ (constraint L) ∈ I+ : L ∈ τ[T] (F)*

and ∀ (constraint not L) ∈ I− : L ∉ τ[T] (F)*

• In natural language: If anyintegrity constraint would be violated in the new state, then no change in the transition associated with T will be performed, else all suchchanges are done.

5.2


Now Back to Foundations of Propagation

WS 2016/17



5.1

5.1 Foundations of Update Propagation5.2 Updates, Transactions, and Constraints in Datalog5.1 Foundations of Update Propagation (cont.)5.3 The Magic Updates Method


Update Propagation: A More Systematic Approach

+

+

+

−

−

5.1

Next, we will try tounderstand in more detail how individualpropagation stepswork,. . .and how the entirepropagation processcan be organized!

From our motivating example we alreadyknow that- individual steps are

similar to derivation steps- the entire process resem-

bles a fixpoint iteration



p(X,Y) ← e(X,Z), p(Z,Y).

Each individual update propagation step obviously is a special form of Forward Reasoning,where – in contrast with our previous T-operators – new changes are derived from facts and previous changes, e.g.:

+ e(4,1) p(1,2). p(1,3).p(2,3). p(2,4).p(3,4). p(1,4).

Y∈{2,3,4}

+ p(4,2)+ p(4,3)+ p(4,4)

1. 2.3.

Three phasesof a single propagation step:

1.

2.

3.

"Matching": U-literal matched with "suitable" body literal

Residual evaluation: Evaluation of the remaining body literals

Residual

Propagation: Transfer of the variable bindings resulting from 1 and 2 tothe rule head and combination of the body literal with thesign of U

UChanges induced

by U

5.1



+ p(4,1)+ p(4,2)+ p(4,3)+ p(4,4)

+ p(3,1)+ p(3,2) + p(3,3)

+ p(2,1)+ p(2,2) + p(1,1)

+ s(1)+ s(3)+ s(3)+ s(4)

+ e-cyclic

If summarizing results of all steps that are possible in a particular state of propagation,a set-oriented iteration processcan be discovered again, which continues until no furtherderivation is possible anymore:

+ e(4,1)

Update propagation can be viewed as a kind of fixpoint iteration, too !

5.1


Update Propagation à la Magic Sets

R RupdateRule Compiler

Fixpoint iteration

F

U !

Update

Induced updatesand intermediateresults

Internal Rules

per change pattern

This observation motivates an approach organizing update propagation very similarlyto query evaluation based on the Magic Sets approach:

5.1


Update Propagation: Rule Compilation

p(X,Y) ← e(X,Z), p(Z,Y).

1. 2.3.

Coding a single propagation stepby means of a Datalog rule:

Matching Residualevaluation

Propagation

p+(X,Y) ← e+(X,Z), p(Z,Y).

1. 2.3.

Matching Residualevaluation

Propagation

Propagation rule

Internal auxiliary relations(à la query_p and answer_p) for temporarilystoring information about base data updates and induced updates:

Delta relations

(or: Delta rule)

( ∆ : Differencebetween old and new state)

5.1


Positive Propagation Rules: Combining Deltas


e_cyclic+ ← s+(X).s+(X) ← p+(X,X).p+(X,Y) ← e+(X,Y).p+(X,Y) ← e+(X,Z), p(Z,Y).p+(X,Y) ← e (X,Z), p+(Z,Y).p+(X,Y) ← e+(X,Z), p+(Z,Y).

R

1

2

3

4

e:

Propagating two changes simultaneously was not needed in our motivating example, but may be indispensable in other situations, e.g.:

5

U4: { + e(4,5) ; + e(5,1) } !

e+(4,5) e+(5,1)

p+(5,1)

p+(4,1)

R∆+

Each body literal mayserve as starting pointfor propagation.

5.1


Positive Propagation Rules and Associated FPI

R

U1: { + e(4,1) } !

p+(4,1)p+(4,2)p+(4,3)p+(4,4)

p+(3,1)p+(3,2) p+(3,3)p+(3,4)s+(4)

p+(2,1)p+(2,2) p+(2,3) p+(2,4)s+(3)e_cyclic+

p+(1,1)p+(1,2) p+(1,3) p+(1,4)s+(2)

s+(1)

e+(4,1)

Fixpoint reached !

Taking note of the change by means of adding a delta fact„triggering“ FPI: Seed fact!

R∆+

5.1


e_cyclic+ ← s+(X).s+(X) ← p+(X,X).p+(X,Y) ← e+(X,Y).p+(X,Y) ← e+(X,Z), p(Z,Y).p+(X,Y) ← e (X,Z), p+(Z,Y).p+(X,Y) ← e+(X,Z), p+(Z,Y).


Positive Propagation Rules and FPI: Erroneous Derivations!

U1: { + e(4,1) } !

p+(4,1)p+(4,2)p+(4,3)p+(4,4)

p+(3,1)p+(3,2) p+(3,3)p+(3,4)s+(4)

p+(2,1)p+(2,2) p+(2,3) p+(2,4)s+(3)e_cyclic+

p+(1,1)p+(1,2) p+(1,3) p+(1,4)s+(2)

s+(1)e+(4,1)

+ p(4,1)+ p(4,2)+ p(4,3)+ p(4,4)

+ p(3,1)+ p(3,2) + p(3,3)

+ p(2,1)+ p(2,2) + p(1,1)

+ s(1)+ s(2)+ s(3)+ s(4)

+ e_cyclic+ e(4,1)

Comparing the "intuitively" determined difference with the resultof fixpoint iteration over the delta rules exhibits a discrepancy:Some "old" paths are indicated as presumably new!

5.1


Analyzing the Mistake

p+(3,4) ← e (3,4), p+(4,4).p(3,4) ← e (3,4).

• Some of the "wrong" delta facts have been generated due to already derivable p-factsbeing "rediscovered"!In these cases, an already existing derivation via the non-recursiverule has been confirmed by a new derivationinduced via the recursive rule:

Already before: Now in addition:

• Other "wrong" delta facts result from further "rediscoveries" arising as a consequenceof the first mistake:

p(2,4) ← e (2,3), p(3,4). p+(2,4) ← e (2,3), p+(3,4).Before: Now:

5.1


Avoiding Duplicates and Multiple Derivations: The Effectivity Test

• In each of these cases, the source of error was a constellation where duplicateshavebeen arising while computing (presumably new) derivations. In these cases, multiple derivationsof the same fact occurred due to either union or projection:

p(X,Y) ← e(X,Y).p(X,Y) ← e(X,Z), p(Z,Y). π

∪Duplicates

viaunion

Duplicatesvia

projection

• It is necessary to detect such multiple derivationsalready possible before the update, thus avoiding erroneous derivation of delta facts! This can be achieved by extendingthe delta rulesby means of a test for derivability in the old state, which determines whether each new delta fact is effectively new:

Effectivity test

• However, such an effectivity test is required only if the respective relation is either defined by more than one rule (union), or if the defining rule contains local variables (projection).

5.1


Positive Propagation Rules with Effectivity Test

e_cyclic+ ← s+(X).s+(X) ← p+(X,X).p+(X,Y) ← e+(X,Y), not p(X,Y).p+(X,Y) ← e+(X,Z), p(Z,Y), not p(X,Y).p+(X,Y) ← e (X,Z), p+(Z,Y), not p(X,Y).p+(X,Y) ← e+(X,Z), p+(Z,Y), not p(X,Y).

REffectivity test

U1: { + e(4,1) } !

p+(4,1)p+(4,2)p+(4,3)p+(4,4)

p+(3,1)p+(3,2) p+(3,3)s+(4)

p+(2,1)p+(2,2) s+(3)e_cyclic+

p+(1,1)s+(2)

s+(1)e+(4,1)

With the effectivity test, FPI delivers the correct result:

R∆+

p(1,2). p(1,3).p(2,3). p(2,4).p(3,4). p(1,4).

Now 'p' is needed during FPI, too, due to the effectivity test!

5.1



Working with Two Database States at the Same Time

• Dilemma: • Effectivity tests for insertionsto be performed in the "old" state of the DB

(before executing the resp. updates). • Effectivity tests for deletionsto be performed in the "new" state of the DB

(after updating the base facts).• If insertions and deletions are combined (as, e.g., in U3), both states are required

simultaneously.

How to work simultaneously with both states?

• Way out: Fortunately, the "missing" state can always be "simulated" via rules andderivation!

• Two variants can be imagined:• "Pessimistic" variant(expecting integrity violations as very likely):

Keep the old state and simulate the new state!• "Optimistic" variant(accepts the – low – risk of a rollback):

Perform changes ahead of integrity checking and simulate the new state!

5.1


Working with Two Database States (2)

F old : F new :

F − :F + :

U

• In set-theoretic terms, the new statecan be defined by the equation

Here F + contains all insertions via U, F − all deletions due to U.

• This equation is valid only if in addition the following assumptionsabout keepingchanges „under control“ are always satisfied:

• F + ∩ F old = ∅• F − ∩ F old = F −

• F + ∩ F − = ∅

F new = ( F old − F − ) ∪ F +

All updates are true.

Net effects are already computed.

5.1


Working with Two Database States (3)

F new = ( F old − F − ) ∪ F +

• The set equation on the previous slide forms the basis for a simulation of the new state from the old state and the "differentials" (F + , F − ), i.e., for the pessimistic approach:

• If prefering to work optimistically, one has to carry F old to the other side of the equation:

F old = ( F new − F + ) ∪ F −

• In the following, we will follow the pessimistic approach(F new simulated) and thus willomit the index old.

• "Simulating" means that first for each base relationthe new state is determined via aderived auxiliary relation, the definition of which can be obtained from the above equa-tion quite easily, e.g.,

enew(X,Y) ← e(X,Y), not e−(X,Y).enew(X,Y) ← e+(X,Y).

Transition rules

5.1


Negative Propagation Rules with Transition Rules: 1st Attempt


p−(X,Y) ← e−(X,Y), not pnew(X,Y).p−(X,Y) ← e−(X,Z), p(Z,Y), not pnew(X,Y).p−(X,Y) ← e (X,Z), p−(Z,Y), not pnew(X,Y).

R∆−

R

pnew(X,Y) ← p(X,Y), not p−(X,Y).pnew(X,Y) ← p+(X,Y).

Rτ

In the motivating example, the effectivity test just needs the new state of the derived relation p, not that of e. If trying to define pnew via transition rules defined as for e, thefollowing internal set of rules will be constructed:

It is immediately visible that the dependency graph of the combined rule set is no longerstratifiableand, moreover, that paradox derivationsare inevitable:

pnew(X,Y) p−(X,Y)not

not

5.1


Naive and Incremental Transition Rules

pnew(X,Y) ← enew(X,Y). pnew(X,Y) ← enew(X,Z), pnew(Z,Y).

• Fortunately, there is a different way of defining transition rulesfor derived relations: The new state of the derived relation (in the rule head) results from the new state of the relations referenced in the rule body!

• We call this – quite obvious – way of defining the new state "naive" transition rules.

• Only for base relations we will have to stick with the previous variant, which will becalled incremental transition rules(as there is no alternative):


p(X,Y) ← e(X,Y).p(X,Y) ← e(X,Z), p(Z,Y).

5.1


Using Incremental Transition Rules Solves the Stratifiability Problem



R∆−

R

Rτpnew(X,Y) ← enew(X,Y). pnew(X,Y) ← enew(X,Z), pnew(Z,Y).

pnew(X,Y) p−(X,Y)not

not

enew

Avoiding naive transition rules and using incremental ones instead solves the problem:

5.1


Transition and Propagation Rules: Missing Interdependency


R∆−

e_cyclic+ ← s+(X).s+(X) ← p+(X,X).p+(X,Y) ← e+(X,Y), not p(X,Y).p+(X,Y) ← e+(X,Z), p(Z,Y), not p(X,Y).p+(X,Y) ← e (X,Z), p+(Z,Y), not p(X,Y).p+(X,Y) ← e+(X,Z), p+(Z,Y), not p(X,Y).

R∆+

If combining positive andnegative propagation rules,. . .

. . .another problem pops up: Both rule sets don‘t reference each other at all!

However, we did observe such side effects in our example!

U3: { + e(4,1) ; − e(2,3) } !

U3 was constructed on purpose in such a way that deletions compensatethe conse-quences of insertions: Instead of 10 positive deltas (in case of U1) there were just4 positive deltas remaining if applying U3 !

?

5.1


Inappropriate Residual Evaluation when Propagating Insertions

p+(4,3) ← e+(4,1), p(1,3),not p(4,3).

• The "error in reasoning"causing this defect was to perform residual evaluation in the old state, thus failing to consider the consequences of deletions (which can be observedin the new state only):

U3: { + e(4,1) ; − e(2,3) } ! p−(2,3) p−(1,3)

• Instead, residual evaluation has to be performed over those facts only which "survive" in the new state:

p+ (X,Y) ← e+(X,Z), p(Z,Y), not p−(Z,Y) , not p(X,Y).

• Combining rules for simplicity‘s sake:

p+(X,Y) ← e+(X,Z), p+(Z,Y), not p(X,Y).

pnew(Z,Y)

5.1


Transition and Propagation Rules with Correct Residual Evaluation

p(X,Y) ← e(X,Y).p(X,Y) ← e(X,Z), p(Z,Y). e_cyclic+ ← s+(X).

s+(X) ← p+(X,X).p+(X,Y) ← e+(X,Y), not p(X,Y).p+(X,Y) ← e+(X,Z), pnew(Z,Y), not p(X,Y).p+(X,Y) ← enew(X,Z), p+(Z,Y), not p(X,Y).


pnew(X,Y) ← enew(X,Y).pnew(X,Y) ← enew(X,Z), pnew(Z,Y).


Rτ

R∆R'

U3: { + e(4,1) ; − e(2,3) } !

e+(4,1)e−(2,3)

p+(4,1) p+(4,2) p−(2,3)p−(2,4)

p+(3,1) p+(3,1) p−(1,3)p−(1,4)

5.1


Correct Usage of Old and New State: Summary

• Residual evaluationin a propagation rule is performed• in the newstate, if defining induced insertions,• in the old state, if defining induced deletions.

• Effectivity testson the other hand is performed• in the old state, if defining induced insertions,• in the newstate, if defining induced deletions.

• Residual evaluationis unnecessary, if the respective original rules are pure projectionor union rules (which means that bodies consists of one literal only).

• Effectivity testsare unnecessary, if the respective relations are defined by just onerule without local variables, i.e., correspond to intersection, join, product, or difference:

p(X) ← q(X). p(X) ← q(X,Y).p(X) ← r(X).

p(X) ← r(X), q(X). p(X,Y) ← q(X,Y), not r(X,Y).p(X,Y) ← r(X), q(Y). p(X,Y,Z) ← q(X,Y), r(Y,Z).

5.1


Rule Transformation in the Example: Current Status






Rτ

R∆R'

In our delta and transition rules we only need the p-rulesfrom the original rule set anymore, not s- and e_cyclic-rules: R'

Thus: Rupdate = R∆ ∪ Rτ ∪ R'

5.1


Dependency Graph of the Entire Internal Rule Set

p+ p−

pnew enew

e− e+

s+

e_cyclic+

p e

not

not

not

Possible seed relations

5.1


Dependency Graph with Stratification

p+p−

pnew

enew

e− e+

s+

e_cyclic+

p

e

not

not

not

Update propagation in this example requires at least two strata:

5.1


Update Propagation and Negation

p(X) ← q(X,Y), not r(Y).

q(1,2). r(1).q(1,3). r(3).q(2,3).

− r(3)

Fold:

p(1).

+ p(2).

• Update propagation "through" negative literalsis possible in a very similar wayas forpositive literals.

• Only (important) deviation: In the 3rd step (propagation) the inverted signis transferred to the induced update:

• Propagation rule (for deletions) in this example:

p+(X) ← q(X,Y), r −(Y), not p(X).

5.1


Update Propagation and Operators

p(X) ← q(X,Y), X > Y+1.

• Test literals(with or without arithmetic operators) may not serve as "entry points" forpropagating base updates:

− q(1,3)

• When evaluating such test literals it is only required to take care that a suitable SIPSis used during residual evaluation: Remember that test literals (like negative literals)may be evaluated in fully instantiated form only:

p(X) ← q(X,Y), X > Z+1, r(X,Z).

− q(1,3)

1 > 3+1

5.1


The Magic Updates Method

WS 2016/17



5.3

5.1 Foundations of Update Propagation5.2 Updates, Transactions and Constraints in Datalog5.3 The Magic Updates Method


Evaluation over Old and New State During Update Propagation

p+p−

pnew

enew

e− e+

s+

e_cyclic+

p

e

not

not

not

5.3


Problem of Materializing Irrelevant Facts Again

• For residual evaluation in case of insertions and for the effectivity testin case of dele-tionsp-facts in old and new stateare required. In order to do so, apparently all original p-rules have to be contained in the internal rule set (old p).

• Consequence: The entirep-relation (in old and new DB-state) iscompletely materializedduring eachpropagation process, even though onlyvery fewp-facts are actually required for propa-gating individual updates.

• Obvious idea – after thinking about this for a moment: Determine all subqueriesfor p- resp. pnew-facts occurring during propagationand just compute answers to these relevant subqueries! Do so via Magic Sets!

• Disadvantage:• Permanent "toggling" between global propagation and local subquery answering

required (complex control, nested FPIs)• Different subqueries will most likely recompute the same intermediate results:

"Sharing"answers would be desirable!

5.3


Sources and Goals During Update Propagation

p+p−

pnew

enew

e− e+

s+

e_cyclic+

p

e

not

not

not

p materialized

integrity constraint

5.3


• In the example, all seed facts occurring really caused"relevant" induced updatesto bedetermined as each propagation path leads either to a materialized relation or to anintegrity constraint (or to both).

• In general, however, just few propagation pathswill be relevant, as many induced updates will never "reach" a relationto be " monitored"due to integrity checking or materialization.

• As presented up till now, update propagation isnot goal-directed(even though incremental)!

• Again the problemresemblesthat of lacking goal-directionwhile producing facts byuncontrolled FPI duringquery evaluation.

Making Propagation Goal-Directed

− +

Materializedrelations

Integrityconstraints

5.3


Problem of Goal-Directed Propagation in the Example

p+p−

pnew

enew

e− e+

s+

e_cyclic+

p

e

not

not

not

Integrity constraint

Slightly modified example: p virtual rather than materialized!

Irrelevantpropagation

5.3


"On the Magic of Updates" (1)

• Solution for both problems (Efficient subquery computation and goal-directed propagation:Just one joint Magic Set transformation for all internal rules!

• The seed factsfor the joint MS-transformation result from coding all goalsof propagationas query facts:• For each materializedrelation p: query_ p+ and query_p−

(Insertions and deletions are both needed for adapting p.)• For each positiveintegrity constraint constraint c: query_c−

(Integrity violation indicated as soon as c turns false.)• For each negativeintegrity constraint constraint not c: query_c+

(Integrity violation indicated as soon as c turns true.)

• These query seedsare used permanentlyfor each update as codings of the propagationgoals – they are not exchanged from one case to the other (as was done for querying).

• In addition, however, we need ever changing update seedscoding the respective base data update to be propagated in each situation.

"Magic Updates" Method

5.3


"On the Magic of Updates" (2)

• The joint Magic Set transformationof the internal update rules for several queries at atime solves both problems outlined simultaneously:

• Update propagation is organized in a goal-directed mannerby coding all "goals" (i.e., integrity constraints and materialized relations) as queries. These queries "admit" only those induced updates as answers that are really needed in order tocheck integrity or adapt views.

• The subqueriesover old and new state arising during propagation "automatically" are taken care of, too, being dynamically generated(as query facts during FPI).They serve as filters for irrelevant facts during propagation.

• Crucial ideas:• StartFPI over the magic rules with severalquery seeds at the same time!• Combinequery and update seeds, controlling propagation both, "from above"

and "from below" at the same time!

• Obvious potential for optimization: Interrupt FPI as soon as any delta fact is generatedthat indicates an integrity violation!

5.3


Magic Updates Method : Architecture

R RupdateRule Compiler1

Fixpoint Iteration

F

U !

Transaction

Induced updatesand intermediate results

Rule Compiler2 Rupdate

magic

query seeds

update seeds

5.3


MU Method: Motivating Example (1)






Rτ

R∆R'

Query seeds: • p materialized: query_p+ , query_p−

• constraint not e_cyclic: query_e_cyclic+

5.3



answer_e_cyclic+ ← query_e_cyclic+, answer_s+(X).answer_s+(X) ← query_s+, answer_p+(X,X).answer_p+(X,Y) ← query_p+, e+(X,Y), not answer_p(X,Y).answer_p+(X,Y) ← query_p+, e+(X,Z), answer_pnew(Z,Y), notanswer_p(X,Y).answer_p+(X,Y) ← query_p+, enew(X,Z), answer_p+(Z,Y), notanswer_p(X,Y).. . .

R∆magic

query_s+ ← query_e_cyclic+.query_p+.query_p−.query_e_cyclic+.

query_p12(X,Y) ← query_p+, e+(X,Y).query_pnew1(Z) ← query_p+, e+(X,Z).query_p12(X,Y) ← query_p+, e+(X,Z), answer_pnew(Z,Y).query_p+1(Z) ← query_p+, enew(X,Z).query_p12(X,Y) ← query_p+, enew(X,Z), answer_p+(Z,Y).. . .

5.3



. . .answer_p−(X,Y) ← query_p−, e−(X,Y), not answer_pnew(X,Y).answer_p−(X,Y) ← query_p−, e−(X,Z), answer_p(Z,Y), notanswer_pnew(X,Y).answer_p−(X,Y) ← query_p−, e (X,Z), answer_p−(Z,Y), notanswer_pnew(X,Y).. . .

R∆magic

query_s+ ← query_e_cyclic+.query_p+.query_p−.query_e_cyclic+.

. . .query_p12(X,Y) ← query_p−, e−(X,Y).query_p1(Z) ← query_p−, e−(X,Z).query_pnew12(X,Y) ← query_p−, e−(X,Z), answer_p(Z,Y).query_p+1(Z) ← query_p−, e(X,Z).query_pnew12(X,Y) ← query_p−, e(X,Z), answer_p−(Z,Y).. . .

5.3



answer_pnew(X,Y) ← query_pnew12(X,Y), answer_enew(X,Y).answer_pnew(X,Y) ← query_pnew12(X,Y), answer_enew(X,Z), answer_pnew(Z,Y).answer_pnew(X,Y) ← query_pnew1(X), answer_enew(X,Y).answer_pnew(X,Y) ← query_pnew1(X), answer_enew(X,Z), answer_pnew(Z,Y).

answer_enew(X,Y) ← query_ enew12(X,Y), e(X,Y), not e−(X,Y).answer_enew(X,Y) ← query_ enew12(X,Y), e+(X,Y).answer_enew(X,Y) ← query_ enew1(X), e(X,Y), not e−(X,Y).answer_enew(X,Y) ← query_ enew1(X), e+(X,Y).

Rτmagic

. . .query_ enew12(X,Y) ← query_pnew12(X,Y).query_ enew1(X) ← query_pnew12(X,Y).query_ pnew12(X,Z) ← query_pnew12(X,Y), answer_enew(X,Z).query_ enew1(X) ← query_pnew1(X).query_ pnew12(X,Z) ← query_pnew1(X), answer_enew(X,Z).. . .

5.3



. . .answer_p(X,Y) ← query_p12(X,Y), e(X,Y).answer_p(X,Y) ← query_p12(X,Y), e(X,Z), answer_p(Z,Y).answer_p(X,Y) ← query_p1(X), e(X,Y).answer_p(X,Y) ← query_p1(X), e(X,Z), answer_p(Z,Y).

R' magic

. . .query_p1(Z) ← query_p12(X,Y), e(X,Z).query_p1(Z) ← query_p1(X), e(X,Z).

Altogether, the Magic Sets Transformation will generate . . .. . . 18 query-rules. . . 20 answer-rules.

It is certainly not reasonable to perform such transformations by hand anymore.Without machine supportthe large number of internal rules can‘t be managed reasonably.Currently, no alternative for efficient update propagation is at sight, however!

5.3



query_s+ ← query_e_cyclic+.query_p12(X,Y) ← query_p+, e+(X,Y)....query_enew1(X) ← query_pnew1(X).

e+(4,1).e−(2,3).query_p+.query_p−.query_e_cyclic+.

query_s+ query_p+1(3)query_p12(4,1) query_p+1(4)query_p12(2,3) query_p1(3)query_p+1(2) query_pnew1(1)

etc.

The correspondingFPI-processesaregetting too complexeven in small examples,too, at least for handcomputation.

Always relevant propaga-tion paths are „explored“,before any answer-facts are produced.

Update seeds

answer_e_cyclic+ ← query_e_cyclic+, s+(X).answer_s+(X) ← query_s+, answer_p+(X,X)....answer_p(X,Y) ← query_p1(X), e(X,Z), answer_p(Z,Y).

5.3


Magic Updates Methode: Overview

• The compilation phaseof the Magic Updates method consists of two parts:• First, the original rule set R is translated into an internal set of update rules Rupdate. • Then, the Magic Sets transformation relative to a set Qupdate of "query seeds" is

applied to Rupdate. The resulting internal rule set is called Rupdate .

• Rupdate consists of three disjoint subsets, . . .• . . . a set R∆ of propagation rules(delta rules) ,• . . . a set Rτ of transition rulesfor simulation relevant parts of the

newDB-state, and• . . . a subset R' of R for computing relevant parts of the old state only.

• In the iteration phasethe given transaction U is first coded as a set Uupdate of "update seeds". Then (depending on the resp. stratifiability of Rupdate ) the semantics of the deductive database

(Rupdate , F ∪ Qupdate ∪ Uupdate)

is determined by applying either IFP or CFP.

magic

magic

5.3

magic


Propagation Rules: Definition

R∆ is obtained from R by applying the following steps:

For each rule L ← L1, . . . , Ln in R generate 2 ∗ n propagation rulesof the format

Lsign1 ← L' isign2, L1 state1, . . . Li-1

state1, Li+1state1, . . . , Ln

state1, not Lstate2

with sign1 ∈ { +,−}, 1 ≤ i ≤ n and

• if L i is a negative literal not L'i :sign2 =def inverse(sign1)

• if L i is a positive literal:sign2 =def sign1L' i =def L i

• if sign1 = + : state1 =def new, state2 =def old• if sign1 = − : state1 =def old, state2 =def new

(Further optimizations of this basic format to be discussed in a moment!)

5.3


Transition Rules: Definition

• Rτ is obtained from (R, F)by applying the following steps:

• For each base relation r/n generate two (incremental) transition rules of the form:

rnew(X1, . . , Xn) ← r(X1, . . , Xn), not r−(X1, . . , Xn).rnew(X1, . . , Xn) ← r+(X1, . . , Xn).

• For each rule L ← L1, . . . , Ln ∈ R generate a (naive) transition rule of the form:

Lnew ← L1new, . . . , Ln

new

• R' contains all those rules from R that define relations referenced in R∆ or in Rτ.

5.3


Seed Facts for Update Propagation

• Qupdate is a set of query-literals composed as follows:• For each materializedrelation p: query_ p+ und query_p−

• For each positiveintegrity constraint constraint c: query_c−

• For each negativeintegrity constraint constraint not c: query_c+

• For a given transaction U = { U1; U2 ; . . . . ; Un }! the corresponding set Uupdate

is as follows:• For each fact L∈ Π + [U] (F) there is a delta fact L+ in Uupdate

.

• For each fact L∈ Π - [U] (F) there is a delta fact L− in Uupdate .

• This definition implies that update propagation always starts from true updates, the net effectof which has already been determined.

5.3


Propagation Rules: Optimizations

• In all propagation rules resulting from rules in R not defining any union or projectionrelations the effectivity test not Lstate2can be discarded.

• A derived relation is called a union relationif it is defined by more than one rule in R.

• A derived relation is called a projection relationif it is defined by exactly one rule in Rand if this rule contains at least local variable.

• In R∆ all those propagation rules can be cancelledwerden which define a relation . . .• . . . not occurring in Qupdate and . . .• . . . not referenced by a relation that occurs in Qupdate.

• In Rτ all those transition rules can be cancelled which are no longer referenced in R∆

after performing the previously mentioned reduction step.

• In addition, all usual optimization for MS-rules are applicable, too.

5.3


Update Propagation and Transaction Processing: Overall Scheduling

Update Propagation is embedded into the course of processing a transactionU as follows:

1. Determine the net effectof U, i.e., the sets Π + [U] (F) and Π − [U] (F) . 2. Determine from this the set Uupdate of current update seeds.

(All query seeds are assumed as being permanently pre-computed.) 3. Compute via suitable FPI the set F∆ of all delta facts resulting from these

seeds (i.e., perform update propagationproper)!4. If F∆ contains any fact c+, for which there is a constraint not c,

or any fact c−, for which there is a constraint c, then stop processing.(Steps 3 and 4 can be interleaved.)

5. Apply the transitionτ [U] (F) and then adapt all materialized relations according to F∆ .

5.3


Selected Topics

WS 2016/17

6. Selected Topics


6.1

6.1 View Updating


View Update Analysis vs. Update Propagation

∇+ p(1) !

+ q(1,2)

− t(1)

− r(2)

+ s(1,2)

∆

+ s(1,2) !

+ q(1,2)

+ p(1)

„Downward“ propagation „Upward“ propagation

+ w(1,3)

6.1

© 2017 Prof. Dr. R. Manthey

VU: A Bit of „Philosophy“

• View Updatingis another change-related topic of view management belonging intothe category of „Update-driven inference“ (like update-propagation).

• Simple formsof view updating have been included into the SQL standardsince rather early versions.

• However, most SQL DBMShave been implementing easy techniques of view updating only rather late (1990s), covering only those cases, where a uniqueandstraight forward realizationof a view update request can be automatically decidedby the DBMS.

• In the research literature(both in the SQL, as well as in the Datalog context moreambitious proposalshave been published since.

• The fixpoint-based approachoutlined in this chapter is „work in progress“ in thisdirection and currently just serves as a „future direction“ part of the lecture, notrelevant for the exam.


View Updating: Motivating Example (1)

Two base tables in an (SQL) database(MS Access used here as a DBMS) –both can be updated without problems.

6.1



CREATE VIEW MasterStudents AS(SELECT MatrNr, Name, FirstName, Sem, MailFROM StudentsWHERE Program ="MSc")

The view „MasterStudents“is updatable, i.e., changescan be directly „performed“in the tabular representationof the view (or via updatestatements).

Such view updates causechanges of the underlyingbase table „Students“ to beautomatically performedby the DBMS.

This is possible becausethere is a unique realizationof the view update in termsof a corresponding tableupdate.

6.1



CREATE VIEW AttentionExercises AS(SELECT Name, FirstName, May08, May15, May27FROM Exercises INNER JOIN Students

ON Student = MatrNr)

This second view, however, turns outto be non-updatable(at least initially)!Any attempt to modify the viewdirectly will be rejectedby anySQL DBMS.

6.1



This is the case, because both join attributes don‘thave the key propertyspecified in this design of the two tables to be joined.

Thus, there might be ambiguitieson how to realizea view update in terms of corresponding base tableupdates.

6.1



As soon as we designate the join attribute„Student“ in table „Exercises“ as a primary key, however, theview suddenly becomes updatable.

After this change of design there always is a unique way of „translating“ the view update into a base table update.

6.1


View Updating in Datalog and in SQL

• Up till now, we did not accept update statements referring to derived relations (views)in Datalog. In SQL, on the other hand, update statements directed towards views arelegalaccording to the standard (and are accepted by all major commercial DBMS).

• However, SQL is rather rigid wrt accepting or rejecting view updates – there is a cleardistinction between updatableand non-updatableviews.

• SQL insists on view updates having a unique translationinto a transaction of basetable updates „implementing“ (or: „realizing“) the view update request.

• In the standard, there is a rather elaborate and complicated set of properties views areexpected to satisfy in order to be updatable. This definition was rather limited till theSQL:1999 standard, but was extended then. Still it is not guaranteed to cover all caseswhere view updates have a unique realization!

• In this chapter, we will lift the „ban“ on view updates in Datalog, too. However, wewe will not distinguish between updatable and non-updatable views, but instead tryto analyze ways how to realize VU requests in case of ambiguous realization options.

6.1


Analysis of View Update Requests: Motivating Example (1)

b

c

d

e

f

g

l(X) r(X)

e(X,Y)


a h

Intended change: + p(a,h) !

6.1



c

d

f

g

l(X) r(X)

e(X,Y)


a h


Possible realization: + e(b,e) !

b e

6.1



b

d

e

g

l(X) r(X)

e(X,Y)

a h


Another realization : + e(c,f) !

c f

6.1




b

d

e

l(X) r(X)

e(X,Y)

a h


Yet another realization : { + e(c,g), + e(g,h)} !

c f

g

6.1


(but with greater „effort“ to change base data)



d

l(X) r(X)

e(X,Y)

a h


Yet another realization : { + e(b,e), + e(g,h)} !

c f

g

b e

6.1


(even less reasonable as there is a redundant step)



l(X) r(X)

e(X,Y)

a h


Realization : + e(d,f) !

c

g

b e

d

f

Side effects: + p(d,f) + p(a,f)+ p(d,g) + p(d,h)+ p(a,g)

6.1




b

c

d

e

f

g

l(X) r(X)

e(X,Y)

a h

constraint not c1with c1 ← e(a,Y), r(Y).

constraint not c2with c2 ← e(X,h), l(X).

constraint not c3with c3 ← e(X,Y), r(X), l(Y).

6.1




b

c

d

e

f

g

l(X) r(X)

e(X,Y)

a h

constraint not c1with c1 ← e(a,Y), r(Y).

constraint not c2with c2 ← e(X,h), l(X).

constraint not c3with c3 ← e(X,Y), r(X), l(Y).

Inconsistentrealizations: + e(a,h) !

+ e(d,h) !

{+ e(d,g), + e(g,c), + e(c,f) } !

6.1


The View Update Problem

• Given: View update request Uint

• Wanted: All realizations Uof Uint, such that

• the new state after execution of U is consistent again,(consistentrealizations)

• no proper subset of U is a consistent realization,(irreduciblerealizations)

• the number of side effects is minimal.(minimal side effectrealizations)

• The result of analyzing the VU request is a selection of meaningful alternativesfor realizing the request by means of a transaction of base data updates.

• The userwho issued the request is then asked to decidewhich option to use (if any).

• Furthermore, it is the responsibility of the userto ensure that the chosen realization corresponds with the change actually happening in reality.

?

6.1


Rule-Based Analysis of VU Requests: General Idea

• As the analysis of possible realizations of a view update request may result in alternativesit seems to be a good idea to represent such alternatives by disjunctive factsand tospecify the systematic analysis by means of disjunctive rules.

• In order to do so, internal relationsare introduced again intended to contain representationsof VU requests and of their potential realizations determined via derivation.

• For each relation p, there will be two „downward“-delta relations(in contrast to the „upward-delta relations introduced for update propagation):

p∇ p+

∇ p−

Insertion requests

Deletion requests

• Update commands for derived relations are represented in the corresponding ∇–relationsas „seed“-facts of a fixpoint-based inference process. Derived ∇-facts for derivedrelations represent derived VU requests. ∇-facts for baserelations represent realizations.

6.1


Disjunctive Datalog: Syntax

• In analogy to our „classical“ Datalog notation for conjunction (AND = comma): Semicolonas disjunction operator (OR)

• Till now: Disjunction only implicitly expressible via several rules defining the same relation.

• Now: Explicit disjunction in rule heads and facts is admissible, too (not in bodies still!). (Disjunctive Datalog)

∧ → ,∨ → ;

p(X) ← q(X).p(X) ← r(X).

p(X) ← q(X) ; r(X).

p(X) ; q(X) ← r(X).

r(a) ; r(b).

6.1


Disjunctive Datalog: Semantics

• Disjunctive rules applied to definite (non-disjunctive) facts:

p(X) ; q(Y) ← r(X,Y).

r(1,2). p(1) ; q(2).

• Definite rules applied to indefinite (disjunctive) facts:

p(X) ← r(X,Y).

r(1,2) ; q(3).p(1) ; q(3).

• Disjunctive rules applied to disjunctive facts:

p(X) ; q(Y) ← r(X,Y).

r(1,2) ; q(3).p(1) ; q(2) ; q(3).

6.1


Disjunctive Fixpoint Operator: Principle

s(X) ← q(X).t1(X) ; t2(X) ← p(X), r(X).

p(a).q(b) ; r(a).

R F

q(b) ; t1(a) ; t2(a). r(a) ; s(b).

s(b) ; t1(a) ; t2(a).

For simplicity‘s sake without formalization:disjunctive T-operator T∨

• Single step: Deriving (disjunctive) facts from (disjunctive) facts via (disjunctive) rules

• Generalization of the „normal“ T-operator• Special „clean up“ techniques „built in“:

Factorization, normalization, subsumption • Fixpoint semantics defined in analogy with Datalogpos

• Extension by negation in a similar way

T∨

T∨

T∨

6.1


Model Tree Representation of Disjunctive Fixpoints

s(X) ← q(X).t1(X) ; t2(X) ← p(X), r(X).

p(a).q(b) ; r(a).

R F

p(a)

q(b) r(a)

Two modelsof F("possible worlds")

s(b) t1(a) t2(a)

Three "possible worlds"of F*

M1 = {p(a), q(b), s(b)}M2 = {p(a), r(a), t1(a)}M3 = {p(a), r(a), t2(a)}

M1 M2 M3

6.1


Disjunctive Fixpoint vs. Model Tree

s(X) ← q(X).t1(X) ; t2(X) ← p(X), r(X).

R

p(a)

q(b) r(a)

s(b) t1(a) t2(a)

p(a).q(b) ; r(a).

F

q(b) ; t1(a) ; t2(a). r(a) ; s(b).

s(b) ; t1(a) ; t2(a).

T∨

T∨

T∨

6.1


Model Tree with Integrity Constraints

s(X) ← q(X).t1(X) ; t2(X) ← p(X), r(X).

p(a) ← true.q(b) ; r(a).

RF

p(a)

q(b) r(a)

s(b) t1(a) t2(a)

false← t1(a).constraint notfalse.

false

"Impossible world"

M1 = {p(a), q(b), s(b)}M2 = {p(a), r(a), t1(a)}M3 = {p(a), r(a), t2(a)}

true

6.1


A Rule-Based Approach to Analyzing VU Requests: Architecture

R R∇Disjunctiverule compiler

Disjunctivefixpoint iteration

F

Uint !Uint !

VU request

Realizations andsub-requests

Internal rules

seed

6.1


View Update Analysis vs. Update Propagation

∇+ p(1) !

+ q(1,2)

− t(1)

− r(2)

+ s(1,2)

∆

+ s(1,2) !

+ q(1,2)

+ p(1)

„Downward“ propagation „Upward“ propagation

+ w(1,3)

6.1


VU Analysis Rules (1)

∇ r+(X) ← ∇ p+(X), not r(X). ∇ q+(X) ← ∇ p+(X), not q(X).

p(X) ← q(X), r(X).

q(1).q(2). r(2).

r(3).p(2)

Rules for "extrapolating"insertion requests∇ p+(X):

∇ p+(1) is realizable by ∇ r+(1).∇ p+(3) is realizable by ∇ q+(3). ∇ p+(4) is realizable by ∇ r+(4) and∇ q+(4).

(Intersection, analogously product and join variants)

RequestRealization

(unique)

6.1



∇ q−(X) ; ∇ r−(X) ← ∇ p−(X).

p(X) ← q(X), r(X).

q(1).q(2). r(2).

r(3).p(2)

• For deletion requeststhere are alternativesfor a realization:

∇ p−(2) can be realized either by ∇ r−(2).or by ∇ q−(2) .

(or by both, but this kind of realization would be reducible)

(Intersection, analogously product and join variants)

• In disjunctive Datalog:

6.1




∇ q−(X) ; ∇ r−(X) ← ∇ p−(X).

p(X) ← q(X), r(X).

q(1).q(2). r(2).

r(3).p(2)

∇ p+(3).∇ p−(2).

Seed facts

∇ q+(3)

∇ q−(2) ∇ r−(2)

Combined insertion and deletion requestwith two realizations

6.1



∇ r− (X) ← ∇ p+(X), not r(X). ∇ q+(X) ← ∇ p+(X), not q(X).

∇ q−(X) ; ∇ r+(X) ← ∇ p−(X).

p(X) ← q(X), not r(X). (Difference)

"Top-down"-propagation of VU requests through negationleads to inverting signs

(as for "bottom-up"-propagation of base data updates)

q(1).q(2). r(2).

r(3).

p(1)

6.1



p(X) ← q(X)p(X) ← r(X).

(Union)

q(1).q(2). r(2).

r(3).

p(1)

∇ r−(X) ← ∇ p−(X), r(X). ∇ q−(X) ← ∇ p−(X), q(X).

∇ q+(X) ; ∇ r+(X) ← ∇ p+(X).

p(2)p(3)

Analysis rules for union are"dual" with those for intersection:


∇ q−(X) ; ∇ r−(X) ← ∇ p−(X).

6.1



p(X) ← q(X,Y). (Projection) q(1,2).q(1,3).q(2,3).

p(1)

∇ q−(X,Y) ← ∇ p−(X), q(X,Y).

p(2)

Deletions from a projection:

All q-facts „matching“ X must disappear!

Insertions into a projection :

∇ q+(X, c1) ; . . . ; ∇ q+(X, cn) ← ∇ p+(X). Some q-fact with fitting X has to begenerated – due to possible integrityviolations, each such fact is a candidate!

with domain(q,2) = {c1 , . . . , cn}

Finite domain forthe 2nd column of q

6.1


Combined Example for Analyzing View Update Requests

p(X) ← q1(X). q1(X) ← r(X), s(X). q2(X) ← t(X,Y).p(X) ← q2(X).

r(1). s(1).r(2). s(2).

s(4).t(1,2).t(1,3).

with domain(t,2) = {1 , 2, 3}

q1(1)q1(2)

q2(1)

p(2)p(1)

∇ q1−(X) ← ∇ p−(X), q1(X). ∇ q2−(X) ← ∇ p−(X), q2(X).

∇ q1+(X) ; ∇ q2+(X) ← ∇ p+(X).

∇ t−(X,Y) ← ∇ q2−(X), t(X,Y). ∇ t+(X, 1) ; ∇ t+(X, 2) ; ∇ t+(X, 3) ← ∇ q2+(X).

∇ r+(X) ← ∇ q1+(X), not r(X). ∇ s+(X) ← ∇ q1+(X), not s(X).

∇ r−(X) ; ∇ s−(X) ← ∇ q1−(X).

R∇

6.1


Combined Example (2)

r(1). s(1).r(2). s(2).

s(4).t(1,2).t(1,3).

q1(1)q1(2)

q2(1)

p(2)p(1)

∇ q1−(X) ← ∇ p−(X), q1(X). ∇ q2−(X) ← ∇ p−(X), q2(X). . . .

∇ t−(X,Y) ← ∇ q2−(X), t(X,Y). . . .

. . .∇ r−(X) ; ∇ s−(X) ← ∇ q1−(X).

∇ p−(1)

∇ q1−(1)∇ q2−(1)

∇ t−(1,2)∇ t−(1,3)

∇ r−(1) ∇ s−(1)

Base data changes able to realize theVU request

Intermediateresults

6.1



r(1). s(1).r(2). s(2).

s(4).t(1,2).t(1,3).

q1(1)q1(2)

q2(1)

p(2)p(1)

∇ q1−(X) ← ∇ p−(X), q1(X). ∇ q2−(X) ← ∇ p−(X), q2(X). . . .

∇ t−(X,Y) ← ∇ q2−(X), t(X,Y). . . .

. . .∇ r−(X) ; ∇ s−(X) ← ∇ q1−(X).

∇ p−(2)

∇ q1−(1)

∇ r−(1) ∇ s−(1)

6.1



r(1). s(1).r(2). s(2).

s(4).t(1,2).t(1,3).

q1(1)q1(2)

q2(1)

p(2)p(1)

. . .∇ q1+(X) ; ∇ q2+(X) ← ∇ p+(X).

. . .∇ t+(X, 1) ; ∇ t+(X, 2) ; ∇ t+(X, 3) ←

∇ q2+(X).

∇ r+(X) ← ∇ q1+(X), not r(X). ∇ s+(X) ← ∇ q1+(X), not s(X). . . .

∇ p+(4)

∇ q1+(1)

∇ r+(4)

∇ q2+(4)

∇ t+(4, 1) ∇ t+(4, 2) ∇ t+(4, 3) ∇ t+(4, 4)

6.1


View Updating and Integrity (1)

p(X) ← q1(X). q1(X) ← r1(X), s(X). p(X) ← q2(X). q2(X) ← r2(X), s(X). false ← q1(X), q2(X).

Modified example with „integrity rule“:

r2(2).Single fact:

∇ p+(2)

∇ q1+(2) ∇ q2+(2)

∇ r1+(2)∇ s+(2)

∇ s+(2)

M1 M2

Two realizations M1 and M2

But: State after performing realizationM1 is inconsistent!

r1(2).s(2).r2(2).

q1(2)

q2(2)false

constraint notfalse.

6.1



• Integrating integrity checkinginto the process of analyzing VU requests:• In each branch of the model tree do hypothetically „perform“ the proposed

realization and analyze the consequences of doing so by means of updatepropagation(integrity violations, but other side effects, too).

• Eliminate branches containing integrity violations!

• Combine both phases in a single fixpoint process:

• Extend the "downward"-delta rules R∇ by rules for update propagationas introduced in chapter 6 ("upward"-delta rules R∆ ).

• Notation: ∆ p+(X) instead of p+(X) and ∆ p−(X) instead of p−(X) • Place all R∆ -rules in higher stratathan all R∇ -rules

(Update propagation is performed after determining realizations)!• Combine both phases by "transfer rules"of the form

∆ p+(X) ← ∇ p+(X). for all base relations p (analoguously for deletions)!

• Close all inconsistent branchesby a rule false ← ∆ false+.

6.1



p(X) ← q1(X). q1(X) ← r1(X), s(X). p(X) ← q2(X). q2(X) ← r2(X), s(X). false ← q1(X), q2(X).

r2(2).

∇ p+(2)

∇ q1+(2) ∇ q2+(2)

∇ r1+(2)∇ s+(2)

∇ s+(2)

∆ r1+(2)∆ s+(2)

∆ s+(2) ∆ q1+(2)∆ q2+(2)

∆ q2+(2)

∆ false+∆ p+(2)

false

∆ p+(2)

A consistentrealization

What if theserequests wererealized?

Unrequestedside effects!

6.1



Structure of the fixpoint process is using both forms of delta rules:

∇

. . . . . ..

.

.

.

∆ ← ∇∇

∆

∇

∆

.

.

.

.

.

.

∆

hypothetical implementation

"top down"-propagation:Analyzing all possibilities

"bottom up"-propagation:Analyzing the consequences

6.1


Correctness Problem (1)

p(X) ← q1(X). q1(X) ← r1(X), s(X). p(X) ← q2(X). q2(X) ← r2(X), not s(X).

r1(1). s(1).r2(1).

q1(1). p(1).

∇ r1−(1) ∇ s−(1)

∇ p−(1)

∇ q1−(1)

∆ r1−(1) ∆ s−(1)

∆ q1−(1) ∆ q1−(1)∆ q2+(1)

∆ p−(1)

VU requestrealized

VU request not realized

Side effect

A further variant of the example:

6.1



∇ s−(1)

∇ p−(1)

∇ q1−(1)

∆ s−(1)

∆ q1−(1)∆ q2+(1)

Exclusion of branches not leading to any realization of a VU request can be achievedby adding integrity constraints:

false← ∇ p−(X), not ∆ p−(X).

.

.

.

false

Intended, but not realized!

6.1




r1(1). s(1).r2(1).

q1(1). p(1).

∇ s−(1)∇ r2−(1)

∇ p−(1)

∇ q1−(1)

∆ s−(1)∆ r2−(1)

∆ q1−(1)

∆ p−(1)

.

.

.

But: This form of "defense"against integrity violations istoo "radical" as the right branchcould be "repaired" by an addi-tional change causing another realization to be introduced!

?

Without further extensionsthe method would be neither correct nor complete !

6.1




r1(1). s(1).r2(1).

q1(1). p(1).

∇ s−(1)∇ r2−(1)

∇ p−(1)

∇ q1−(1)avoid q2(1)

∆ s−(1)∆ r2−(1)

∆ q1−(1)

∆ p−(1)

.

.

.

Idea for incorporating„repairs“:

1) Impose „constraints“avoid / preserve,

if „need for repair“might possibly arise:

2) Repair if violations ofthese „constraints“do actually occur:

avoid q2(X) ← ∇ p-(X), not q2(X).

∇ r 2(X) ← avoid q2(X), r2(X), ∇ s(X).

6.1


And so on, and so on . . . 6.1

This is more or less the current state of affairsin (our) research concerning view updating . . .

. . . more to come hopefully soon!

intelligent information systems - uni-bonn.de · © 2017 prof. dr. r. manthey intelligent...

Documents