dynamically time-capped possibilistic testing of subclassof …tettaman/k-cap2015-slides.pdf ·...

Introduction Principles Possibilistic Scoring Candidate Axiom Testing Subsumption Axiom Testing Experiments Conclusion

Dynamically Time-Capped Possibilistic Testing ofSubClassOf Axioms Against RDF Data to Enrich

Schemas

Andrea G. B. Tettamanzi, Catherine Faron-Zucker,and Fabien Gandon

Univ. Nice Sophia Antipolis, CNRS, Inria, I3S, UMR 7271, France

K-Cap 2015, Palisades, NY

1 / 25


Introduction: Ontology Learning

Top-down construction of ontologies has limitations

aprioristic and dogmatic

does not scale well

does not lend itself to a collaborative effort

Bottom-up, grass-roots approach to ontology and KB creation

start from RDF facts and learn OWL 2 axioms

Recent contributions towards OWL 2 ontology learning

FOIL-like algorithms for learning concept definitions

statistical schema induction via association rule mining

light-weight schema enrichment (DL-Learner framework)

All these methods apply and extend ILP techniques.

2 / 25


Introduction: Ontology validation, Axiom Scoring

Need for evaluating and validating ontologies

General methodological investigations, surveys

Tools like OOPS! for detecting pitfalls

Integrity constraint validation

Ontology learning and validation rely on axiom scoring

We have recently proposed a possibilistic scoring heuristic[A. Tettamanzi, C. Faron-Zucker, and F. Gandon. “Testing OWL Axioms

against RDF Facts: A possibilistic approach”, EKAW 2014]

Computationally heavy, but there is evidence that testing timetends to be inversely proportional to score

Research Question:

1 Can time capping alleviate the computation of the heuristicwithout giving up the precision of the scores?

3 / 25


Content

Content of an Axiom

Definition (Content of Axiom φ)

We define content(φ), as the finite set of formulas, which can betested against an RDF dataset K, constructed from theset-theoretic formulas expressing the direct OWL 2 semantics of φby grounding them.

E.g., φ = dbo:LaunchPad v dbo:Infrastructure

∀x ∈ ∆I , x ∈ dbo:LaunchPadI ⇒ x ∈ dbo:InfrastructureI

content(φ) = dbo:LaunchPad(r)⇒ dbo:Infrastructure(r) :r is a resource occurring in DBPedia

By construction, for all ψ ∈ content(φ), φ |= ψ.

4 / 25


Content

Confirmation and Counterexample of an Axiom

Given ψ ∈ content(φ) and an RDF dataset K, three cases:

1 K |= ψ: −→ ψ is a confirmation of φ;

2 K |= ¬ψ: −→ ψ is a counterexample of φ;

3 K 6|= ψ and K 6|= ¬ψ: −→ ψ is neither of the above

Selective confirmation: a ψ favoring φ rather than ¬φ.φ = Raven v Black −→ ψ = a black raven (vs. a green apple)

Idea

Restrict content(φ) just to those ψ which can be counterexamplesof φ. Leave out all ψ which would be trivial confirmations of φ.

5 / 25


Support, Confirmation, and Counterexample

Support, Confirmation, and Counterexample of an Axiom

Definition

Given axiom φ, let us define

uφ = ‖content(φ)‖u+φ = the number of confirmations of φ

u−φ = the number of counterexamples of φ

Some properties:

u+φ + u−φ ≤ uφ (there may be ψ s.t. K 6|= ψ and K 6|= ¬ψ)

u+φ = u−¬φ (confirmations of φ are counterexamples of ¬φ)

u−φ = u+¬φ (counterexamples of φ are confirmations of ¬φ)

uφ = u¬φ (φ and ¬φ have the same support)

6 / 25


Possibility Theory

Possibility Theory

Definition (Possibility Distribution)

π : Ω→ [0, 1]

Definition (Possibility and Necessity Measures)

Π(A) = maxω∈A

π(ω);

N(A) = 1− Π(A) = minω∈A1− π(ω).

For all subsets A ⊆ Ω,

1 Π(∅) = N(∅) = 0, Π(Ω) = N(Ω) = 1;2 Π(A) = 1− N(A) (duality);3 N(A) > 0 implies Π(A) = 1, Π(A) < 1 implies N(A) = 0.

In case of complete ignorance on A, Π(A) = Π(A) = 1.7 / 25


Possibility and Necessity of an Axiom


Π(φ) = 1−

√√√√1−

(uφ − u−φ

uφ

)2

N(φ) =

√√√√1−

(uφ − u+

φ

uφ

)2

if Π(φ) = 1, 0 otherwise.

0 20 40 60 80 100

0.0

0.2

0.4

0.6

0.8

1.0

no. of counterexamples

poss

ibili

ty

0 20 40 60 80 100

0.0

0.2

0.4

0.6

0.8

1.0

no. of confirmations

nece

ssity

8 / 25



Acceptance/Rejection Index

Combination of possibility and necessity of an axiom:

Definition

ARI(φ) = N(φ)− N(¬φ) = N(φ) + Π(φ)− 1

−1 ≤ ARI(φ) ≤ 1 for all axiom φ

ARI(φ) < 0 suggests rejection of φ (Π(φ) < 1)

ARI(φ) > 0 suggests acceptance of φ (N(φ) > 0)

ARI(φ) ≈ 0 reflects ignorance about the status of φ

9 / 25


OWL 2 → SPARQL

To test axioms, we define a mapping Q(E , x) from OWL 2expressions to SPARQL graph patterns such that

SELECT DISTINCT ?x WHERE Q(E , ?x)

returns [Q(E , x)], all known instances of class expression E and

ASK Q(E , a)

checks whether E (a) is in the RDF base.

For an atomic concept A (a valid IRI),

Q(A, ?x) = ?x a A .

10 / 25


Concept Negation: Q(¬C , ?x)

Problem

Open-world hypothesis, but no ¬ in RDF!

We approximate an open-world semantics as follows:

Q(¬C , ?x) = ?x a ?dc .

FILTER NOT EXISTS ?z a ?dc . Q(C , ?z)

(1)

For an atomic class expression A, this becomes

Q(¬A, ?x) = ?x a ?dc .

FILTER NOT EXISTS ?z a ?dc . ?z a A .

(2)

11 / 25


SubClassOf(C D) Axioms

To test SubClassOf axioms, we must define their logical contentbased on their OWL 2 semantics:

(C v D)I = CI ⊆ DI

≡ ∀x x ∈ CI ⇒ x ∈ DI

Therefore, following the principle of selective confirmation,

uCvD = ‖D(a) : K |= C (a)‖,

because, if C (a) holds,

C (a)⇒ D(a) ≡ ¬C (a) ∨ D(a) ≡ ⊥ ∨ D(a) ≡ D(a)

12 / 25


Support, Confirmations and Counterexamples of C v D

uCvD can be computed by

SELECT (count(DISTINCT ?x) AS ?u) WHERE Q(C , ?x).

As for the computational definition of u+CvD and u−CvD :

confirmations: a s.t. a ∈ [Q(C , x)] and a ∈ [Q(D, x)];

counterexamples: a s.t. a ∈ [Q(C , x)] and a ∈ [Q(¬D, x)].

Therefore,

u+CvD can be computed by

SELECT (count(DISTINCT ?x) AS ?numConfirmations)

WHERE Q(C , ?x) Q(D, ?x)

u−CvD can be computed by

SELECT (count(DISTINCT ?x) AS ?numCounterexamples)

WHERE Q(C , ?x) Q(¬D, ?x) 13 / 25


Test a SubClassOf axiom (plain version, w/o time cap)

Input: φ, an axiom of the form SubClassOf(C D);Output: Π(φ), N(φ), confirmations, counterexamples.1: Compute uφ using the corresponding SPARQL query;2: compute u+

φ using the corresponding SPARQL query;

3: if 0 < u+φ ≤ 100 then

4: query a list of confirmations;5: if u+

φ < uφ then

6: compute u−φ using the corresponding SPARQL query;

7: if 0 < u−φ ≤ 100 then8: query a list of counterexamples;9: else

10: u−φ ← 0;

11: compute Π(φ) and N(φ) based on uφ, u+φ , and u−φ .

14 / 25


Comparison with a Probability-Based Score

−1.0 −0.5 0.0 0.5 1.0

0.0

0.2

0.4

0.6

0.8

1.0


Bühm

ann a

nd L

ehm

ann’s

Score

15 / 25


Scalable Axiom Testing

T (φ) = O((1 + ARI(φ))−1

)or O (exp(−ARI(φ)))

−1.0 −0.5 0.0 0.5 1.0

020000

40000

60000


Ela

psed T

ime (

min

)

16 / 25



Time Predictor

How much time should we allow in order to be reasonably sure weare not throwing the baby out with the bathwater, while avoidingto waste time on hopeless tests?

Studying the elapsed times for accepted axioms, we observed thatthe time it takes to test C v D tends to be proportional to

TP(C v D) = uCvD · nicC ,

where nicC denotes the number of intersecting classes of C .A computational definition of nicC is the following SPARQL query:

SELECT (count(DISTINCT ?A) AS ?nic)

WHERE Q(C , ?x) ?x a ?A .

where A represents an atomic class expression.17 / 25



T (C v D)/TP(C v D)

−1.0 −0.5 0.0 0.5 1.01e−

09

1e−

07

1e−

05

1e−

03


Ela

psed T

ime to T

ime P

redic

tor

Ratio

18 / 25



TP(C v D) as a function of the cardinality rank of C

0 100 200 300 400

1e+

01

1e+

03

1e+

05

1e+

07

Class Rank

Tim

e P

redic

tor

19 / 25



Test a SubClassOf axiom (time-capped version)

Input: φ, an axiom of the form SubClassOf(C D);a, b, the coefficients of the linear time cap equation.

Output: Π(φ), N(φ), confirmations, counterexamples.1: Compute uφ and nic using the corresponding SPARQL queries;2: TP(φ)← uφ · nic;

3: compute u+φ using the corresponding SPARQL query;

4: if 0 < u+φ ≤ 100 then

5: query a list of confirmations;6: if u+

φ < uφ then

7: tmax(φ)← a + b · TP(φ)8: waiting up to tmax(φ) min do9: compute u−φ using the corresponding SPARQL query;

10: if time-out then11: u−φ ← uφ − u+

φ ;

12: else if 0 < u−φ ≤ 100 then

13: query a list of counterexamples;14: else15: u−φ ← 0;

16: compute Π(φ) and N(φ) based on uφ, u+φ , and u−φ .

20 / 25


Experiments

Experimental Setup:

DBpedia 3.9 in English as RDF fact repository

Local dump (812,546,748 RDF triples) loaded into Jena TDB

Method coded in Java, using Jena ARQ and TDB

12 6-core Intel Xeon CPUs @2.60GHz (15,360 KB cache), 128GB RAM, 4 TB HD (128 GB SSD cache), Ubuntu 64-bit OS.

Systematically generate and test SubClassOf axioms involvingatomic classes only

For each of the 442 classes C referred to in the RDF store

Construct all C v D : C and D share at least one instance

Test these axioms in increasing time-predictor order

Compare with scores obtained w/o time cap

21 / 25


Time-Capped Score vs. Probability-Based Score

−1.0 −0.5 0.0 0.5 1.0

0.0

0.2

0.4

0.6

0.8

1.0


Bühm

ann a

nd L

ehm

ann’s

Score

22 / 25


Results

Speed-up:

Testing 722 axioms w/o time cap took 290 days of cpu timeWe managed to test 5,050 axioms in < 342 h 30’ (244s/axiom) w/ time cap142-fold reduction in computing time

Precision loss due to time capping:

632 axioms were tested both w/ and w/o time capOutcome different on 25 of them: a 3.96% error rate

Absolute accuracy:

Comparison to a gold standard of DBpedia OntologySubClassOf axioms + SubClassOf axioms that can beinferred from themOf the 5,050 tested axioms, 1,915 occur in the gold standard327 (17%) get an ARI < 1/334 (1.78%) get an ARI < −1/3

23 / 25


Conclusions & Future Work

Contributions

Axiom scoring heuristics based on possibility theory

A framework based on the proposed heuristics

Time capping to reduce the computational overhead

Results

Experiments strongly support the validity of our hypothesis

142-fold speed-up with < 4% increase of error rate

Human evaluation suggests most axioms accepted by mistakeare inverted subsumptions or involve ill-defined concepts

Future work

Extend experimental evaluation to other types of axioms

Enlarge test base w/ additional RDF datasets from the LOD

24 / 25


The End

Thank you for your attention!

25 / 25

dynamically time-capped possibilistic testing of subclassof …tettaman/k-cap2015-slides.pdf ·...

Documents