negotiating over ontological correspondences with asymmetric and incomplete knowledge

Negotiating over Ontological Correspondences

with Asymmetric and Incomplete Knowledge

Terry Payne & Valentina TammaUniversity of Liverpool

[email protected]@liverpool.ac.uk

Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge

Terry Payne University of Liverpool

Open Systems, Ontologies and Alignment

• Agents can assume different ontological models• Modelled implicitly, or explicitly by defining entities (classes, roles etc), typically using

some logical theory, i.e. an Ontology

• Alignment Systems align similar ontologies

• If we assume that different alignments exist, how do agents choose which to use?

2

Alignment

Correspondence



Align Everything?

• What does the agent know?• Pre-computed alignments exist, and can be shared

• Different agents may possess different alignment fragments from different sources.

• Do we need everything to be aligned?• An agent may aggregate several ontologies for a variety of domains

• A task may be relevant to only a single module within an ontology

• Fragments of the ontological space may be confidential, or commercially sensitive.

3



Aims and Contribution• Correspondence Inclusion Dialogue (CID)

• Allows two agents to exchange knowledge about correspondences to agree upon a mutually acceptable final alignment AL.

• This alignment aligns only those entities in each agents’ working ontologies, without disclosing the ontologies, or all of the known correspondences.

• Assumptions1. Each agent knows about different correspondences from different sources

2. This knowledge is partial, and possibly ambiguous; i.e. more than one correspondence exists for a given entity

3. Agents associate a utility (Degree of Belief) κc to each unique correspondence

4. Correspondences with a joint utility below the admissibility threshold ϵ should be rejected

4



• Correspondences are modelled as Beliefs

• The degree of belief κc reflects the agent specific belief of the utility of c aligning its corresponding entities in the two working ontologies

• The beliefs of both agents for a correspondence can be combined into a joint belief

Modelling and Aggregating Beliefs

5

� = hc,ci,where correspondence c = he, e0,=i and e 2 W, e0 2 W 0

joint(c) =

8><

>:

avg(x

c

,

x̂

c

) if beliefs from both agents x and x̂ are disclosed

12 (

x̂

c

) if the beliefs of c is only known to x

avg(x

c

,

x

upper

) when x̂ ’s belief of c is not yet known



Knowledge Model

6

Working Ontology (a fragment of the agent’s ontology) contains the entities to be aligned

Alignment Store contains the correspondences known to the agent

Joint Belief Store tracks all beliefs

(commitments) disclosed by both

agents

As beliefs are shared, the agent

builds up a joint degree of belief of

the utility of the correspondence

Joint Belief Store JB

bob = hha, x,=i, 0.6i�

alice = hha, x,=i, 0.8i

�

alice = hhb, x,=i, 0.5i

bob = hhb, x,=i, 0.8i· · ·

Alignment Store �

Ontology O

joint(ha, x,=i) = 0.7joint(hb, x,=i) = 0.65�

alice = hhb, z,=i, 0.9i�

alice = hha, x,=i, 0.8i�

alice = hhb, w,=i, 0.6i�

alice = hhb, x,=i, 0.5i Alice

· · ·

W =

{a, b, a v b}

Public KnowledgePrivate Knowledge



Ambiguity and Attacks

7

publication article author

submittedPaper reviewedPaper paper editor

• Alignments typically consist of one-to-one mappings• Combining correspondences from different alignment fragments can

result in one-to-many correspondences; i.e. ambiguity

• Agents therefore need a mechanism for selecting the ‘best’ set of correspondences



Correspondence Inclusion Dialogue (CID)

• Inquiry Dialogue, use to determine mutually acceptable correspondences to facilitate some task

• Agents take turns to select and propose a belief they know of, that has not yet been asserted, based on its utility κc

• A shared, or asserted correspondence is:

• accepted based on their combined κc (i.e. joint(c))

• rejected if joint(c) < ϵ, the admissibility threshold • objected to if an agent believes a better correspondence exists for

one of the entities in the correspondence

• The dialogue is presented formally in the paper

8



Commitment Strategy

• Correspondences are selected (asserted) for disclosure:

1. Agents assert the best undisclosed correspondence (i.e. with the highest κc) in any round

2. Disclosed correspondences should be grounded in the working ontology

3. A correspondence should not be disclosed if the joint degree of belief is guaranteed to be less than the admissibility threshold.

9



Objecting Strategy

10

… 2 Alice -> Bob:ASSERT <0.9, {http://l#b = http://h#z}> 3 Bob -> Alice:OBJECT <0.8, {http://l#b = http://h#x}> to <0.0, {http://l#b = http://h#z}> 4 Alice -> Bob:OBJECT <0.8, {http://l#a = http://h#x}> to <0.5, {http://l#b = http://h#x}> 5 Bob -> Alice:ACCEPT <0.6, {http://l#a = http://h#x}> 6 Bob -> Alice:OBJECT <0.4, {http://l#b = http://h#w}> to <0.0, {http://l#b = http://h#z}> 7 Alice -> Bob:ACCEPT <0.6, {http://l#b = http://h#w}> …

a b c

z w x y

• Alternate correspondences are counter-proposed (through an objection):

1. If an agent has an undisclosed correspondence that shares an entity and:

2. It’s estimated κc is greater than the joint degree of belief for the current correspondence

• Each objection results in the agent’s actual degree of belief being disclosed.



Example DialoguePublic Knowledge Private Knowledge

Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�

bob = hhb, x,=i, 0.8i�

bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i



Alignment Store �

�alice = hhb, z,=i, 0.9i�


�

alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i

W = {a, b, c,a v b}

Ontology O Joint Belief Store JB

Alice asserts the correspondence with the highest κc




Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�




alice = hhb, z,=i, 0.9i


Alignment Store �



�


W = {a, b, c,a v b}


hAlice, assert, hhb, z,=i, 0.9i, nili

�alice = hhb, z,=i, 0.9i

Alice asserts the correspondence with the highest κc




Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�






Alignment Store �



�


W = {a, b, c,a v b}




Bob realises that joint(⟨b,z,=⟩) = 0.45, and finds an alternative c with jointest(⟨b,x,=⟩) = 0.85




Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�






Alignment Store �



�


W = {a, b, c,a v b}




joint(hb, z,=i) = 0.45

hBob, object, hhb, x,=i, 0.8i, hhb, z,=i, 0.0ii

bob = hhb, z,=i, 0.0i�

bob = hhb, x,=i, 0.8i�bob = hhb, z,=i, 0.0i


bob = hhb, x,=i, 0.8iUsing the Upper Bound

Alice’s previous assertion was a belief with a utility of 0.9. Therefore the

utility of ⟨b,x,=⟩ ≤ 0.9.As Bob’s ⟨b,x,=⟩ is 0.8, he calculates

jointest(⟨b,x,=⟩) = avg(0.9, 0.8)




Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�






Alignment Store �



�


W = {a, b, c,a v b}









bob = hhb, x,=i, 0.8i





Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�






Alignment Store �



�


W = {a, b, c,a v b}








Alice calculates joint(⟨b,x,=⟩) is 0.65. As jointest(⟨a,x,=⟩) is 0.9, which is greater than joint(⟨b,x,=⟩), she objects.



hAlice, object, hha, x,=i, 0.8i, hhb, x,=i, 0.5ii

joint(hb, x,=i) = 0.65




Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�






Alignment Store �



�


W = {a, b, c,a v b}








Bob calculates joint(⟨a,x,=⟩) is 0.7. As he has no further objections, he accepts this.





�

alice = hhb, x,=i, 0.5i�


hBob, accepts, hha, x,=i, 0.6ii




joint(ha, x,=i) = 0.7




Ontology O

W = {w, x, y, z,x v w}

Alignment Store �

�






Alignment Store �



�


W = {a, b, c,a v b}








The agents continue until they have no more correspondences to consider.





�

alice = hhb, x,=i, 0.5i�


hBob, accepts, hha, x,=i, 0.6ii

�

bob = hha, x,=i, 0.6i




bob = hha, x,=i, 0.6i

joint(ha, x,=i) = 0.7joint(ha, x,=i) = 0.7

joint(hb, w,=i) = 0.5 joint(hb, w,=i) = 0.5

hBob, object, hhb, w,=i, 0.4i, hhb, z,=i, 0.0iihAlice, accepts, hhb, w,=i, 0.6ii· · ·

bob = hhb, w,=i, 0.4i�alice = hhb, w,=i, 0.6i

�bob = hhb, w,=i, 0.4i alice = hhb, w,=i, 0.6i



Modelling the objections as an attack graph

• The objections can form a Dungian attack graph• Attacks can be directed by the difference in the degree of belief

of each correspondence • Bi-directional attacks are resolved by random selection of one of the

alternatives

• Can then use grounded semantics to determine the extension

16

⟨a,x,=⟩ 0.7

⟨b,x,=⟩ 0.65

⟨b,w,=⟩ 0.5

⟨b,z,=⟩ 0.45



Empirical Analysis: Datasets

• Datasets taken from the Ontology Alignment Evaluation Initiative (2012) Conference Track• 7 Ontologies, each describing the conference domain

• Each included a reference ontology, used by OAEI to evaluate alignment performance.

• 17 Alignment systems used to generate pairwise alignments between the ontologies • 21 pairwise alignments tested

• 357 alignments in total

17



Experimental Method• Evaluate alignment generated through two agents

negotiating• Each agent starts with 8 of 17 randomly selected alignments

form the OAEI repository.

• Degree of Belief κc based on the probability of a correspondence appearing in the alignments

• Admissibility thresholds ϵ were varied between 1/16 (no filtering) to 1 in 1/16ths

• Each experiment was run 500 times for statistical significance

18



Hypothesis 1: Are we fit for purpose?

• Compare each alignment AL generated through dialogue with OAEI reference ontology

• Performance evaluated using Precision, Recall and F-measure

• Baseline result based on selecting a single alignment at random

19



Hypothesis 1: Are we fit for purpose?

20

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

2/16 4/16 6/16 8/16 10/16 12/16 14/16 16/16Diffe

renc

e in

F-m

easu

re fr

om R

ando

m A

lignm

ent A

vera

ge

Admissibility Threshold

Delta f-measure performance for all 21 Ontology Pairs

F-measure for each ontology pair found to be statistically higher than baseline for 3/16 ≤ ϵ < 14/16



Hypothesis 2: Can filtering out correspondences help?

• Calculate percentage of unique mappings disclosed/messages exchanged during the negotiation.

• Results averaged across all ontology pairs

• Evaluate the significance of the upper-bound κupper in the strategy

• Determine the percent of correspondences in final alignment AL

21



Hypothesis 2: Can filtering out correspondences help?

22

0

20

40

60

80

100

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Perc

enta

ge o

f Com

bine

d U

niqu

e C

orre

spon

denc

es

Admissibility Threshold

Percentage of Correspondences Disclosed or in the Final Alignment

% in final alignment% disclosed (using the upper boud)

% disclosed (no upper bound)

jointest(c) ≈ 0.5 when κupper is fixed (i.e. not used) and ϵ ≤ 9/16

the number of ambiguous correspondences falls as ϵ increases

When ϵ > 9/16, low κc correspondences are filtered



Conclusions• Developed a formal Inquiry Dialogue that supports the

sharing of ontological correspondences between agents• Only those correspondences relating to the agents working ontology

are aligned, thus avoiding unnecessary alignment

• Implemented a full version of the dialogue for evaluation• The resulting alignment performs significantly better in most cases

than the average performance of other approaches, when tested with a reference alignment

• By modelling the opponent’s assertions, an agent can more accurately estimate the joint utility of the correspondence, resulting in an exponential drop in correspondences disclosed and messages exchanged as the threshold increases

23

negotiating over ontological correspondences with asymmetric and incomplete knowledge

Science